2024-04-24T07:57:39.915 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T07:57:39.915 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212 --name rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/orig.config.yaml 2024-04-24T07:57:39.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T07:57:40.052 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212 branch: rishabh-main-apr17-a654945 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: null first_in_suite: false job_id: '7671212' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-main-apr17-a654945 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-main-apr17-a654945 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7954 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: rishabh-main-apr17-a654945 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJm27NWkUvn7FQKvtuY2zOrYZnGYdgvsb5QaGmZ9d01nF2rRjsV4+2BTYnsmMe64icBRQBzRTVRW62YR2f4blA= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCIplYqbcIpOMlShY3VrL4gBfN3by1EWstzcyolU16HgJ9GbAurPtVbxOL15MErBMhmoon1wYFD3LnY0zhrPdh4= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_07:32:23 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-24T07:57:40.052 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa; will attempt to use it 2024-04-24T07:57:40.052 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks 2024-04-24T07:57:40.053 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T07:57:40.054 INFO:teuthology.task.internal:Checking packages... 2024-04-24T07:57:40.077 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1f81fb397ae98da7563d451a78c61574c8f4e6e0' 2024-04-24T07:57:40.078 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T07:57:40.078 INFO:teuthology.packaging:ref: None 2024-04-24T07:57:40.078 INFO:teuthology.packaging:tag: None 2024-04-24T07:57:40.078 INFO:teuthology.packaging:branch: rishabh-main-apr17-a654945 2024-04-24T07:57:40.078 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T07:57:40.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-main-apr17-a654945 2024-04-24T07:57:40.221 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3116.g1f81fb39 2024-04-24T07:57:40.222 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T07:57:40.229 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T07:57:40.229 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T07:57:40.243 INFO:teuthology.task.internal:Saving configuration 2024-04-24T07:57:40.262 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T07:57:40.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T07:57:40.295 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 07:51:58.796811', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJm27NWkUvn7FQKvtuY2zOrYZnGYdgvsb5QaGmZ9d01nF2rRjsV4+2BTYnsmMe64icBRQBzRTVRW62YR2f4blA='} 2024-04-24T07:57:40.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 07:51:58.798781', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCIplYqbcIpOMlShY3VrL4gBfN3by1EWstzcyolU16HgJ9GbAurPtVbxOL15MErBMhmoon1wYFD3LnY0zhrPdh4='} 2024-04-24T07:57:40.316 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T07:57:40.323 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-24T07:57:40.323 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-24T07:57:40.323 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T07:57:40.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa072932160>, signals=[15]) 2024-04-24T07:57:40.399 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T07:57:40.405 INFO:teuthology.task.internal:Opening connections... 2024-04-24T07:57:40.405 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T07:57:40.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T07:57:40.487 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-04-24T07:57:40.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T07:57:40.563 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T07:57:40.592 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-24T07:57:40.619 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-24T07:57:40.619 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T07:57:40.685 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-24T07:57:40.713 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-04-24T07:57:40.733 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-04-24T07:57:40.734 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-04-24T07:57:40.797 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-04-24T07:57:40.797 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-04-24T07:57:40.797 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-04-24T07:57:40.797 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T07:57:40.798 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T07:57:40.799 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-04-24T07:57:40.981 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T07:57:41.007 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T07:57:41.014 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T07:57:41.014 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-04-24T07:57:41.018 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-04-24T07:57:41.034 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T07:57:41.050 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T07:57:41.050 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-04-24T07:57:41.075 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-04-24T07:57:41.188 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T07:57:41.245 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T07:57:41.260 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T07:57:41.260 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-24T07:57:41.260 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T07:57:41.261 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T07:57:41.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T07:57:41.277 DEBUG:teuthology.task.kernel:sha1 for host.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.278 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T07:57:41.278 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-24T07:57:41.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T07:57:41.279 DEBUG:teuthology.task.kernel:sha1 for host.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.279 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T07:57:41.279 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-04-24T07:57:41.334 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-04-24T07:57:41.335 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.335 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T07:57:41.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.339 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-437.el9.x86_64 2024-04-24T07:57:41.339 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.340 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T07:57:41.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:57:41.478 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-04-24T07:57:41.493 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum upgrade -y linux-firmware 2024-04-24T07:57:45.954 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-04-24T07:57:47.358 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-04-24T07:57:50.509 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 19 MB 00:02 2024-04-24T07:57:55.968 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-04-24T07:57:59.969 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-24T07:58:01.460 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T07:58:01.919 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 808 kB/s | 6.2 MB 00:07 2024-04-24T07:58:03.070 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T07:58:03.587 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-24T07:58:04.897 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T07:58:08.178 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T07:58:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-24T07:58:10.167 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T07:58:10.168 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T07:58:10.169 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T07:58:10.289 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.b... 2024-04-24T07:58:10.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:58:10.442 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T07:58:10.443 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T07:58:10.445 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi149.front.sepia.ceph.com to complete... 2024-04-24T07:58:11.877 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T07:58:11.878 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T07:58:11.878 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T07:58:11.996 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.a... 2024-04-24T07:58:11.996 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T07:58:12.138 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T07:58:12.139 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T07:58:12.141 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-04-24T07:58:24.918 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24 07:58:24 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T07:58:24.918 INFO:teuthology.orchestra.run.smithi149.stderr:FINISHED --2024-04-24 07:58:24-- 2024-04-24T07:58:24.918 INFO:teuthology.orchestra.run.smithi149.stderr:Total wall clock time: 14s 2024-04-24T07:58:24.918 INFO:teuthology.orchestra.run.smithi149.stderr:Downloaded: 1 files, 341M in 14s (24.1 MB/s) 2024-04-24T07:58:24.921 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.b... 2024-04-24T07:58:24.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T07:58:27.021 INFO:teuthology.orchestra.run.smithi149.stdout:Verifying... ######################################## 2024-04-24T07:58:27.281 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing... ######################################## 2024-04-24T07:58:27.396 INFO:teuthology.orchestra.run.smithi149.stdout:Updating / installing... 2024-04-24T07:58:34.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24 07:58:34 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T07:58:34.505 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-04-24 07:58:34-- 2024-04-24T07:58:34.506 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 22s 2024-04-24T07:58:34.506 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 341M in 22s (15.6 MB/s) 2024-04-24T07:58:34.509 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.a... 2024-04-24T07:58:34.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T07:58:35.512 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T07:58:36.623 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-04-24T07:58:36.772 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-04-24T07:58:36.892 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-04-24T07:58:44.829 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T07:59:12.089 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T07:59:12.089 DEBUG:teuthology.orchestra.run.smithi149:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T07:59:12.157 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T07:59:12.158 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T07:59:12.159 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T07:59:12.160 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T07:59:12.161 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T07:59:12.162 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T07:59:12.163 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T07:59:12.164 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T07:59:12.165 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T07:59:12.166 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T07:59:12.167 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T07:59:12.168 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T07:59:12.168 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T07:59:12.168 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T07:59:12.168 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T07:59:12.168 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T07:59:12.169 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T07:59:12.170 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T07:59:12.171 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T07:59:12.172 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T07:59:12.173 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T07:59:12.174 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T07:59:12.175 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T07:59:12.176 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T07:59:12.177 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T07:59:12.178 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T07:59:12.179 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T07:59:12.180 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T07:59:12.181 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T07:59:12.182 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T07:59:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T07:59:12.184 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T07:59:12.185 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T07:59:12.186 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T07:59:12.187 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T07:59:12.188 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T07:59:12.189 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T07:59:12.190 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T07:59:12.191 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T07:59:12.192 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T07:59:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T07:59:12.194 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T07:59:12.195 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T07:59:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T07:59:12.197 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T07:59:12.198 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T07:59:12.199 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T07:59:12.200 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T07:59:12.201 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T07:59:12.202 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T07:59:12.203 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T07:59:12.204 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T07:59:12.205 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T07:59:12.206 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T07:59:12.207 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T07:59:12.208 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T07:59:12.209 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T07:59:12.210 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T07:59:12.211 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T07:59:12.212 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T07:59:12.213 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T07:59:12.214 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T07:59:12.215 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T07:59:12.216 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T07:59:12.217 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T07:59:12.218 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T07:59:12.219 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T07:59:12.220 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T07:59:12.221 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T07:59:12.222 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T07:59:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T07:59:12.224 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T07:59:12.225 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T07:59:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T07:59:12.227 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T07:59:12.228 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T07:59:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T07:59:12.230 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T07:59:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T07:59:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T07:59:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T07:59:12.234 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T07:59:12.238 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T07:59:12.239 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T07:59:12.240 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T07:59:12.241 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T07:59:12.242 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T07:59:12.243 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T07:59:12.244 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T07:59:12.245 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T07:59:12.246 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T07:59:12.247 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T07:59:12.248 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T07:59:12.249 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T07:59:12.250 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T07:59:12.251 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T07:59:12.252 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T07:59:12.253 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T07:59:12.254 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T07:59:12.255 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T07:59:12.256 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T07:59:12.257 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T07:59:12.258 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T07:59:12.259 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T07:59:12.260 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T07:59:12.261 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T07:59:12.262 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T07:59:12.263 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T07:59:12.264 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T07:59:12.265 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T07:59:12.266 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T07:59:12.267 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T07:59:12.268 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T07:59:12.269 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T07:59:12.270 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T07:59:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T07:59:12.272 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T07:59:12.273 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T07:59:12.274 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T07:59:12.275 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T07:59:12.276 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T07:59:12.277 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T07:59:12.278 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T07:59:12.279 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T07:59:12.280 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T07:59:12.281 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T07:59:12.282 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T07:59:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T07:59:12.284 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T07:59:12.285 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T07:59:12.286 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T07:59:12.287 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T07:59:12.288 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T07:59:12.289 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T07:59:12.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T07:59:12.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T07:59:12.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T07:59:12.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T07:59:12.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T07:59:12.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T07:59:12.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T07:59:12.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T07:59:12.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T07:59:12.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T07:59:12.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T07:59:12.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T07:59:12.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T07:59:12.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T07:59:12.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T07:59:12.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T07:59:12.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T07:59:12.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T07:59:12.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T07:59:12.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T07:59:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T07:59:12.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T07:59:12.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T07:59:12.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T07:59:12.317 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.318 DEBUG:teuthology.orchestra.run.smithi149:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:for file in vmlinuz System.map config; do 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout: fi 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:done 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T07:59:12.341 INFO:teuthology.orchestra.run.smithi149.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.342 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-04-24T07:59:12.343 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-04-24T07:59:12.416 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-04-24T07:59:12.416 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-04-24T07:59:12.416 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-04-24T07:59:12.416 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-04-24T07:59:12.416 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-04-24T07:59:12.417 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T07:59:12.418 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T07:59:12.419 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T07:59:12.419 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:12.419 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T07:59:12.977 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-04-24T07:59:14.272 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T07:59:14.303 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-04-24T07:59:14.306 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-04-24T07:59:14.342 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.xXHcEPp2pG 2024-04-24T07:59:14.342 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xXHcEPp2pG 2024-04-24T07:59:14.421 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.xXHcEPp2pG 2024-04-24T07:59:14.556 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.xXHcEPp2pG is 6KB 2024-04-24T07:59:14.603 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.xXHcEPp2pG 2024-04-24T07:59:14.617 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T07:59:14.679 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T07:59:14.679 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T07:59:14.679 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T07:59:14.679 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T07:59:14.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:14.855 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-04-24T07:59:28.009 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T07:59:28.010 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T07:59:28.099 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.099 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T07:59:28.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T07:59:28.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T07:59:28.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T07:59:28.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T07:59:28.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T07:59:28.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T07:59:28.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T07:59:28.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T07:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T07:59:28.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T07:59:28.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T07:59:28.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T07:59:28.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T07:59:28.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T07:59:28.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T07:59:28.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T07:59:28.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T07:59:28.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T07:59:28.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T07:59:28.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T07:59:28.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T07:59:28.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T07:59:28.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T07:59:28.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T07:59:28.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T07:59:28.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T07:59:28.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T07:59:28.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T07:59:28.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T07:59:28.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T07:59:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T07:59:28.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T07:59:28.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T07:59:28.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T07:59:28.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T07:59:28.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T07:59:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T07:59:28.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T07:59:28.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T07:59:28.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T07:59:28.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T07:59:28.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T07:59:28.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T07:59:28.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T07:59:28.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T07:59:28.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T07:59:28.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T07:59:28.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T07:59:28.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T07:59:28.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T07:59:28.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T07:59:28.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T07:59:28.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T07:59:28.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T07:59:28.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T07:59:28.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T07:59:28.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T07:59:28.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T07:59:28.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T07:59:28.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T07:59:28.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T07:59:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T07:59:28.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T07:59:28.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T07:59:28.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T07:59:28.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T07:59:28.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T07:59:28.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T07:59:28.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T07:59:28.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T07:59:28.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T07:59:28.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T07:59:28.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T07:59:28.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T07:59:28.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T07:59:28.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T07:59:28.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T07:59:28.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T07:59:28.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T07:59:28.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T07:59:28.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T07:59:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T07:59:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T07:59:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T07:59:28.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T07:59:28.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T07:59:28.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T07:59:28.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T07:59:28.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T07:59:28.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T07:59:28.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T07:59:28.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T07:59:28.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T07:59:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T07:59:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T07:59:28.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T07:59:28.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T07:59:28.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T07:59:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T07:59:28.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T07:59:28.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T07:59:28.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T07:59:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T07:59:28.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T07:59:28.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T07:59:28.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T07:59:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T07:59:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T07:59:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T07:59:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T07:59:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T07:59:28.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T07:59:28.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T07:59:28.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T07:59:28.250 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.250 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi090.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T07:59:28.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-04-24T07:59:28.350 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-04-24T07:59:28.350 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-04-24T07:59:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T07:59:28.352 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T07:59:28.353 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T07:59:28.353 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:28.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T07:59:28.925 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-04-24T07:59:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T07:59:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-04-24T07:59:30.727 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-04-24T07:59:30.745 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.UpbAk7SFa9 2024-04-24T07:59:30.745 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UpbAk7SFa9 2024-04-24T07:59:30.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.UpbAk7SFa9 2024-04-24T07:59:30.961 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.UpbAk7SFa9 is 6KB 2024-04-24T07:59:31.013 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.UpbAk7SFa9 2024-04-24T07:59:31.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T07:59:31.100 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T07:59:31.100 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T07:59:31.101 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T07:59:31.101 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T07:59:31.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T07:59:31.254 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-04-24T07:59:44.870 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-24T07:59:44.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T07:59:44.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:01.270 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-24T08:00:01.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:00:01.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:03.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-04-24T08:00:12.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:00:12.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:15.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-04-24T08:00:19.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T08:00:27.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:00:27.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:28.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:00:28.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:31.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T08:00:43.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:00:43.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:00:46.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T08:01:01.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:01:01.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:01:27.491 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:01:42.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:01:42.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:01:42.893 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-04-24T08:01:43.546 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:01:43.546 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:01:43.546 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-04-24T08:01:43.599 INFO:teuthology.orchestra.run.smithi149.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:01:43.600 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:01:43.600 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:01:43.600 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:01:44.600 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-24T08:01:44.601 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:01:44.745 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-04-24T08:01:44.794 DEBUG:teuthology.parallel:result is None 2024-04-24T08:02:01.894 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:02:19.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:02:19.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:02:20.423 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-24T08:02:21.134 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:02:21.134 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:02:21.135 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-24T08:02:21.189 INFO:teuthology.orchestra.run.smithi090.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:02:21.189 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:02:21.189 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:02:21.190 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:02:22.190 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-24T08:02:22.191 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:02:22.348 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-04-24T08:02:22.428 DEBUG:teuthology.parallel:result is None 2024-04-24T08:02:22.428 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T08:02:22.439 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T08:02:22.439 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:02:22.447 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:02:22.503 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T08:02:22.510 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T08:02:22.519 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T08:02:22.519 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:02:22.524 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:02:22.595 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T08:02:22.602 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T08:02:22.602 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:02:22.609 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:02:22.693 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:02:22.713 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:02:22.723 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:02:22.745 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:02:22.750 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T08:02:22.757 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T08:02:22.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:02:22.764 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:02:22.851 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T08:02:22.864 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T08:02:22.864 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:02:22.871 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:02:22.925 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:02:23.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:02:23.121 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:02:23.205 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:02:23.288 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:02:23.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:02:23.373 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:02:23.459 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:02:23.549 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:02:23.631 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:02:23.716 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:02:23.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:02:23.798 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-04-24T08:02:23.806 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-04-24T08:02:23.916 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:02:23.958 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:02:24.378 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T08:02:24.386 INFO:teuthology.task.internal:Starting timer... 2024-04-24T08:02:24.387 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T08:02:24.397 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T08:02:24.407 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T08:02:24.408 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-04-24T08:02:24.526 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-04-24T08:02:24.531 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-04-24T08:02:24.687 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-04-24T08:02:24.692 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T08:02:24.692 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-04-24T08:02:24.745 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-04-24T08:02:24.746 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-04-24T08:02:24.804 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-04-24T08:02:24.804 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-04-24T08:02:24.804 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:02:24.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:02:24.873 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:02:24.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:02:24.959 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T08:02:24.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:02:25.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:02:25.110 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T08:02:25.121 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T08:02:25.128 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T08:02:25.270 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T08:02:25.294 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T08:02:25.296 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T08:07:35.382 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-04-24T08:07:35.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:07:35.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:07:35.500 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-24T08:07:35.662 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:07:35.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:07:35.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:07:35.770 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-04-24T08:07:35.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T08:07:35.922 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T08:07:35.935 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T08:07:35.935 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:07:35.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:07:35.943 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:07:35.943 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:07:36.027 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:07:36.031 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:07:36.068 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:07:36.072 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:07:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-04-24T08:07:36.148 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-04-24T08:07:36.175 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-04-24T08:07:36.179 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-04-24T08:07:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:07:36.218 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:07:36.259 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:07:36.261 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:07:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.414 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:07:36.422 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:07:36.422 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-04-24T08:07:36.423 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.423 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.423 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.423 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:07:36.424 INFO:teuthology.run_tasks:Running task install... 2024-04-24T08:07:36.432 DEBUG:teuthology.task.install:project ceph 2024-04-24T08:07:36.432 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'}} 2024-04-24T08:07:36.432 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'} 2024-04-24T08:07:36.432 INFO:teuthology.task.install:Using flavor: default 2024-04-24T08:07:36.438 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T08:07:36.438 INFO:teuthology.task.install:extra packages: [] 2024-04-24T08:07:36.438 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-04-24T08:07:36.438 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:07:36.439 INFO:teuthology.packaging:ref: None 2024-04-24T08:07:36.439 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:07:36.439 INFO:teuthology.packaging:branch: None 2024-04-24T08:07:36.439 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:07:36.823 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> None 2024-04-24T08:07:36.823 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-24T08:07:39.082 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-24T08:07:39.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-24T08:07:39.086 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-04-24T08:07:39.086 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:07:39.086 INFO:teuthology.packaging:ref: None 2024-04-24T08:07:39.086 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:07:39.086 INFO:teuthology.packaging:branch: None 2024-04-24T08:07:39.086 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:07:39.086 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-24T08:07:39.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-24T08:07:39.309 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-04-24T08:07:39.309 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-04-24T08:07:39.321 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-04-24T08:07:39.321 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-04-24T08:07:39.361 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:07:39.361 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:07:39.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:07:39.373 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:07:39.373 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:07:39.373 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:07:39.454 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T08:07:39.454 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:07:39.454 INFO:teuthology.packaging:ref: None 2024-04-24T08:07:39.454 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:07:39.454 INFO:teuthology.packaging:branch: None 2024-04-24T08:07:39.454 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:07:39.455 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:07:39.463 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T08:07:39.463 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:07:39.463 INFO:teuthology.packaging:ref: None 2024-04-24T08:07:39.463 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:07:39.464 INFO:teuthology.packaging:branch: None 2024-04-24T08:07:39.464 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:07:39.464 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:07:39.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:07:39.589 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:07:39.691 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:07:39.704 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:07:39.773 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-04-24T08:07:39.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-04-24T08:07:39.782 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-04-24T08:07:39.786 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-04-24T08:07:40.099 INFO:teuthology.orchestra.run.smithi090.stdout:48 files removed 2024-04-24T08:07:40.104 INFO:teuthology.orchestra.run.smithi149.stdout:48 files removed 2024-04-24T08:07:40.138 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw 2024-04-24T08:07:40.143 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw 2024-04-24T08:07:40.716 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 493 kB/s | 76 kB 00:00 2024-04-24T08:07:40.766 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 402 kB/s | 76 kB 00:00 2024-04-24T08:07:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 59 kB/s | 9.3 kB 00:00 2024-04-24T08:07:41.026 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 47 kB/s | 9.3 kB 00:00 2024-04-24T08:07:41.126 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 13 kB/s | 2.2 kB 00:00 2024-04-24T08:07:41.259 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 12 kB/s | 2.2 kB 00:00 2024-04-24T08:07:42.294 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-24T08:07:42.392 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-04-24T08:07:44.393 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T08:07:44.762 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-24T08:07:49.556 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-04-24T08:07:51.223 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T08:07:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T08:07:56.577 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 762 kB/s | 6.2 MB 00:08 2024-04-24T08:07:58.210 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-24T08:07:58.919 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-24T08:08:00.029 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T08:08:01.360 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:08:01.380 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:01.380 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:08:01.381 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-04-24T08:08:01.382 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout:Install 28 Packages 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 2 Packages 2024-04-24T08:08:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:01.384 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 53 M 2024-04-24T08:08:01.384 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:08:02.304 INFO:teuthology.orchestra.run.smithi090.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 16 MB/s | 5.2 MB 00:00 2024-04-24T08:08:02.354 INFO:teuthology.orchestra.run.smithi090.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 486 kB/s | 24 kB 00:00 2024-04-24T08:08:02.405 INFO:teuthology.orchestra.run.smithi090.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 13 MB/s | 653 kB 00:00 2024-04-24T08:08:02.455 INFO:teuthology.orchestra.run.smithi090.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 9.4 MB/s | 474 kB 00:00 2024-04-24T08:08:02.656 INFO:teuthology.orchestra.run.smithi090.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 11 MB/s | 7.6 MB 00:00 2024-04-24T08:08:02.681 INFO:teuthology.orchestra.run.smithi090.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-24T08:08:02.714 INFO:teuthology.orchestra.run.smithi090.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 3.6 MB/s | 119 kB 00:00 2024-04-24T08:08:02.748 INFO:teuthology.orchestra.run.smithi090.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.6 MB/s | 155 kB 00:00 2024-04-24T08:08:02.799 INFO:teuthology.orchestra.run.smithi090.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-04-24T08:08:02.841 INFO:teuthology.orchestra.run.smithi090.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-04-24T08:08:02.874 INFO:teuthology.orchestra.run.smithi090.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-04-24T08:08:02.983 INFO:teuthology.orchestra.run.smithi090.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-04-24T08:08:03.100 INFO:teuthology.orchestra.run.smithi090.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 476 kB/s | 107 kB 00:00 2024-04-24T08:08:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-04-24T08:08:03.200 INFO:teuthology.orchestra.run.smithi090.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-04-24T08:08:03.476 INFO:teuthology.orchestra.run.smithi090.stdout:(16/30): ceph-common-18.2.0-0.el9.x86_64.rpm 12 MB/s | 18 MB 00:01 2024-04-24T08:08:03.526 INFO:teuthology.orchestra.run.smithi090.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-24T08:08:03.560 INFO:teuthology.orchestra.run.smithi090.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 816 kB/s | 292 kB 00:00 2024-04-24T08:08:03.635 INFO:teuthology.orchestra.run.smithi090.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 268 kB/s | 42 kB 00:00 2024-04-24T08:08:03.812 INFO:teuthology.orchestra.run.smithi090.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 389 kB/s | 111 kB 00:00 2024-04-24T08:08:03.912 INFO:teuthology.orchestra.run.smithi090.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-24T08:08:03.937 INFO:teuthology.orchestra.run.smithi090.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 821 kB/s | 309 kB 00:00 2024-04-24T08:08:04.004 INFO:teuthology.orchestra.run.smithi090.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 532 kB/s | 49 kB 00:00 2024-04-24T08:08:04.038 INFO:teuthology.orchestra.run.smithi090.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-04-24T08:08:04.139 INFO:teuthology.orchestra.run.smithi090.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-24T08:08:04.481 INFO:teuthology.orchestra.run.smithi090.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-24T08:08:04.606 INFO:teuthology.orchestra.run.smithi090.stdout:(27/30): librados2-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.3 MB 00:00 2024-04-24T08:08:04.723 INFO:teuthology.orchestra.run.smithi090.stdout:(28/30): librbd1-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-04-24T08:08:04.773 INFO:teuthology.orchestra.run.smithi090.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-04-24T08:08:05.015 INFO:teuthology.orchestra.run.smithi090.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-24T08:08:05.018 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:05.019 INFO:teuthology.orchestra.run.smithi090.stdout:Total 15 MB/s | 53 MB 00:03 2024-04-24T08:08:05.241 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:08:05.346 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:08:05.346 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:08:06.004 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-24T08:08:06.289 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:08:06.290 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:08:07.874 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:08:08.018 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-24T08:08:08.075 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-24T08:08:08.418 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-24T08:08:08.436 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-04-24T08:08:08.437 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:08:08.438 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:Install 28 Packages 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 2 Packages 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 53 M 2024-04-24T08:08:08.439 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:08:08.603 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-24T08:08:08.730 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:08.916 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:09.027 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-04-24T08:08:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:09.181 INFO:teuthology.orchestra.run.smithi149.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 16 MB/s | 5.2 MB 00:00 2024-04-24T08:08:09.191 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:09.205 INFO:teuthology.orchestra.run.smithi149.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 983 kB/s | 24 kB 00:00 2024-04-24T08:08:09.243 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-24T08:08:09.285 INFO:teuthology.orchestra.run.smithi149.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 8.1 MB/s | 653 kB 00:00 2024-04-24T08:08:09.331 INFO:teuthology.orchestra.run.smithi149.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 10 MB/s | 474 kB 00:00 2024-04-24T08:08:09.359 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-04-24T08:08:09.473 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-24T08:08:09.501 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:09.506 INFO:teuthology.orchestra.run.smithi149.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 12 MB/s | 7.6 MB 00:00 2024-04-24T08:08:09.540 INFO:teuthology.orchestra.run.smithi149.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.4 MB/s | 45 kB 00:00 2024-04-24T08:08:09.573 INFO:teuthology.orchestra.run.smithi149.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 3.6 MB/s | 119 kB 00:00 2024-04-24T08:08:09.607 INFO:teuthology.orchestra.run.smithi149.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 155 kB 00:00 2024-04-24T08:08:09.646 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:09.674 INFO:teuthology.orchestra.run.smithi149.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2024-04-24T08:08:09.715 INFO:teuthology.orchestra.run.smithi149.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.2 MB/s | 297 kB 00:00 2024-04-24T08:08:09.749 INFO:teuthology.orchestra.run.smithi149.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-04-24T08:08:09.834 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-24T08:08:09.875 INFO:teuthology.orchestra.run.smithi149.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-24T08:08:09.878 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-24T08:08:09.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-24T08:08:09.967 INFO:teuthology.orchestra.run.smithi149.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 495 kB/s | 107 kB 00:00 2024-04-24T08:08:09.988 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-24T08:08:10.030 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-24T08:08:10.089 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-24T08:08:10.144 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-24T08:08:10.202 INFO:teuthology.orchestra.run.smithi149.stdout:(14/30): ceph-common-18.2.0-0.el9.x86_64.rpm 14 MB/s | 18 MB 00:01 2024-04-24T08:08:10.226 INFO:teuthology.orchestra.run.smithi149.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 175 kB/s | 45 kB 00:00 2024-04-24T08:08:10.251 INFO:teuthology.orchestra.run.smithi149.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 427 kB/s | 160 kB 00:00 2024-04-24T08:08:10.310 INFO:teuthology.orchestra.run.smithi149.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 732 kB/s | 42 kB 00:00 2024-04-24T08:08:10.352 INFO:teuthology.orchestra.run.smithi149.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-24T08:08:10.494 INFO:teuthology.orchestra.run.smithi149.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-04-24T08:08:10.519 INFO:teuthology.orchestra.run.smithi149.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 532 kB/s | 111 kB 00:00 2024-04-24T08:08:10.551 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-24T08:08:10.569 INFO:teuthology.orchestra.run.smithi149.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.4 MB/s | 309 kB 00:00 2024-04-24T08:08:10.594 INFO:teuthology.orchestra.run.smithi149.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 348 kB/s | 26 kB 00:00 2024-04-24T08:08:10.601 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-24T08:08:10.628 INFO:teuthology.orchestra.run.smithi149.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 842 kB/s | 49 kB 00:00 2024-04-24T08:08:10.653 INFO:teuthology.orchestra.run.smithi149.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T08:08:10.729 INFO:teuthology.orchestra.run.smithi149.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T08:08:10.735 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-24T08:08:10.767 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-04-24T08:08:10.812 INFO:teuthology.orchestra.run.smithi149.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-24T08:08:10.840 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-24T08:08:10.914 INFO:teuthology.orchestra.run.smithi149.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-04-24T08:08:11.046 INFO:teuthology.orchestra.run.smithi149.stdout:(28/30): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-04-24T08:08:11.130 INFO:teuthology.orchestra.run.smithi149.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 10 MB/s | 3.3 MB 00:00 2024-04-24T08:08:11.222 INFO:teuthology.orchestra.run.smithi149.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 9.7 MB/s | 3.0 MB 00:00 2024-04-24T08:08:11.226 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:11.227 INFO:teuthology.orchestra.run.smithi149.stdout:Total 19 MB/s | 53 MB 00:02 2024-04-24T08:08:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-24T08:08:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-24T08:08:11.448 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:08:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-24T08:08:11.556 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:08:11.557 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:08:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-04-24T08:08:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-04-24T08:08:12.482 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:08:12.483 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:08:14.179 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:08:14.295 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:14.342 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:14.345 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-24T08:08:14.390 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-24T08:08:14.486 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-24T08:08:14.894 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-24T08:08:15.028 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:15.047 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:15.095 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-24T08:08:15.168 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:15.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-04-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:15.310 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:15.330 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:15.415 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:15.464 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-24T08:08:15.551 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-04-24T08:08:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-24T08:08:15.687 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:15.801 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:16.004 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-24T08:08:16.048 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-24T08:08:16.122 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-24T08:08:16.189 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-24T08:08:16.231 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-24T08:08:16.291 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-24T08:08:16.344 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-24T08:08:16.754 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-24T08:08:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-24T08:08:17.024 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-24T08:08:17.060 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-04-24T08:08:17.131 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-24T08:08:17.619 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-24T08:08:17.646 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-24T08:08:17.731 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-24T08:08:17.838 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-04-24T08:08:17.874 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-04-24T08:08:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:20.395 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:21.062 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-24T08:08:21.115 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-24T08:08:21.320 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-24T08:08:21.321 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:08:21.321 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:08:21.321 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:21.366 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-04-24T08:08:27.075 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:27.981 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:08:28.082 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:28.088 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-24T08:08:28.219 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-24T08:08:28.219 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:29.908 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-24T08:08:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-24T08:08:29.910 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:29.911 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-24T08:08:29.911 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-04-24T08:08:30.803 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:08:30.804 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:08:30.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:30.806 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:08:31.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-test 2024-04-24T08:08:32.057 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:34 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:08:32.100 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-04-24T08:08:32.101 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:32.193 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:08:32.194 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:32.194 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:08:32.194 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:32.194 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:08:32.194 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout:Install 4 Packages 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:32.195 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 40 M 2024-04-24T08:08:32.196 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 165 M 2024-04-24T08:08:32.196 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:08:32.390 INFO:teuthology.orchestra.run.smithi090.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 161 kB 00:00 2024-04-24T08:08:32.515 INFO:teuthology.orchestra.run.smithi090.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-24T08:08:32.565 INFO:teuthology.orchestra.run.smithi090.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 364 kB/s | 64 kB 00:00 2024-04-24T08:08:32.961 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:08:33.057 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:33.063 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-24T08:08:33.149 INFO:teuthology.orchestra.run.smithi090.stdout:(4/4): ceph-test-18.2.0-0.el9.x86_64.rpm 45 MB/s | 40 MB 00:00 2024-04-24T08:08:33.152 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:33.153 INFO:teuthology.orchestra.run.smithi090.stdout:Total 42 MB/s | 40 MB 00:00 2024-04-24T08:08:33.182 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-24T08:08:33.182 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:33.210 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:08:33.234 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:08:33.235 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:08:33.658 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:08:33.659 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:08:34.314 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:08:34.508 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T08:08:34.620 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T08:08:34.667 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-04-24T08:08:34.846 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-24T08:08:34.847 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-24T08:08:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-04-24T08:08:35.782 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:08:35.783 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:35.785 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:08:35.785 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:08:35.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:35.785 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:08:36.190 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-test 2024-04-24T08:08:36.968 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:31 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:08:37.107 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-04-24T08:08:37.108 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-04-24T08:08:37.109 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:37.110 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 40 M 2024-04-24T08:08:37.110 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 165 M 2024-04-24T08:08:37.110 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:08:37.557 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 161 kB 00:00 2024-04-24T08:08:38.291 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): ceph-test-18.2.0-0.el9.x86_64.rpm 45 MB/s | 40 MB 00:00 2024-04-24T08:08:38.684 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 56 kB/s | 64 kB 00:01 2024-04-24T08:08:38.868 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): socat-1.7.4.1-5.el9.x86_64.rpm 210 kB/s | 305 kB 00:01 2024-04-24T08:08:38.872 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:38.873 INFO:teuthology.orchestra.run.smithi149.stdout:Total 23 MB/s | 40 MB 00:01 2024-04-24T08:08:38.948 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:08:38.976 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-24T08:08:38.982 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:08:38.982 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:08:39.216 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-24T08:08:39.446 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:08:39.446 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:08:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:08:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T08:08:40.289 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T08:08:40.320 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-24T08:08:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-24T08:08:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-04-24T08:08:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-04-24T08:08:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T08:08:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:41.956 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:08:42.282 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph 2024-04-24T08:08:43.063 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:45 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:08:43.194 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:08:43.196 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:08:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:08:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:43.199 INFO:teuthology.orchestra.run.smithi090.stdout:Install 33 Packages 2024-04-24T08:08:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:43.200 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 M 2024-04-24T08:08:43.200 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 109 M 2024-04-24T08:08:43.200 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:08:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 32 kB/s | 6.4 kB 00:00 2024-04-24T08:08:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 5.4 MB/s | 1.4 MB 00:00 2024-04-24T08:08:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 6.5 MB/s | 2.1 MB 00:00 2024-04-24T08:08:44.330 INFO:teuthology.orchestra.run.smithi090.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 3.0 MB/s | 240 kB 00:00 2024-04-24T08:08:44.469 INFO:teuthology.orchestra.run.smithi090.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-24T08:08:44.552 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-24T08:08:44.811 INFO:teuthology.orchestra.run.smithi090.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 28 MB/s | 18 MB 00:00 2024-04-24T08:08:44.844 INFO:teuthology.orchestra.run.smithi090.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 494 kB/s | 253 kB 00:00 2024-04-24T08:08:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-24T08:08:44.911 INFO:teuthology.orchestra.run.smithi090.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-04-24T08:08:44.961 INFO:teuthology.orchestra.run.smithi090.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 2.5 MB/s | 1.2 MB 00:00 2024-04-24T08:08:44.996 INFO:teuthology.orchestra.run.smithi090.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.4 MB/s | 125 kB 00:00 2024-04-24T08:08:45.021 INFO:teuthology.orchestra.run.smithi090.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 508 kB/s | 106 kB 00:00 2024-04-24T08:08:45.055 INFO:teuthology.orchestra.run.smithi090.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-04-24T08:08:45.089 INFO:teuthology.orchestra.run.smithi090.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 524 kB/s | 35 kB 00:00 2024-04-24T08:08:45.122 INFO:teuthology.orchestra.run.smithi090.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 771 kB/s | 51 kB 00:00 2024-04-24T08:08:45.156 INFO:teuthology.orchestra.run.smithi090.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 629 kB/s | 42 kB 00:00 2024-04-24T08:08:45.190 INFO:teuthology.orchestra.run.smithi090.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 895 kB/s | 172 kB 00:00 2024-04-24T08:08:45.248 INFO:teuthology.orchestra.run.smithi090.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 349 kB/s | 43 kB 00:00 2024-04-24T08:08:45.299 INFO:teuthology.orchestra.run.smithi090.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 217 kB/s | 11 kB 00:00 2024-04-24T08:08:45.358 INFO:teuthology.orchestra.run.smithi090.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-04-24T08:08:45.383 INFO:teuthology.orchestra.run.smithi090.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 213 kB/s | 18 kB 00:00 2024-04-24T08:08:45.408 INFO:teuthology.orchestra.run.smithi090.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 468 kB/s | 23 kB 00:00 2024-04-24T08:08:45.433 INFO:teuthology.orchestra.run.smithi090.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 393 kB/s | 19 kB 00:00 2024-04-24T08:08:45.459 INFO:teuthology.orchestra.run.smithi090.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 397 kB/s | 20 kB 00:00 2024-04-24T08:08:45.492 INFO:teuthology.orchestra.run.smithi090.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-04-24T08:08:45.517 INFO:teuthology.orchestra.run.smithi090.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 557 kB/s | 46 kB 00:00 2024-04-24T08:08:45.543 INFO:teuthology.orchestra.run.smithi090.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 945 kB/s | 79 kB 00:00 2024-04-24T08:08:45.568 INFO:teuthology.orchestra.run.smithi090.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 330 kB/s | 16 kB 00:00 2024-04-24T08:08:45.594 INFO:teuthology.orchestra.run.smithi090.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-04-24T08:08:45.619 INFO:teuthology.orchestra.run.smithi090.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 723 kB/s | 36 kB 00:00 2024-04-24T08:08:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-24T08:08:45.695 INFO:teuthology.orchestra.run.smithi090.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T08:08:45.728 INFO:teuthology.orchestra.run.smithi090.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 268 kB/s | 20 kB 00:00 2024-04-24T08:08:45.770 INFO:teuthology.orchestra.run.smithi090.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.8 MB/s | 427 kB 00:00 2024-04-24T08:08:45.774 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:45.774 INFO:teuthology.orchestra.run.smithi090.stdout:Total 12 MB/s | 30 MB 00:02 2024-04-24T08:08:46.146 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:08:46.231 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:08:46.232 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:08:46.788 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:08:46.789 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:08:46.872 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-24T08:08:46.872 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-04-24T08:08:46.872 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-04-24T08:08:46.872 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T08:08:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T08:08:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:47.218 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:08:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:08:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:47.219 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:08:47.585 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph 2024-04-24T08:08:48.083 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:08:48.141 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T08:08:48.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T08:08:48.239 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T08:08:48.361 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:43 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:08:48.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T08:08:48.492 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:08:48.494 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:48.494 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:08:48.494 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:48.494 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:08:48.494 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:08:48.497 INFO:teuthology.orchestra.run.smithi149.stdout:Install 33 Packages 2024-04-24T08:08:48.498 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:48.498 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 30 M 2024-04-24T08:08:48.498 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 109 M 2024-04-24T08:08:48.498 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:08:48.502 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T08:08:48.557 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T08:08:48.599 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T08:08:48.636 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T08:08:48.681 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T08:08:48.725 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T08:08:48.785 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T08:08:48.846 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T08:08:48.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T08:08:48.951 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T08:08:49.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T08:08:49.287 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T08:08:49.367 INFO:teuthology.orchestra.run.smithi149.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 38 kB/s | 6.4 kB 00:00 2024-04-24T08:08:49.393 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T08:08:49.434 INFO:teuthology.orchestra.run.smithi149.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-04-24T08:08:49.496 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T08:08:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 6.3 MB/s | 2.1 MB 00:00 2024-04-24T08:08:49.539 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T08:08:49.593 INFO:teuthology.orchestra.run.smithi149.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 4.1 MB/s | 240 kB 00:00 2024-04-24T08:08:49.605 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T08:08:49.694 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T08:08:49.710 INFO:teuthology.orchestra.run.smithi149.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-24T08:08:49.948 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T08:08:50.011 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T08:08:50.052 INFO:teuthology.orchestra.run.smithi149.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 28 MB/s | 18 MB 00:00 2024-04-24T08:08:50.114 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T08:08:50.229 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T08:08:50.269 INFO:teuthology.orchestra.run.smithi149.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 375 kB/s | 253 kB 00:00 2024-04-24T08:08:50.285 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T08:08:50.436 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T08:08:50.486 INFO:teuthology.orchestra.run.smithi149.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 246 kB/s | 106 kB 00:00 2024-04-24T08:08:50.553 INFO:teuthology.orchestra.run.smithi149.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 1.5 MB/s | 1.2 MB 00:00 2024-04-24T08:08:50.620 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-04-24T08:08:50.651 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-24T08:08:50.695 INFO:teuthology.orchestra.run.smithi149.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 319 kB/s | 135 kB 00:00 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:08:50.727 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:50.879 INFO:teuthology.orchestra.run.smithi149.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 943 kB/s | 172 kB 00:00 2024-04-24T08:08:50.912 INFO:teuthology.orchestra.run.smithi149.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 294 kB/s | 125 kB 00:00 2024-04-24T08:08:50.937 INFO:teuthology.orchestra.run.smithi149.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 601 kB/s | 35 kB 00:00 2024-04-24T08:08:50.996 INFO:teuthology.orchestra.run.smithi149.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 721 kB/s | 42 kB 00:00 2024-04-24T08:08:51.030 INFO:teuthology.orchestra.run.smithi149.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 438 kB/s | 51 kB 00:00 2024-04-24T08:08:51.063 INFO:teuthology.orchestra.run.smithi149.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 421 kB/s | 215 kB 00:00 2024-04-24T08:08:51.180 INFO:teuthology.orchestra.run.smithi149.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 237 kB/s | 43 kB 00:00 2024-04-24T08:08:51.239 INFO:teuthology.orchestra.run.smithi149.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 186 kB/s | 11 kB 00:00 2024-04-24T08:08:51.306 INFO:teuthology.orchestra.run.smithi149.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 624 kB/s | 172 kB 00:00 2024-04-24T08:08:51.331 INFO:teuthology.orchestra.run.smithi149.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 194 kB/s | 18 kB 00:00 2024-04-24T08:08:51.373 INFO:teuthology.orchestra.run.smithi149.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.1 MB/s | 358 kB 00:00 2024-04-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi149.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 253 kB/s | 23 kB 00:00 2024-04-24T08:08:51.423 INFO:teuthology.orchestra.run.smithi149.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-04-24T08:08:51.449 INFO:teuthology.orchestra.run.smithi149.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 262 kB/s | 20 kB 00:00 2024-04-24T08:08:51.474 INFO:teuthology.orchestra.run.smithi149.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 618 kB/s | 46 kB 00:00 2024-04-24T08:08:51.558 INFO:teuthology.orchestra.run.smithi149.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-24T08:08:51.583 INFO:teuthology.orchestra.run.smithi149.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 497 kB/s | 79 kB 00:00 2024-04-24T08:08:51.617 INFO:teuthology.orchestra.run.smithi149.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 116 kB/s | 16 kB 00:00 2024-04-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi149.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 538 kB/s | 36 kB 00:00 2024-04-24T08:08:51.675 INFO:teuthology.orchestra.run.smithi149.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 769 kB/s | 90 kB 00:00 2024-04-24T08:08:51.709 INFO:teuthology.orchestra.run.smithi149.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-24T08:08:51.742 INFO:teuthology.orchestra.run.smithi149.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 302 kB/s | 20 kB 00:00 2024-04-24T08:08:51.835 INFO:teuthology.orchestra.run.smithi149.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.3 MB/s | 427 kB 00:00 2024-04-24T08:08:51.840 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:51.840 INFO:teuthology.orchestra.run.smithi149.stdout:Total 9.0 MB/s | 30 MB 00:03 2024-04-24T08:08:52.200 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:08:52.286 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:08:52.286 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:08:52.394 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-24T08:08:52.471 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-24T08:08:52.471 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:52.471 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:08:52.471 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:08:52.471 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:08:52.472 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:52.771 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:08:52.772 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:08:52.978 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:53.271 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:08:53.338 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:53.428 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-24T08:08:53.558 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-24T08:08:53.558 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:08:53.558 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:54.154 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:08:54.272 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T08:08:54.328 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T08:08:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T08:08:54.534 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T08:08:54.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T08:08:54.712 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T08:08:54.869 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T08:08:54.929 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T08:08:55.007 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T08:08:55.047 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T08:08:55.111 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T08:08:55.482 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T08:08:55.594 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T08:08:55.655 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T08:08:55.732 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-04-24T08:08:55.812 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T08:08:55.813 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T08:08:55.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T08:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T08:08:55.815 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T08:08:55.893 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T08:08:56.148 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T08:08:56.211 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T08:08:56.301 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T08:08:56.410 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T08:08:56.483 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T08:08:56.636 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T08:08:56.827 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-04-24T08:08:56.865 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-24T08:08:56.914 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:08:56.915 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:08:56.916 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:56.917 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:08:56.939 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-24T08:08:56.939 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:56.940 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:08:56.940 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:08:56.940 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:08:56.940 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:57.365 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-base 2024-04-24T08:08:58.183 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:00 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:08:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:08:58.306 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:08:58.307 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:08:58.307 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:08:58.465 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install cephadm 2024-04-24T08:08:58.607 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:08:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:59.183 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-24T08:08:59.227 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:01 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:08:59.255 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-24T08:08:59.256 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:59.256 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:08:59.256 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:08:59.256 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:08:59.256 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:59.348 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:08:59.349 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:08:59.349 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:59.349 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:08:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 209 k 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 200 k 2024-04-24T08:08:59.350 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:08:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm-18.2.0-0.el9.noarch.rpm 1.9 MB/s | 209 kB 00:00 2024-04-24T08:08:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:08:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.9 MB/s | 209 kB 00:00 2024-04-24T08:08:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:08:59.465 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:08:59.465 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:08:59.469 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:08:59.469 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:08:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-24T08:08:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:08:59.574 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-24T08:08:59.575 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:08:59.575 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:08:59.575 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:08:59.575 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:08:59.575 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:08:59.642 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-24T08:08:59.771 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-24T08:08:59.771 INFO:teuthology.orchestra.run.smithi149.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:08:59.771 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:01.152 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:01.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:01.408 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:01.408 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:01.408 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-04-24T08:09:01.408 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:01.408 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:01.464 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-04-24T08:09:01.464 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T08:09:01.465 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T08:09:01.466 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T08:09:01.467 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T08:09:01.754 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T08:09:02.543 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:04 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-04-24T08:09:02.629 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:09:02.630 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:09:02.631 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:02.632 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:02.669 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:02.669 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:09:02.670 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:02.671 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 142 k 2024-04-24T08:09:02.671 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 399 k 2024-04-24T08:09:02.671 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:02.752 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.7 MB/s | 142 kB 00:00 2024-04-24T08:09:02.752 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:02.753 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.7 MB/s | 142 kB 00:00 2024-04-24T08:09:02.753 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:02.762 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:02.762 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:02.827 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:02.828 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:02.999 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:03.021 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-base 2024-04-24T08:09:03.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:03.134 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:03.134 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:09:03.134 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:09:03.134 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:03.830 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:58 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:03.916 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:03.959 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:03.960 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:03.961 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:04.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install cephadm 2024-04-24T08:09:04.476 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:04.477 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:04.477 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:04.477 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:04.477 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:04.477 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:04.779 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:59 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:04.867 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr 2024-04-24T08:09:04.902 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 209 k 2024-04-24T08:09:04.903 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 200 k 2024-04-24T08:09:04.904 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:05.015 INFO:teuthology.orchestra.run.smithi149.stdout:cephadm-18.2.0-0.el9.noarch.rpm 1.8 MB/s | 209 kB 00:00 2024-04-24T08:09:05.016 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:05.016 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.8 MB/s | 209 kB 00:00 2024-04-24T08:09:05.016 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:05.023 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:05.023 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:05.026 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:05.027 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:05.119 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:05.634 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:07 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:05.714 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:05.756 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:05.757 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:05.757 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:05.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T08:09:05.855 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:05.970 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:06.429 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:06.569 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:08 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:06.696 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:06.697 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:09:06.697 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:06.697 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:06.698 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:06.699 INFO:teuthology.orchestra.run.smithi090.stdout:Install 7 Packages 2024-04-24T08:09:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:06.699 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.0 M 2024-04-24T08:09:06.699 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 8.1 M 2024-04-24T08:09:06.699 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:06.764 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:07.085 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T08:09:07.410 INFO:teuthology.orchestra.run.smithi090.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 90 kB/s | 23 kB 00:00 2024-04-24T08:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 48 kB/s | 15 kB 00:00 2024-04-24T08:09:07.527 INFO:teuthology.orchestra.run.smithi090.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 4.5 MB/s | 1.7 MB 00:00 2024-04-24T08:09:07.836 INFO:teuthology.orchestra.run.smithi090.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 21 kB/s | 9.0 kB 00:00 2024-04-24T08:09:07.858 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:02 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:07.894 INFO:teuthology.orchestra.run.smithi090.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 94 kB/s | 41 kB 00:00 2024-04-24T08:09:07.989 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:07.990 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:07.991 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:09:07.991 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:07.991 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 142 k 2024-04-24T08:09:07.991 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 399 k 2024-04-24T08:09:07.991 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:08.019 INFO:teuthology.orchestra.run.smithi090.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 81 kB/s | 31 kB 00:00 2024-04-24T08:09:08.089 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.4 MB/s | 142 kB 00:00 2024-04-24T08:09:08.090 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:08.090 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.4 MB/s | 142 kB 00:00 2024-04-24T08:09:08.090 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:08.100 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:08.100 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:08.120 INFO:teuthology.orchestra.run.smithi090.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 669 kB/s | 188 kB 00:00 2024-04-24T08:09:08.130 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:08.130 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-04-24T08:09:08.132 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:08.159 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:08.160 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:08.165 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:08.166 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:08.306 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:08.307 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:08.337 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:08.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:08.526 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:08.526 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:09:08.526 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:09:08.526 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:08.724 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:08.884 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T08:09:08.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T08:09:09.047 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T08:09:09.087 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T08:09:09.135 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-04-24T08:09:09.450 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-04-24T08:09:09.482 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-24T08:09:09.870 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:09.870 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:09.870 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:09.871 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:09.871 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:09.871 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:10.162 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr 2024-04-24T08:09:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-24T08:09:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-04-24T08:09:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-04-24T08:09:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-04-24T08:09:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T08:09:10.266 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T08:09:10.266 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T08:09:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T08:09:10.579 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:09:10.579 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:09:10.579 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:10.579 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:10.881 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T08:09:10.924 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:05 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:11.007 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:11.049 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:11.050 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:11.050 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:11.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T08:09:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:13 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:11.779 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:09:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:11.781 INFO:teuthology.orchestra.run.smithi090.stdout:Install 12 Packages 2024-04-24T08:09:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:11.782 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 38 M 2024-04-24T08:09:11.782 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 209 M 2024-04-24T08:09:11.782 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:11.886 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:06 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:12.018 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:09:12.019 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout:Install 7 Packages 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:12.020 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.0 M 2024-04-24T08:09:12.021 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 8.1 M 2024-04-24T08:09:12.021 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:12.521 INFO:teuthology.orchestra.run.smithi090.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 26 MB/s | 7.4 MB 00:00 2024-04-24T08:09:12.588 INFO:teuthology.orchestra.run.smithi090.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 558 kB/s | 195 kB 00:00 2024-04-24T08:09:12.616 INFO:teuthology.orchestra.run.smithi149.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 103 kB/s | 23 kB 00:00 2024-04-24T08:09:12.641 INFO:teuthology.orchestra.run.smithi149.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 59 kB/s | 15 kB 00:00 2024-04-24T08:09:12.655 INFO:teuthology.orchestra.run.smithi090.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-24T08:09:12.706 INFO:teuthology.orchestra.run.smithi090.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 178 kB/s | 33 kB 00:00 2024-04-24T08:09:12.716 INFO:teuthology.orchestra.run.smithi149.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 5.2 MB/s | 1.7 MB 00:00 2024-04-24T08:09:12.800 INFO:teuthology.orchestra.run.smithi149.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 49 kB/s | 9.0 kB 00:00 2024-04-24T08:09:12.806 INFO:teuthology.orchestra.run.smithi090.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 121 kB/s | 18 kB 00:00 2024-04-24T08:09:12.856 INFO:teuthology.orchestra.run.smithi090.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 235 kB/s | 35 kB 00:00 2024-04-24T08:09:12.883 INFO:teuthology.orchestra.run.smithi149.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 185 kB/s | 31 kB 00:00 2024-04-24T08:09:12.908 INFO:teuthology.orchestra.run.smithi149.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 153 kB/s | 41 kB 00:00 2024-04-24T08:09:12.990 INFO:teuthology.orchestra.run.smithi090.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.5 MB/s | 3.0 MB 00:00 2024-04-24T08:09:13.017 INFO:teuthology.orchestra.run.smithi149.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 871 kB/s | 188 kB 00:00 2024-04-24T08:09:13.022 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:13.022 INFO:teuthology.orchestra.run.smithi149.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-04-24T08:09:13.023 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:13.023 INFO:teuthology.orchestra.run.smithi090.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-24T08:09:13.052 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:13.052 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-04-24T08:09:13.207 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:13.207 INFO:teuthology.orchestra.run.smithi090.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-04-24T08:09:13.208 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:13.341 INFO:teuthology.orchestra.run.smithi090.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-24T08:09:13.612 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:13.694 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T08:09:13.708 INFO:teuthology.orchestra.run.smithi090.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-24T08:09:13.708 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:13.710 INFO:teuthology.orchestra.run.smithi090.stdout:Total 20 MB/s | 38 MB 00:01 2024-04-24T08:09:13.743 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T08:09:13.835 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T08:09:13.884 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T08:09:13.930 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-04-24T08:09:14.241 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-04-24T08:09:14.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-24T08:09:14.349 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:14.427 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:14.427 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T08:09:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T08:09:15.063 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:15.064 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:15.279 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T08:09:15.279 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:15.279 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:15.280 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:15.578 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T08:09:15.976 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:16.127 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T08:09:16.203 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T08:09:16.343 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:11 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:16.361 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T08:09:16.411 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T08:09:16.474 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:09:16.476 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout:Install 12 Packages 2024-04-24T08:09:16.477 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:16.478 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 38 M 2024-04-24T08:09:16.478 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 209 M 2024-04-24T08:09:16.478 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:16.658 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T08:09:16.692 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T08:09:16.831 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T08:09:17.053 INFO:teuthology.orchestra.run.smithi149.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 25 MB/s | 7.4 MB 00:00 2024-04-24T08:09:17.083 INFO:teuthology.orchestra.run.smithi149.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 593 kB/s | 195 kB 00:00 2024-04-24T08:09:17.133 INFO:teuthology.orchestra.run.smithi149.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-04-24T08:09:17.200 INFO:teuthology.orchestra.run.smithi149.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 224 kB/s | 33 kB 00:00 2024-04-24T08:09:17.225 INFO:teuthology.orchestra.run.smithi149.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 200 kB/s | 18 kB 00:00 2024-04-24T08:09:17.250 INFO:teuthology.orchestra.run.smithi149.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 713 kB/s | 35 kB 00:00 2024-04-24T08:09:17.342 INFO:teuthology.orchestra.run.smithi149.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-24T08:09:17.459 INFO:teuthology.orchestra.run.smithi149.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.0 MB/s | 3.0 MB 00:00 2024-04-24T08:09:17.518 INFO:teuthology.orchestra.run.smithi149.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.3 MB/s | 250 kB 00:00 2024-04-24T08:09:17.651 INFO:teuthology.orchestra.run.smithi149.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 11 MB/s | 4.7 MB 00:00 2024-04-24T08:09:17.763 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T08:09:17.768 INFO:teuthology.orchestra.run.smithi149.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T08:09:17.923 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T08:09:18.102 INFO:teuthology.orchestra.run.smithi149.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-24T08:09:18.103 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:18.104 INFO:teuthology.orchestra.run.smithi149.stdout:Total 23 MB/s | 38 MB 00:01 2024-04-24T08:09:18.729 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:18.807 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:18.807 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:19.465 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:19.465 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:19.581 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T08:09:20.393 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:20.523 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T08:09:20.601 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T08:09:20.745 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T08:09:20.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T08:09:20.782 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-24T08:09:20.803 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T08:09:21.106 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T08:09:21.143 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T08:09:21.269 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T08:09:22.164 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T08:09:22.303 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T08:09:23.929 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T08:09:24.483 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-24T08:09:24.483 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-04-24T08:09:24.483 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T08:09:24.484 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T08:09:24.865 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T08:09:24.865 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:24.865 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:24.865 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-04-24T08:09:24.865 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:24.866 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:25.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T08:09:25.157 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-24T08:09:25.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-rook 2024-04-24T08:09:26.109 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:28 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:26.235 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:26.236 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:26.236 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:09:26.236 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:09:26.237 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout:Install 13 Packages 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:26.238 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.2 M 2024-04-24T08:09:26.239 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2024-04-24T08:09:26.239 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:26.830 INFO:teuthology.orchestra.run.smithi090.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 397 kB/s | 47 kB 00:00 2024-04-24T08:09:26.872 INFO:teuthology.orchestra.run.smithi090.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 159 kB/s | 26 kB 00:00 2024-04-24T08:09:26.897 INFO:teuthology.orchestra.run.smithi090.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 103 kB/s | 19 kB 00:00 2024-04-24T08:09:27.031 INFO:teuthology.orchestra.run.smithi090.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-24T08:09:27.090 INFO:teuthology.orchestra.run.smithi090.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-04-24T08:09:27.115 INFO:teuthology.orchestra.run.smithi090.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 784 kB/s | 222 kB 00:00 2024-04-24T08:09:27.140 INFO:teuthology.orchestra.run.smithi090.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 496 kB/s | 54 kB 00:00 2024-04-24T08:09:27.350 INFO:teuthology.orchestra.run.smithi090.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-04-24T08:09:27.384 INFO:teuthology.orchestra.run.smithi090.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-24T08:09:27.451 INFO:teuthology.orchestra.run.smithi090.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 894 kB/s | 59 kB 00:00 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T08:09:27.463 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T08:09:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 911 kB/s | 206 kB 00:00 2024-04-24T08:09:27.518 INFO:teuthology.orchestra.run.smithi090.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-24T08:09:27.585 INFO:teuthology.orchestra.run.smithi090.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.4 MB/s | 1.0 MB 00:00 2024-04-24T08:09:27.592 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:27.592 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-04-24T08:09:27.800 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:27.846 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:27.846 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:09:27.971 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:27.972 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:27.988 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:27.989 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:28.422 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-rook 2024-04-24T08:09:28.758 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:28.976 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T08:09:29.038 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T08:09:29.112 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T08:09:29.158 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T08:09:29.171 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:24 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:29.215 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T08:09:29.305 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:09:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout:Install 13 Packages 2024-04-24T08:09:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:29.309 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.2 M 2024-04-24T08:09:29.309 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 26 M 2024-04-24T08:09:29.309 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:29.355 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T08:09:29.499 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T08:09:29.566 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T08:09:29.892 INFO:teuthology.orchestra.run.smithi149.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 452 kB/s | 47 kB 00:00 2024-04-24T08:09:30.426 INFO:teuthology.orchestra.run.smithi149.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 30 kB/s | 19 kB 00:00 2024-04-24T08:09:30.493 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T08:09:30.517 INFO:teuthology.orchestra.run.smithi149.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 35 kB/s | 26 kB 00:00 2024-04-24T08:09:30.550 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T08:09:30.593 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T08:09:30.655 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T08:09:30.686 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-24T08:09:30.976 INFO:teuthology.orchestra.run.smithi149.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 205 kB/s | 222 kB 00:01 2024-04-24T08:09:31.356 INFO:teuthology.orchestra.run.smithi149.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 171 kB/s | 159 kB 00:00 2024-04-24T08:09:31.518 INFO:teuthology.orchestra.run.smithi149.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 279 kB/s | 279 kB 00:00 2024-04-24T08:09:31.560 INFO:teuthology.orchestra.run.smithi149.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 161 kB/s | 32 kB 00:00 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T08:09:31.670 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T08:09:31.671 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T08:09:31.728 INFO:teuthology.orchestra.run.smithi149.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 71 kB/s | 54 kB 00:00 2024-04-24T08:09:31.761 INFO:teuthology.orchestra.run.smithi149.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-04-24T08:09:31.786 INFO:teuthology.orchestra.run.smithi149.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 916 kB/s | 206 kB 00:00 2024-04-24T08:09:31.886 INFO:teuthology.orchestra.run.smithi149.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 900 kB/s | 90 kB 00:00 2024-04-24T08:09:32.020 INFO:teuthology.orchestra.run.smithi149.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-04-24T08:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-04-24T08:09:32.063 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout:Total 817 kB/s | 2.2 MB 00:02 2024-04-24T08:09:32.272 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:32.319 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:32.319 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:32.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T08:09:32.465 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:32.466 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:33.176 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:35 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:33.267 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:33.299 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:33.299 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:09:33.300 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 7.0 M 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 33 M 2024-04-24T08:09:33.301 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:33.520 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T08:09:33.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T08:09:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T08:09:33.717 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T08:09:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T08:09:33.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T08:09:34.057 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T08:09:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 823 kB/s | 127 kB 00:00 2024-04-24T08:09:34.132 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T08:09:34.230 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 887 kB/s | 248 kB 00:00 2024-04-24T08:09:34.413 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-24T08:09:34.447 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 267 kB/s | 58 kB 00:00 2024-04-24T08:09:34.522 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-04-24T08:09:34.614 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 256 kB/s | 51 kB 00:00 2024-04-24T08:09:34.614 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:34.615 INFO:teuthology.orchestra.run.smithi090.stdout:Total 5.3 MB/s | 7.0 MB 00:01 2024-04-24T08:09:34.741 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:34.780 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:34.780 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:34.932 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:34.932 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:35.062 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T08:09:35.149 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T08:09:35.191 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T08:09:35.253 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T08:09:35.283 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-24T08:09:35.320 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:35.541 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T08:09:35.603 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T08:09:36.174 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T08:09:36.175 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T08:09:36.175 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T08:09:36.175 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T08:09:36.175 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T08:09:36.218 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T08:09:36.354 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T08:09:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:37.026 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T08:09:37.420 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-24T08:09:37.420 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-04-24T08:09:37.420 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T08:09:37.420 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T08:09:37.421 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T08:09:37.421 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T08:09:37.739 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T08:09:37.739 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:37.739 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:37.739 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:37.740 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:37.826 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:32 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:37.958 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 7.0 M 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 33 M 2024-04-24T08:09:37.959 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:38.088 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-fuse 2024-04-24T08:09:38.573 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 892 kB/s | 127 kB 00:00 2024-04-24T08:09:38.691 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 953 kB/s | 248 kB 00:00 2024-04-24T08:09:38.843 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:38.865 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-24T08:09:38.970 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 915 k 2024-04-24T08:09:38.971 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 2.7 M 2024-04-24T08:09:38.972 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:38.990 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 194 kB/s | 58 kB 00:00 2024-04-24T08:09:39.140 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 968 kB/s | 548 kB 00:00 2024-04-24T08:09:39.173 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 166 kB/s | 51 kB 00:00 2024-04-24T08:09:39.174 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:39.175 INFO:teuthology.orchestra.run.smithi149.stdout:Total 5.7 MB/s | 7.0 MB 00:01 2024-04-24T08:09:39.219 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 6.6 MB/s | 835 kB 00:00 2024-04-24T08:09:39.286 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 423 kB/s | 80 kB 00:00 2024-04-24T08:09:39.288 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:39.289 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.8 MB/s | 915 kB 00:00 2024-04-24T08:09:39.306 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:39.328 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:39.345 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:39.345 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:39.398 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:39.398 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:39.499 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:39.499 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:39.505 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:39.505 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:39.772 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:39.894 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:39.958 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T08:09:40.032 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-24T08:09:40.127 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T08:09:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T08:09:40.946 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T08:09:41.055 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T08:09:41.142 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-24T08:09:41.142 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-04-24T08:09:41.152 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T08:09:41.179 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-24T08:09:41.456 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T08:09:41.456 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:41.456 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:41.457 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T08:09:41.457 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:41.457 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:41.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados-devel 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T08:09:42.042 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:09:42.367 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:42.368 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:42.606 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:42.698 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-fuse 2024-04-24T08:09:42.730 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:42.730 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 126 k 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 456 k 2024-04-24T08:09:42.731 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:42.804 INFO:teuthology.orchestra.run.smithi090.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 126 kB 00:00 2024-04-24T08:09:42.805 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:42.805 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.7 MB/s | 126 kB 00:00 2024-04-24T08:09:42.806 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:42.812 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:42.812 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:42.884 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:42.885 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:43.189 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:43.457 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:38 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:43.597 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:43.597 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:43.597 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:43.597 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout:Install 2 Packages 2024-04-24T08:09:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:43.599 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 915 k 2024-04-24T08:09:43.599 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 2.7 M 2024-04-24T08:09:43.599 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:43.817 INFO:teuthology.orchestra.run.smithi149.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 835 kB 00:00 2024-04-24T08:09:43.832 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:43.842 INFO:teuthology.orchestra.run.smithi149.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 514 kB/s | 80 kB 00:00 2024-04-24T08:09:43.843 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:43.844 INFO:teuthology.orchestra.run.smithi149.stdout:Total 3.6 MB/s | 915 kB 00:00 2024-04-24T08:09:43.878 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:44.059 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:44.060 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:44.426 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs2 2024-04-24T08:09:44.559 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T08:09:44.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-24T08:09:45.205 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:47 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:45.329 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:45.331 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:45.331 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:45.430 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs-devel 2024-04-24T08:09:45.580 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-24T08:09:45.580 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-04-24T08:09:45.948 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T08:09:45.948 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:45.949 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:45.949 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T08:09:45.949 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:45.949 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:46.201 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:46.328 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:46.329 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:09:46.330 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:46.330 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 k 2024-04-24T08:09:46.330 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 137 k 2024-04-24T08:09:46.330 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:46.331 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados-devel 2024-04-24T08:09:46.404 INFO:teuthology.orchestra.run.smithi090.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 411 kB/s | 30 kB 00:00 2024-04-24T08:09:46.405 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:46.405 INFO:teuthology.orchestra.run.smithi090.stdout:Total 401 kB/s | 30 kB 00:00 2024-04-24T08:09:46.405 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:46.410 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:46.410 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:46.431 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:46.431 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:46.560 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:46.702 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:47.125 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:42 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:47.257 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:47.257 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 126 k 2024-04-24T08:09:47.258 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 456 k 2024-04-24T08:09:47.259 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:47.282 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:47.354 INFO:teuthology.orchestra.run.smithi149.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 126 kB 00:00 2024-04-24T08:09:47.355 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:47.355 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.3 MB/s | 126 kB 00:00 2024-04-24T08:09:47.355 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:47.362 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:47.362 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:47.436 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:47.436 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:47.528 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:47.528 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:47.528 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:47.529 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:47.529 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:47.529 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:47.604 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:47.817 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados2 2024-04-24T08:09:48.349 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:48.563 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:48.647 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:48.657 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:48.689 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:48.691 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:48.691 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:48.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librbd1 2024-04-24T08:09:48.973 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs2 2024-04-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:51 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:49.639 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:49.680 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:49.681 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:49.681 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:49.741 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:49.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rados 2024-04-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi149.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:49.868 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:49.869 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:49.870 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:49.964 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs-devel 2024-04-24T08:09:50.478 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:50.563 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:50.604 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:50.605 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:50.606 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:50.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rgw 2024-04-24T08:09:50.690 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:45 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:50.822 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:50.822 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:50.822 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 30 k 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 137 k 2024-04-24T08:09:50.823 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:50.926 INFO:teuthology.orchestra.run.smithi149.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 298 kB/s | 30 kB 00:00 2024-04-24T08:09:50.927 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:50.927 INFO:teuthology.orchestra.run.smithi149.stdout:Total 292 kB/s | 30 kB 00:00 2024-04-24T08:09:50.927 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:50.932 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:50.932 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:50.948 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:50.949 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:51.072 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:51.278 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:51.409 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:53 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:51.494 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:51.618 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-cephfs 2024-04-24T08:09:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:52.007 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:52.007 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:52.007 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:09:52.007 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:52.008 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:52.008 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:52.261 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados2 2024-04-24T08:09:52.363 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:54 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:52.455 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:52.499 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:52.500 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:52.500 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:52.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rbd 2024-04-24T08:09:53.020 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:53.105 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:53.146 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:53.147 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:53.147 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:53.237 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librbd1 2024-04-24T08:09:53.312 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:55 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:53.397 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:53.438 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:53.440 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:09:53.440 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:53.518 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-fuse 2024-04-24T08:09:53.941 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:54.026 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:54.068 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:54.069 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:54.070 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:54.151 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rados 2024-04-24T08:09:54.224 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:56 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:54.349 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:54.350 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:54.350 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:54.350 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 86 k 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 230 k 2024-04-24T08:09:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:54.428 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-04-24T08:09:54.428 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:54.429 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-04-24T08:09:54.429 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:54.437 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:54.437 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:54.509 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:54.509 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:54.660 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:54.830 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:54.886 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:49 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:54.972 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:55.014 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:55.015 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:55.015 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:55.106 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rgw 2024-04-24T08:09:55.498 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:55.810 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:55.811 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:55.811 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:55.811 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:55.811 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:55.811 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:55.828 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:55.915 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:55.957 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:55.958 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:55.958 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:56.039 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-cephfs 2024-04-24T08:09:56.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-mirror 2024-04-24T08:09:56.804 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:51 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:56.894 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:56.936 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:56.937 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:56.937 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:56.944 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:58 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:09:57.044 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rbd 2024-04-24T08:09:57.077 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.0 M 2024-04-24T08:09:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 12 M 2024-04-24T08:09:57.079 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:09:57.209 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 23 MB/s | 3.0 MB 00:00 2024-04-24T08:09:57.210 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:57.210 INFO:teuthology.orchestra.run.smithi090.stdout:Total 23 MB/s | 3.0 MB 00:00 2024-04-24T08:09:57.210 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:09:57.221 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:09:57.221 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:09:57.311 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:09:57.311 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:09:57.790 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:57.803 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:09:57.868 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:57.876 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-24T08:09:57.917 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:57.918 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-04-24T08:09:57.919 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:09:57.937 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:58.005 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-fuse 2024-04-24T08:09:58.752 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:53 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:09:58.880 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:09:58.880 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:58.880 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:09:58.880 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:58.880 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:09:58.881 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 86 k 2024-04-24T08:09:58.882 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 230 k 2024-04-24T08:09:58.882 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:09:58.980 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 878 kB/s | 86 kB 00:00 2024-04-24T08:09:58.980 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:09:58.981 INFO:teuthology.orchestra.run.smithi149.stdout:Total 862 kB/s | 86 kB 00:00 2024-04-24T08:09:58.981 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:09:58.989 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:09:58.990 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:09:59.063 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:09:59.063 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:09:59.230 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:09:59.366 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:59.553 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:09:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-04-24T08:09:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:09:59.553 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:09:59.836 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:09:59.966 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-nbd 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:00.205 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:10:00.500 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-mirror 2024-04-24T08:10:00.734 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:02 ago on Wed 24 Apr 2024 08:07:58 AM UTC. 2024-04-24T08:10:00.872 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:10:00.872 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-24T08:10:00.873 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:10:00.874 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 169 k 2024-04-24T08:10:00.874 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 490 k 2024-04-24T08:10:00.874 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T08:10:00.951 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.2 MB/s | 169 kB 00:00 2024-04-24T08:10:00.952 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:10:00.952 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.1 MB/s | 169 kB 00:00 2024-04-24T08:10:00.952 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T08:10:00.962 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T08:10:00.963 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T08:10:01.038 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T08:10:01.039 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T08:10:01.221 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T08:10:01.276 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:56 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:10:01.383 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:01.407 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:10:01.407 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:10:01.408 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:01.409 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 3.0 M 2024-04-24T08:10:01.409 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 12 M 2024-04-24T08:10:01.409 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:10:01.541 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 23 MB/s | 3.0 MB 00:00 2024-04-24T08:10:01.542 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:10:01.543 INFO:teuthology.orchestra.run.smithi149.stdout:Total 22 MB/s | 3.0 MB 00:00 2024-04-24T08:10:01.543 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:10:01.553 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:10:01.554 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:10:01.642 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:10:01.643 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:10:02.003 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:02.104 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:10:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:10:02.225 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:02.296 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:02.297 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:10:02.297 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:10:02.297 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:10:02.297 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:10:02.297 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:02.656 DEBUG:teuthology.parallel:result is None 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:03.835 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:10:04.159 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-nbd 2024-04-24T08:10:04.914 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:59 ago on Wed 24 Apr 2024 08:08:05 AM UTC. 2024-04-24T08:10:05.041 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-04-24T08:10:05.042 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-04-24T08:10:05.043 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:05.043 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 169 k 2024-04-24T08:10:05.043 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 490 k 2024-04-24T08:10:05.043 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T08:10:05.135 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 169 kB 00:00 2024-04-24T08:10:05.136 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:10:05.136 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.8 MB/s | 169 kB 00:00 2024-04-24T08:10:05.137 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T08:10:05.146 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T08:10:05.146 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T08:10:05.219 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T08:10:05.219 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T08:10:05.394 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T08:10:05.563 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:06.079 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:10:06.395 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T08:10:06.737 DEBUG:teuthology.parallel:result is None 2024-04-24T08:10:06.737 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:10:06.737 INFO:teuthology.packaging:ref: None 2024-04-24T08:10:06.737 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:10:06.737 INFO:teuthology.packaging:branch: None 2024-04-24T08:10:06.737 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:10:06.738 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-24T08:10:06.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-24T08:10:06.884 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:10:06.954 INFO:teuthology.orchestra.run.smithi090.stdout:18.2.0-0.el9 2024-04-24T08:10:06.954 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-04-24T08:10:06.955 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-04-24T08:10:06.956 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-24T08:10:06.956 INFO:teuthology.packaging:ref: None 2024-04-24T08:10:06.957 INFO:teuthology.packaging:tag: v18.2.0 2024-04-24T08:10:06.957 INFO:teuthology.packaging:branch: None 2024-04-24T08:10:06.957 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:10:06.957 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-24T08:10:06.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-24T08:10:07.174 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:10:07.244 INFO:teuthology.orchestra.run.smithi149.stdout:18.2.0-0.el9 2024-04-24T08:10:07.244 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-04-24T08:10:07.244 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-04-24T08:10:07.246 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T08:10:07.246 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:10:07.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:10:07.335 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:10:07.335 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:10:07.425 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T08:10:07.426 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:10:07.426 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:10:07.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:10:07.598 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:10:07.598 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:10:07.682 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:10:07.798 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T08:10:07.799 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:10:07.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:10:07.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:10:08.013 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:10:08.013 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:10:08.097 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:10:08.221 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T08:10:08.221 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:10:08.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:10:08.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:10:08.387 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:10:08.387 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:10:08.477 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:10:08.561 INFO:teuthology.run_tasks:Running task print... 2024-04-24T08:10:08.570 INFO:teuthology.task.print:**** done install task... 2024-04-24T08:10:08.570 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T08:10:08.677 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'} 2024-04-24T08:10:08.677 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-04-24T08:10:08.678 INFO:tasks.cephadm:Cluster fsid is 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:10:08.678 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T08:10:08.678 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-24T08:10:08.679 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi090': '172.21.15.90', 'mon.smithi149': '172.21.15.149'} 2024-04-24T08:10:08.679 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T08:10:08.679 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-04-24T08:10:08.768 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-04-24T08:10:08.857 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-24T08:10:08.857 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:10:09.000 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78823/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3116-g1f81fb39', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3116.g1f81fb39'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 01:00:13.634707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-main-apr17-a654945', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T08:10:09.215 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19b26365dac54ee709df8c5dc2a7b7e89a50aba5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-24T08:10:09.216 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19b26365dac54ee709df8c5dc2a7b7e89a50aba5/centos/9/x86_64/flavors/default/cephadm 2024-04-24T08:10:09.216 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19b26365dac54ee709df8c5dc2a7b7e89a50aba5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:10.106 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Apr 24 08:10 /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:10.107 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19b26365dac54ee709df8c5dc2a7b7e89a50aba5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:11.081 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Apr 24 08:10 /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:11.082 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:11.145 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:10:11.215 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-04-24T08:10:11.215 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-04-24T08:10:11.222 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-04-24T08:10:13.261 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-24T08:10:13.914 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-24T08:10:39.324 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-04-24T08:10:39.325 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T08:10:39.955 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:10:40.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-04-24T08:10:40.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-04-24T08:10:40.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-04-24T08:10:40.272 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-04-24T08:10:40.363 INFO:tasks.cephadm:Writing seed config... 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-24T08:10:40.364 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-24T08:10:40.365 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T08:10:40.366 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T08:10:40.367 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T08:10:40.367 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T08:10:40.367 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:10:40.367 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T08:10:40.424 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 10d437de-0212-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T08:10:40.424 DEBUG:teuthology.orchestra.run.smithi090:mon.smithi090> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service 2024-04-24T08:10:40.469 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T08:10:40.470 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid 10d437de-0212-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:10:40.705 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:10:40.705 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', '10d437de-0212-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-04-24T08:10:40.774 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:10:40.775 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T08:10:40.775 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-04-24T08:10:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:10:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-04-24T08:10:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-04-24T08:10:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:10:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:10:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:10:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:10:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-24T08:10:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-24T08:10:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:10:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-04-24T08:10:40.960 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:10:40.960 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T08:10:40.960 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-04-24T08:10:40.960 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-04-24T08:10:40.979 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:10:40.979 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:10:40.997 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:10:40.997 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:10:41.023 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-24T08:10:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-24T08:10:41.045 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:10:41.045 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-04-24T08:10:41.045 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:10:41.046 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 139895581484032 on /run/cephadm/10d437de-0212-11ef-bc93-c7b262605968.lock 2024-04-24T08:10:41.046 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 139895581484032 acquired on /run/cephadm/10d437de-0212-11ef-bc93-c7b262605968.lock 2024-04-24T08:10:41.048 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-04-24T08:10:41.049 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-04-24T08:10:41.049 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-24T08:10:41.064 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-04-24T08:10:41.064 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-04-24T08:10:41.076 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T08:10:41.077 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-24T08:10:41.077 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1763sec hoplimit 64 pref medium 2024-04-24T08:10:41.088 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T08:10:41.088 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T08:10:41.088 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:10:41.089 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T08:10:41.089 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-04-24T08:10:41.089 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:10:41.091 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-24T08:10:41.091 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-24T08:10:41.093 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T08:10:41.094 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T08:10:41.097 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-24T08:10:42.181 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-04-24T08:10:42.182 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-04-24T08:10:42.182 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T08:10:42.182 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-04-24T08:10:42.182 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-04-24T08:10:42.182 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-04-24T08:10:42.183 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T08:10:43.430 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-04-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-04-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T08:10:44.591 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-04-24T08:10:44.592 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-04-24T08:10:45.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCFvihmkBJQAxAAdRgxSFU87kLcFuaZRfqaEA== 2024-04-24T08:10:46.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCGvihmwd41CRAAVtYfDAyZ4EKVOwACFvF2OA== 2024-04-24T08:10:47.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCHvihmFlCEDhAAyreooMRJQmilEhM2aamVMQ== 2024-04-24T08:10:47.915 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-04-24T08:10:49.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:10:49.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-24T08:10:49.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:10:49.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for smithi090 [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = pacific 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:10:49.003 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-04-24T08:10:50.110 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.smithi090 on 2024-04-24T08:10:50.518 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T08:10:50.761 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T08:10:51.056 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-10d437de-0212-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968.target. 2024-04-24T08:10:51.056 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-10d437de-0212-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968.target. 2024-04-24T08:10:51.427 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090 2024-04-24T08:10:51.427 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service: Unit ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service not loaded. 2024-04-24T08:10:51.739 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968.target.wants/ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service → /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968@.service. 2024-04-24T08:10:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:52 smithi090 podman[24629]: 2024-04-24 08:10:52.05122953 +0000 UTC m=+0.032053185 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-04-24T08:10:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:57 smithi090 podman[24629]: 2024-04-24 08:10:57.187078494 +0000 UTC m=+5.167902196 container init 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T08:10:57.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:57 smithi090 podman[24629]: 2024-04-24 08:10:57.197861551 +0000 UTC m=+5.178685264 container start 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS) 2024-04-24T08:10:57.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:57 smithi090 ceph-mon[24648]: mkfs 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:10:57.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:57 smithi090 ceph-mon[24648]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-24T08:10:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:58 smithi090 ceph-mon[24648]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-24T08:10:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:58 smithi090 ceph-mon[24648]: monmap e1: 1 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} 2024-04-24T08:10:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:58 smithi090 ceph-mon[24648]: fsmap 2024-04-24T08:10:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:58 smithi090 ceph-mon[24648]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:10:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:10:58 smithi090 ceph-mon[24648]: mgrmap e1: no daemons active 2024-04-24T08:11:01.202 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:11:01.202 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T08:11:01.228 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:11:01.229 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:11:01.229 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T08:11:01.229 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:11:01.231 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-04-24T08:11:01.231 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-04-24T08:11:01.422 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:01 smithi090 bash[24629]: 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 2024-04-24T08:11:01.422 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:01 smithi090 systemd[1]: Started Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:11:02.274 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:01 smithi090 ceph-mon[24648]: from='client.? 172.21.15.90:0/430622943' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T08:11:02.619 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi090 (age 4s) 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T08:11:02.620 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.844+0000 7fae64400700 1 Processor -- start 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.845+0000 7fae64400700 1 -- start start 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.845+0000 7fae64400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.845+0000 7fae64400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c107ac0 con 0x7fae5c107170 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.846+0000 7fae61a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.846+0000 7fae61a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54446/0 (socket says 172.21.15.90:54446) 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.846+0000 7fae61a00700 1 -- 172.21.15.90:0/2780560594 learned_addr learned my addr 172.21.15.90:0/2780560594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.846+0000 7fae61a00700 1 -- 172.21.15.90:0/2780560594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c107c00 con 0x7fae5c107170 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.847+0000 7fae61a00700 1 --2- 172.21.15.90:0/2780560594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae500098a0 tx=0x7fae50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8d41c4697de56c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.847+0000 7fae5be00700 1 -- 172.21.15.90:0/2780560594 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae50004180 con 0x7fae5c107170 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.847+0000 7fae5be00700 1 -- 172.21.15.90:0/2780560594 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae500042e0 con 0x7fae5c107170 2024-04-24T08:11:02.621 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.847+0000 7fae5be00700 1 -- 172.21.15.90:0/2780560594 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae500036a0 con 0x7fae5c107170 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 -- 172.21.15.90:0/2780560594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 msgr2=0x7fae5c107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 --2- 172.21.15.90:0/2780560594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae500098a0 tx=0x7fae50009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 -- 172.21.15.90:0/2780560594 shutdown_connections 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 --2- 172.21.15.90:0/2780560594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c107580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 -- 172.21.15.90:0/2780560594 >> 172.21.15.90:0/2780560594 conn(0x7fae5c0ff7e0 msgr2=0x7fae5c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 -- 172.21.15.90:0/2780560594 shutdown_connections 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.848+0000 7fae64400700 1 -- 172.21.15.90:0/2780560594 wait complete. 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.849+0000 7fae64400700 1 Processor -- start 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.849+0000 7fae64400700 1 -- start start 2024-04-24T08:11:02.622 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.849+0000 7fae64400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.849+0000 7fae64400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c19b760 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.850+0000 7fae61a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.850+0000 7fae61a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54462/0 (socket says 172.21.15.90:54462) 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.850+0000 7fae61a00700 1 -- 172.21.15.90:0/430622943 learned_addr learned my addr 172.21.15.90:0/430622943 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.850+0000 7fae61a00700 1 -- 172.21.15.90:0/430622943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009550 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.850+0000 7fae61a00700 1 --2- 172.21.15.90:0/430622943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae5c104c60 tx=0x7fae50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae50003ce0 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50020070 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae50011630 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c19b960 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c19bd80 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae50022030 con 0x7fae5c107170 2024-04-24T08:11:02.623 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.851+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae50025d40 con 0x7fae5c107170 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.852+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c194a00 con 0x7fae5c107170 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.855+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fae50025480 con 0x7fae5c107170 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.934+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fae5c02cc30 con 0x7fae5c107170 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.935+0000 7fae5a000700 1 -- 172.21.15.90:0/430622943 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+318 (secure 0 0 0) 0x7fae50036070 con 0x7fae5c107170 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.936+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 msgr2=0x7fae5c19b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.936+0000 7fae64400700 1 --2- 172.21.15.90:0/430622943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae5c104c60 tx=0x7fae50004750 comp rx=0 tx=0).stop 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.936+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 shutdown_connections 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.936+0000 7fae64400700 1 --2- 172.21.15.90:0/430622943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae5c107170 0x7fae5c19b220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.936+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 >> 172.21.15.90:0/430622943 conn(0x7fae5c0ff7e0 msgr2=0x7fae5c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.937+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 shutdown_connections 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:01.937+0000 7fae64400700 1 -- 172.21.15.90:0/430622943 wait complete. 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-04-24T08:11:02.624 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T08:11:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:03 smithi090 ceph-mon[24648]: from='client.? 172.21.15.90:0/39278848' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:11:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:03 smithi090 ceph-mon[24648]: from='client.? 172.21.15.90:0/39278848' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T08:11:03.965 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:11:03.966 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:11:03.969 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.207+0000 7f6525e00700 1 Processor -- start 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.207+0000 7f6525e00700 1 -- start start 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.207+0000 7f6525e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.207+0000 7f6525e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6520108650 con 0x7f6520107d00 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.208+0000 7f651ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.208+0000 7f651ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54478/0 (socket says 172.21.15.90:54478) 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.208+0000 7f651ea00700 1 -- 172.21.15.90:0/1396099824 learned_addr learned my addr 172.21.15.90:0/1396099824 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.209+0000 7f651ea00700 1 -- 172.21.15.90:0/1396099824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6520108790 con 0x7f6520107d00 2024-04-24T08:11:03.970 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.209+0000 7f651ea00700 1 --2- 172.21.15.90:0/1396099824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f65100098a0 tx=0x7f6510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd79375d0c7588e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:03.971 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.209+0000 7f651e000700 1 -- 172.21.15.90:0/1396099824 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6510004180 con 0x7f6520107d00 2024-04-24T08:11:03.971 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.209+0000 7f651e000700 1 -- 172.21.15.90:0/1396099824 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65100042e0 con 0x7f6520107d00 2024-04-24T08:11:03.971 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.209+0000 7f651e000700 1 -- 172.21.15.90:0/1396099824 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65100036a0 con 0x7f6520107d00 2024-04-24T08:11:03.971 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 -- 172.21.15.90:0/1396099824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 msgr2=0x7f6520108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:03.971 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 --2- 172.21.15.90:0/1396099824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f65100098a0 tx=0x7f6510009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:03.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 -- 172.21.15.90:0/1396099824 shutdown_connections 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 --2- 172.21.15.90:0/1396099824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f6520108110 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 -- 172.21.15.90:0/1396099824 >> 172.21.15.90:0/1396099824 conn(0x7f652007b220 msgr2=0x7f652007b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 -- 172.21.15.90:0/1396099824 shutdown_connections 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.210+0000 7f6525e00700 1 -- 172.21.15.90:0/1396099824 wait complete. 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.211+0000 7f6525e00700 1 Processor -- start 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f6525e00700 1 -- start start 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f6525e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f6525e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652019b750 con 0x7f6520107d00 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f651ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f651ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54486/0 (socket says 172.21.15.90:54486) 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f651ea00700 1 -- 172.21.15.90:0/39278848 learned_addr learned my addr 172.21.15.90:0/39278848 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.212+0000 7f651ea00700 1 -- 172.21.15.90:0/39278848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009550 con 0x7f6520107d00 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f651ea00700 1 --2- 172.21.15.90:0/39278848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6520108a30 tx=0x7f6510004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:03.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6510003ce0 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510020070 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652019b950 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6510011630 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652019bd70 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.213+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6510022030 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.214+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6510025da0 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.214+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6520194a50 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.217+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6510025480 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.293+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f652002ceb0 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.303+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6510025620 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.303+0000 7f6524a00700 1 -- 172.21.15.90:0/39278848 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f6510036020 con 0x7f6520107d00 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 msgr2=0x7f652019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 --2- 172.21.15.90:0/39278848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6520108a30 tx=0x7f6510004750 comp rx=0 tx=0).stop 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 shutdown_connections 2024-04-24T08:11:03.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 --2- 172.21.15.90:0/39278848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6520107d00 0x7f652019b210 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:03.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 >> 172.21.15.90:0/39278848 conn(0x7f652007b220 msgr2=0x7f652018f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:03.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 shutdown_connections 2024-04-24T08:11:03.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:03.305+0000 7f6525e00700 1 -- 172.21.15.90:0/39278848 wait complete. 2024-04-24T08:11:03.977 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-04-24T08:11:04.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:04 smithi090 ceph-mon[24648]: from='client.? 172.21.15.90:0/129345131' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:11:05.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.559+0000 7f5efaa00700 1 Processor -- start 2024-04-24T08:11:05.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.559+0000 7f5efaa00700 1 -- start start 2024-04-24T08:11:05.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.559+0000 7f5efaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.559+0000 7f5efaa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4106460 con 0x7f5ef4105b10 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.560+0000 7f5ef3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.560+0000 7f5ef3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54498/0 (socket says 172.21.15.90:54498) 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.560+0000 7f5ef3e00700 1 -- 172.21.15.90:0/1369990363 learned_addr learned my addr 172.21.15.90:0/1369990363 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.560+0000 7f5ef3e00700 1 -- 172.21.15.90:0/1369990363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef41065a0 con 0x7f5ef4105b10 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.560+0000 7f5ef3e00700 1 --2- 172.21.15.90:0/1369990363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009e80 tx=0x7f5ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8456fcd5448867dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.561+0000 7f5ef3400700 1 -- 172.21.15.90:0/1369990363 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ee400e070 con 0x7f5ef4105b10 2024-04-24T08:11:05.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.561+0000 7f5ef3400700 1 -- 172.21.15.90:0/1369990363 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ee4003e90 con 0x7f5ef4105b10 2024-04-24T08:11:05.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.561+0000 7f5ef3400700 1 -- 172.21.15.90:0/1369990363 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ee4004380 con 0x7f5ef4105b10 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.561+0000 7f5efaa00700 1 -- 172.21.15.90:0/1369990363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 msgr2=0x7f5ef4105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.561+0000 7f5efaa00700 1 --2- 172.21.15.90:0/1369990363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009e80 tx=0x7f5ee400b330 comp rx=0 tx=0).stop 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.562+0000 7f5efaa00700 1 -- 172.21.15.90:0/1369990363 shutdown_connections 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.562+0000 7f5efaa00700 1 --2- 172.21.15.90:0/1369990363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef4105f20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.562+0000 7f5efaa00700 1 -- 172.21.15.90:0/1369990363 >> 172.21.15.90:0/1369990363 conn(0x7f5ef4101180 msgr2=0x7f5ef4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.562+0000 7f5efaa00700 1 -- 172.21.15.90:0/1369990363 shutdown_connections 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.562+0000 7f5efaa00700 1 -- 172.21.15.90:0/1369990363 wait complete. 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.563+0000 7f5efaa00700 1 Processor -- start 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.563+0000 7f5efaa00700 1 -- start start 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5efaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5efaa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef407c180 con 0x7f5ef4105b10 2024-04-24T08:11:05.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5ef3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5ef3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54508/0 (socket says 172.21.15.90:54508) 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5ef3e00700 1 -- 172.21.15.90:0/129345131 learned_addr learned my addr 172.21.15.90:0/129345131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.564+0000 7f5ef3e00700 1 -- 172.21.15.90:0/129345131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009b30 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef3e00700 1 --2- 172.21.15.90:0/129345131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4000f80 tx=0x7f5ee4004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ee400e040 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ee4003cf0 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ee401b930 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef407a290 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef407a710 con 0x7f5ef4105b10 2024-04-24T08:11:05.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5ee4019040 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.565+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5ee401a6d0 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.566+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4062380 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.569+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5ee4029060 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.645+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5ef4107980 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.646+0000 7f5ef1600700 1 -- 172.21.15.90:0/129345131 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f5ee401f070 con 0x7f5ef4105b10 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 msgr2=0x7f5ef407bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 --2- 172.21.15.90:0/129345131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4000f80 tx=0x7f5ee4004920 comp rx=0 tx=0).stop 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 shutdown_connections 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 --2- 172.21.15.90:0/129345131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ef4105b10 0x7f5ef407bc40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 >> 172.21.15.90:0/129345131 conn(0x7f5ef4101180 msgr2=0x7f5ef4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.647+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 shutdown_connections 2024-04-24T08:11:05.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:04.648+0000 7f5efaa00700 1 -- 172.21.15.90:0/129345131 wait complete. 2024-04-24T08:11:05.306 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-04-24T08:11:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:05 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:11:09.522 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[24644]: 2024-04-24T08:11:09.104+0000 7f1037800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:11:09.522 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[24644]: 2024-04-24T08:11:09.104+0000 7f1037800700 -1 mon.smithi090@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T08:11:09.522 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 podman[24893]: 2024-04-24 08:11:09.250267308 +0000 UTC m=+3.590855097 container died 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD) 2024-04-24T08:11:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 podman[24893]: 2024-04-24 08:11:09.631853344 +0000 UTC m=+3.972441136 container cleanup 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20231212) 2024-04-24T08:11:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 bash[24893]: ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090 2024-04-24T08:11:10.233 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:09 smithi090 podman[24907]: 2024-04-24 08:11:09.799372321 +0000 UTC m=+0.533553785 container remove 112c38b86b3693693c6fa345946bb2aff1e2ed22be3bbe60d7ec7f4bfb6347e2 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T08:11:10.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service: Deactivated successfully. 2024-04-24T08:11:10.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 systemd[1]: Stopped Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:11:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service: Consumed 1.917s CPU time. 2024-04-24T08:11:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 systemd[1]: Starting Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:11:10.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 podman[24998]: 2024-04-24 08:11:10.556858419 +0000 UTC m=+0.037086444 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-04-24T08:11:10.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 podman[24998]: 2024-04-24 08:11:10.667089193 +0000 UTC m=+0.147317256 container create ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:11:10.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 podman[24998]: 2024-04-24 08:11:10.973180374 +0000 UTC m=+0.453408456 container init ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T08:11:11.055 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T08:11:11.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:10 smithi090 podman[24998]: 2024-04-24 08:11:10.985079375 +0000 UTC m=+0.465307412 container start ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1) 2024-04-24T08:11:11.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:11:11.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-04-24T08:11:11.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: pidfile_write: ignore empty --pid-file 2024-04-24T08:11:11.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: load: jerasure load: lrc 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Git sha 0 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Compile date 2023-08-03 19:21:13 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: DB SUMMARY 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: DB Session ID: 2GWCOGZCGWYMJ8QMNQ42 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi090/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi090/store.db: 000009.log size: 99548 ; 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:11:11.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.env: 0x562ed92e1720 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.info_log: 0x562f11b9d2c0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.statistics: (nil) 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.use_fsync: 0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:11:11.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.db_log_dir: 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.wal_dir: 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:11:11.297 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.write_buffer_manager: 0x562f10e0e3c0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:11:11.298 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.unordered_write: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.row_cache: None 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.wal_filter: None 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.wal_compression: 0 2024-04-24T08:11:11.299 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:11:11.300 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_open_files: -1 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Compression algorithms supported: 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:11:11.301 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kZSTD supported: 0 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000010 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.merge_operator: 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_filter: None 2024-04-24T08:11:11.302 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562f11b9d3c0) 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_type: 0 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_index_type: 0 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_shortening: 1 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: checksum: 4 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: no_block_cache: 0 2024-04-24T08:11:11.303 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache: 0x562f10eaf350 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_options: 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: capacity : 536870912 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_shard_bits : 4 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: strict_capacity_limit : 0 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_compressed: (nil) 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: persistent_cache: (nil) 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size: 4096 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size_deviation: 10 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_restart_interval: 16 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_block_restart_interval: 1 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_block_size: 4096 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: partition_filters: 0 2024-04-24T08:11:11.304 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: use_delta_encoding: 1 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: filter_policy: bloomfilter 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: whole_key_filtering: 1 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: verify_compression: 0 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: format_version: 5 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_index_compression: 1 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_align: 0 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: prepopulate_block_cache: 0 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression: NoCompression 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:11:11.305 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:11:11.306 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:11:11.306 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.num_levels: 7 2024-04-24T08:11:11.306 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:11:11.306 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:11:11.306 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:11:11.307 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:11:11.308 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:11:11.309 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.table_properties_collectors: 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:11:11.310 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.ttl: 2592000 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 bash[24998]: ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:11:11.311 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 775784da-522a-4526-b09e-6d404a0b9e36 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946271041410, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946271043306, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 338, "table_properties": {"data_size": 92178, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13638, "raw_average_key_size": 49, "raw_value_size": 84735, "raw_average_value_size": 305, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713946271, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "775784da-522a-4526-b09e-6d404a0b9e36", "db_session_id": "2GWCOGZCGWYMJ8QMNQ42", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946271043570, "job": 1, "event": "recovery_finished"} 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi090/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562f10f4c000 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: DB pointer 0x562f10f38000 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** DB Stats ** 2024-04-24T08:11:11.312 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L0 2/0 93.82 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Sum 2/0 93.82 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T08:11:11.313 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative compaction: 0.00 GB write, 11.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval compaction: 0.00 GB write, 11.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:11:11.314 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache BinnedLRUCache@0x562f10eaf350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: starting mon.smithi090 rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi090 fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???) e1 preinit fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).mds e1 new map 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).mds e1 print_map 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: e1 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: legacy client fscid: -1 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: No filesystems configured 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:11:11.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expand map: {default=false} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta from 'false' to 'false' 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expanded map: {default=false} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expand map: {default=info} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta from 'info' to 'info' 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expanded map: {default=info} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expand map: {default=daemon} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expanded map: {default=daemon} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expand map: {default=debug} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta from 'debug' to 'debug' 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: expand_channel_meta expanded map: {default=debug} 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 systemd[1]: Started Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:11:11.316 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-24T08:11:11.317 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: monmap e1: 1 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} 2024-04-24T08:11:11.317 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: fsmap 2024-04-24T08:11:11.317 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:11:11.317 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:11 smithi090 ceph-mon[25016]: mgrmap e1: no daemons active 2024-04-24T08:11:12.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.648+0000 7fe0b3c00700 1 Processor -- start 2024-04-24T08:11:12.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.648+0000 7fe0b3c00700 1 -- start start 2024-04-24T08:11:12.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.649+0000 7fe0b3c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:12.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.649+0000 7fe0b3c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac106460 con 0x7fe0ac105b10 2024-04-24T08:11:12.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.649+0000 7fe0b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:12.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.649+0000 7fe0b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34542/0 (socket says 172.21.15.90:34542) 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.649+0000 7fe0b1200700 1 -- 172.21.15.90:0/4289730047 learned_addr learned my addr 172.21.15.90:0/4289730047 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.650+0000 7fe0b1200700 1 -- 172.21.15.90:0/4289730047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac1065a0 con 0x7fe0ac105b10 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.650+0000 7fe0b1200700 1 --2- 172.21.15.90:0/4289730047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac3470ba9d11e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.651+0000 7fe0abe00700 1 -- 172.21.15.90:0/4289730047 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe09c004180 con 0x7fe0ac105b10 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.651+0000 7fe0abe00700 1 -- 172.21.15.90:0/4289730047 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe09c0042e0 con 0x7fe0ac105b10 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.651+0000 7fe0abe00700 1 -- 172.21.15.90:0/4289730047 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe09c011a50 con 0x7fe0ac105b10 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.651+0000 7fe0b3c00700 1 -- 172.21.15.90:0/4289730047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 msgr2=0x7fe0ac105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.651+0000 7fe0b3c00700 1 --2- 172.21.15.90:0/4289730047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:12.430 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.652+0000 7fe0b3c00700 1 -- 172.21.15.90:0/4289730047 shutdown_connections 2024-04-24T08:11:12.431 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.652+0000 7fe0b3c00700 1 --2- 172.21.15.90:0/4289730047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac105f20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:12.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.652+0000 7fe0b3c00700 1 -- 172.21.15.90:0/4289730047 >> 172.21.15.90:0/4289730047 conn(0x7fe0ac101160 msgr2=0x7fe0ac103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:12.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.652+0000 7fe0b3c00700 1 -- 172.21.15.90:0/4289730047 shutdown_connections 2024-04-24T08:11:12.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.652+0000 7fe0b3c00700 1 -- 172.21.15.90:0/4289730047 wait complete. 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.653+0000 7fe0b3c00700 1 Processor -- start 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.653+0000 7fe0b3c00700 1 -- start start 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.653+0000 7fe0b3c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.654+0000 7fe0b3c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac199d70 con 0x7fe0ac105b10 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.654+0000 7fe0b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.654+0000 7fe0b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34548/0 (socket says 172.21.15.90:34548) 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.654+0000 7fe0b1200700 1 -- 172.21.15.90:0/2204035617 learned_addr learned my addr 172.21.15.90:0/2204035617 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.654+0000 7fe0b1200700 1 -- 172.21.15.90:0/2204035617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c009550 con 0x7fe0ac105b10 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0b1200700 1 --2- 172.21.15.90:0/2204035617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe09c000c00 tx=0x7fe09c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe09c003b90 con 0x7fe0ac105b10 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe09c0248b0 con 0x7fe0ac105b10 2024-04-24T08:11:12.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac199f70 con 0x7fe0ac105b10 2024-04-24T08:11:12.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe09c01a8e0 con 0x7fe0ac105b10 2024-04-24T08:11:12.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac19a390 con 0x7fe0ac105b10 2024-04-24T08:11:12.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.655+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe09c02d3f0 con 0x7fe0ac105b10 2024-04-24T08:11:12.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.656+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe09c019450 con 0x7fe0ac105b10 2024-04-24T08:11:12.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.656+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac062380 con 0x7fe0ac105b10 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.659+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe09c024db0 con 0x7fe0ac105b10 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.735+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe0ac107980 con 0x7fe0ac105b10 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.739+0000 7fe0aa000700 1 -- 172.21.15.90:0/2204035617 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe09c024a20 con 0x7fe0ac105b10 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 msgr2=0x7fe0ac199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 --2- 172.21.15.90:0/2204035617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe09c000c00 tx=0x7fe09c011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 shutdown_connections 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 --2- 172.21.15.90:0/2204035617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0ac105b10 0x7fe0ac199830 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 >> 172.21.15.90:0/2204035617 conn(0x7fe0ac101160 msgr2=0x7fe0ac101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 shutdown_connections 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:11.741+0000 7fe0b3c00700 1 -- 172.21.15.90:0/2204035617 wait complete. 2024-04-24T08:11:12.436 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T08:11:12.461 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:11:12.461 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-04-24T08:11:12.462 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T08:11:12.463 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T08:11:12.464 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-24T08:11:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-10d437de-0212-11ef-bc93-c7b262605968@mgr.smithi090.jsrfxo 2024-04-24T08:11:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-10d437de-0212-11ef-bc93-c7b262605968@mgr.smithi090.jsrfxo.service: Unit ceph-10d437de-0212-11ef-bc93-c7b262605968@mgr.smithi090.jsrfxo.service not loaded. 2024-04-24T08:11:13.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:12 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2204035617' entity='client.admin' 2024-04-24T08:11:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968.target.wants/ceph-10d437de-0212-11ef-bc93-c7b262605968@mgr.smithi090.jsrfxo.service → /etc/systemd/system/ceph-10d437de-0212-11ef-bc93-c7b262605968@.service. 2024-04-24T08:11:13.916 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:11:13.916 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T08:11:13.930 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:11:13.931 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:11:13.931 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T08:11:13.931 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:11:13.953 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:11:13.953 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-04-24T08:11:13.969 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-04-24T08:11:14.955 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:14 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3027615585' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "10d437de-0212-11ef-bc93-c7b262605968", 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:11:15.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:11:15.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:11:15.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:11:15.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:11:15.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:11:15.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:10:57.273961+0000", 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.540+0000 7f9e24a00700 1 Processor -- start 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.540+0000 7f9e24a00700 1 -- start start 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.540+0000 7f9e24a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.541+0000 7f9e24a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20079700 con 0x7f9e2007acc0 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.541+0000 7f9e1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.541+0000 7f9e1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34574/0 (socket says 172.21.15.90:34574) 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.541+0000 7f9e1f400700 1 -- 172.21.15.90:0/3700359457 learned_addr learned my addr 172.21.15.90:0/3700359457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.542+0000 7f9e1f400700 1 -- 172.21.15.90:0/3700359457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e20079840 con 0x7f9e2007acc0 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.542+0000 7f9e1f400700 1 --2- 172.21.15.90:0/3700359457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e100098a0 tx=0x7f9e10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684f6304c69fa00a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:15.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.542+0000 7f9e1e000700 1 -- 172.21.15.90:0/3700359457 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e10004180 con 0x7f9e2007acc0 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.542+0000 7f9e1e000700 1 -- 172.21.15.90:0/3700359457 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e100042e0 con 0x7f9e2007acc0 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.542+0000 7f9e1e000700 1 -- 172.21.15.90:0/3700359457 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e10011a50 con 0x7f9e2007acc0 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 -- 172.21.15.90:0/3700359457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 msgr2=0x7f9e200791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 --2- 172.21.15.90:0/3700359457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e100098a0 tx=0x7f9e10009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 -- 172.21.15.90:0/3700359457 shutdown_connections 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 --2- 172.21.15.90:0/3700359457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e200791c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 -- 172.21.15.90:0/3700359457 >> 172.21.15.90:0/3700359457 conn(0x7f9e20100fb0 msgr2=0x7f9e201033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 -- 172.21.15.90:0/3700359457 shutdown_connections 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.543+0000 7f9e24a00700 1 -- 172.21.15.90:0/3700359457 wait complete. 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.544+0000 7f9e24a00700 1 Processor -- start 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e24a00700 1 -- start start 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e24a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e24a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2019bdf0 con 0x7f9e2007acc0 2024-04-24T08:11:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34580/0 (socket says 172.21.15.90:34580) 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.545+0000 7f9e1f400700 1 -- 172.21.15.90:0/3027615585 learned_addr learned my addr 172.21.15.90:0/3027615585 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e1f400700 1 -- 172.21.15.90:0/3027615585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10009550 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e1f400700 1 --2- 172.21.15.90:0/3027615585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e10003a00 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e10003b60 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2019bff0 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e1001a800 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2019c410 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.546+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9e1001ac60 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.547+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9e100193c0 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.547+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2010bc10 con 0x7f9e2007acc0 2024-04-24T08:11:15.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.550+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9e10024bb0 con 0x7f9e2007acc0 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.630+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9e2019f4a0 con 0x7f9e2007acc0 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.631+0000 7f9e17e00700 1 -- 172.21.15.90:0/3027615585 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f9e10024440 con 0x7f9e2007acc0 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 msgr2=0x7f9e2019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 --2- 172.21.15.90:0/3027615585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e100047c0 comp rx=0 tx=0).stop 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 shutdown_connections 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 --2- 172.21.15.90:0/3027615585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e2007acc0 0x7f9e2019b8b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 >> 172.21.15.90:0/3027615585 conn(0x7f9e20100fb0 msgr2=0x7f9e201026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 shutdown_connections 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:14.633+0000 7f9e24a00700 1 -- 172.21.15.90:0/3027615585 wait complete. 2024-04-24T08:11:15.495 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-04-24T08:11:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:18 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2481787179' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:11:18.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:18.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:18.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "10d437de-0212-11ef-bc93-c7b262605968", 2024-04-24T08:11:18.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:11:18.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:11:18.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:11:19.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:11:19.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:11:19.003 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:11:19.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:11:19.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:10:57.273961+0000", 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.202+0000 7fcc7c400700 1 Processor -- start 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.202+0000 7fcc7c400700 1 -- start start 2024-04-24T08:11:19.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.203+0000 7fcc7c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.203+0000 7fcc7c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740744d0 con 0x7fcc74106650 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.203+0000 7fcc79a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.203+0000 7fcc79a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34596/0 (socket says 172.21.15.90:34596) 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.204+0000 7fcc79a00700 1 -- 172.21.15.90:0/3099445774 learned_addr learned my addr 172.21.15.90:0/3099445774 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.204+0000 7fcc79a00700 1 -- 172.21.15.90:0/3099445774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74074610 con 0x7fcc74106650 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.204+0000 7fcc79a00700 1 --2- 172.21.15.90:0/3099445774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc680098a0 tx=0x7fcc68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddecd057551473ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.204+0000 7fcc73e00700 1 -- 172.21.15.90:0/3099445774 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc68004180 con 0x7fcc74106650 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc73e00700 1 -- 172.21.15.90:0/3099445774 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc680042e0 con 0x7fcc74106650 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc73e00700 1 -- 172.21.15.90:0/3099445774 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc68011a50 con 0x7fcc74106650 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 -- 172.21.15.90:0/3099445774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 msgr2=0x7fcc74108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 --2- 172.21.15.90:0/3099445774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc680098a0 tx=0x7fcc68009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 -- 172.21.15.90:0/3099445774 shutdown_connections 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 --2- 172.21.15.90:0/3099445774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74108a30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:19.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 -- 172.21.15.90:0/3099445774 >> 172.21.15.90:0/3099445774 conn(0x7fcc74100070 msgr2=0x7fcc74102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.205+0000 7fcc7c400700 1 -- 172.21.15.90:0/3099445774 shutdown_connections 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.206+0000 7fcc7c400700 1 -- 172.21.15.90:0/3099445774 wait complete. 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.206+0000 7fcc7c400700 1 Processor -- start 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc7c400700 1 -- start start 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc7c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc7c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741064f0 con 0x7fcc74106650 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc79a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc79a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34602/0 (socket says 172.21.15.90:34602) 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.207+0000 7fcc79a00700 1 -- 172.21.15.90:0/2481787179 learned_addr learned my addr 172.21.15.90:0/2481787179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc79a00700 1 -- 172.21.15.90:0/2481787179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009550 con 0x7fcc74106650 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc79a00700 1 --2- 172.21.15.90:0/2481787179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc68003b90 con 0x7fcc74106650 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc680248b0 con 0x7fcc74106650 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc6801a8e0 con 0x7fcc74106650 2024-04-24T08:11:19.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.208+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc74104660 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.209+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74104ae0 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.209+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcc680193c0 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.209+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcc68019670 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.210+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74109b30 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.212+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fcc68024dd0 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.288+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcc74105140 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.289+0000 7fcc72000700 1 -- 172.21.15.90:0/2481787179 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fcc68024ad0 con 0x7fcc74106650 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.290+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 msgr2=0x7fcc74105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.290+0000 7fcc7c400700 1 --2- 172.21.15.90:0/2481787179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.290+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 shutdown_connections 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.290+0000 7fcc7c400700 1 --2- 172.21.15.90:0/2481787179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc74106650 0x7fcc74105fb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.290+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 >> 172.21.15.90:0/2481787179 conn(0x7fcc74100070 msgr2=0x7fcc74101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.291+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 shutdown_connections 2024-04-24T08:11:19.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:18.291+0000 7fcc7c400700 1 -- 172.21.15.90:0/2481787179 wait complete. 2024-04-24T08:11:19.017 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-04-24T08:11:22.412 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/7688697' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "10d437de-0212-11ef-bc93-c7b262605968", 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:11:22.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:11:22.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:11:22.431 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:11:22.432 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:11:22.433 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:11:22.434 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.435 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:10:57.273961+0000", 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:22.436 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.647+0000 7f2c4aa00700 1 Processor -- start 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.647+0000 7f2c4aa00700 1 -- start start 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.647+0000 7f2c4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.647+0000 7f2c4aa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44106460 con 0x7f2c44105b10 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.648+0000 7f2c43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.648+0000 7f2c43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36192/0 (socket says 172.21.15.90:36192) 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.648+0000 7f2c43e00700 1 -- 172.21.15.90:0/4217495127 learned_addr learned my addr 172.21.15.90:0/4217495127 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.648+0000 7f2c43e00700 1 -- 172.21.15.90:0/4217495127 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c441065a0 con 0x7f2c44105b10 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.648+0000 7f2c43e00700 1 --2- 172.21.15.90:0/4217495127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0e95a812ccdaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.649+0000 7f2c42a00700 1 -- 172.21.15.90:0/4217495127 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c34004180 con 0x7f2c44105b10 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.649+0000 7f2c42a00700 1 -- 172.21.15.90:0/4217495127 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c340042e0 con 0x7f2c44105b10 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.649+0000 7f2c42a00700 1 -- 172.21.15.90:0/4217495127 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c34011a50 con 0x7f2c44105b10 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.649+0000 7f2c4aa00700 1 -- 172.21.15.90:0/4217495127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 msgr2=0x7f2c44105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.649+0000 7f2c4aa00700 1 --2- 172.21.15.90:0/4217495127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:22.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.650+0000 7f2c4aa00700 1 -- 172.21.15.90:0/4217495127 shutdown_connections 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.650+0000 7f2c4aa00700 1 --2- 172.21.15.90:0/4217495127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c44105f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.650+0000 7f2c4aa00700 1 -- 172.21.15.90:0/4217495127 >> 172.21.15.90:0/4217495127 conn(0x7f2c44101160 msgr2=0x7f2c44103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.650+0000 7f2c4aa00700 1 -- 172.21.15.90:0/4217495127 shutdown_connections 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.650+0000 7f2c4aa00700 1 -- 172.21.15.90:0/4217495127 wait complete. 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.651+0000 7f2c4aa00700 1 Processor -- start 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.651+0000 7f2c4aa00700 1 -- start start 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.651+0000 7f2c4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.651+0000 7f2c4aa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419bf70 con 0x7f2c44105b10 2024-04-24T08:11:22.438 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.652+0000 7f2c43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:22.439 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.652+0000 7f2c43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36196/0 (socket says 172.21.15.90:36196) 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.652+0000 7f2c43e00700 1 -- 172.21.15.90:0/7688697 learned_addr learned my addr 172.21.15.90:0/7688697 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.652+0000 7f2c43e00700 1 -- 172.21.15.90:0/7688697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c34009550 con 0x7f2c44105b10 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c43e00700 1 --2- 172.21.15.90:0/7688697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c340119e0 tx=0x7f2c34011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c34003b90 con 0x7f2c44105b10 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c3401a8a0 con 0x7f2c44105b10 2024-04-24T08:11:22.441 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c34019930 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4419c170 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4419c590 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.653+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2c340243f0 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.654+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2c340249e0 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.654+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c44062380 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.657+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2c3401aa10 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.738+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2c44107980 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.739+0000 7f2c40c00700 1 -- 172.21.15.90:0/7688697 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f2c3401abb0 con 0x7f2c44105b10 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 msgr2=0x7f2c4419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 --2- 172.21.15.90:0/7688697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c340119e0 tx=0x7f2c34011ac0 comp rx=0 tx=0).stop 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 shutdown_connections 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 --2- 172.21.15.90:0/7688697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c44105b10 0x7f2c4419ba30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:22.442 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 >> 172.21.15.90:0/7688697 conn(0x7f2c44101160 msgr2=0x7f2c44101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:22.443 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 shutdown_connections 2024-04-24T08:11:22.443 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:21.740+0000 7f2c4aa00700 1 -- 172.21.15.90:0/7688697 wait complete. 2024-04-24T08:11:22.443 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: mgrmap e2: smithi090.jsrfxo(active, starting, since 0.0104028s) 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:23.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:23 smithi090 ceph-mon[25016]: from='mgr.14100 172.21.15.90:0/393458708' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:24.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:24 smithi090 ceph-mon[25016]: mgrmap e3: smithi090.jsrfxo(active, since 1.02162s) 2024-04-24T08:11:25.690 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:25 smithi090 ceph-mon[25016]: mgrmap e4: smithi090.jsrfxo(active, since 2s) 2024-04-24T08:11:25.690 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:25 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1793594615' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:11:25.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:25.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:25.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "10d437de-0212-11ef-bc93-c7b262605968", 2024-04-24T08:11:25.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:11:25.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:11:26.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:11:26.008 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:11:26.008 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:11:26.009 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:11:26.010 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:10:57.273961+0000", 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9373400700 1 Processor -- start 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9373400700 1 -- start start 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9373400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9373400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c10f210 con 0x7f936c0802e0 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9370a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9370a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36260/0 (socket says 172.21.15.90:36260) 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9370a00700 1 -- 172.21.15.90:0/377032403 learned_addr learned my addr 172.21.15.90:0/377032403 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.032+0000 7f9370a00700 1 -- 172.21.15.90:0/377032403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c10f350 con 0x7f936c0802e0 2024-04-24T08:11:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.033+0000 7f9370a00700 1 --2- 172.21.15.90:0/377032403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93600098a0 tx=0x7f9360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce16d705434626 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.033+0000 7f936b400700 1 -- 172.21.15.90:0/377032403 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9360004180 con 0x7f936c0802e0 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.033+0000 7f936b400700 1 -- 172.21.15.90:0/377032403 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93600042e0 con 0x7f936c0802e0 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.033+0000 7f936b400700 1 -- 172.21.15.90:0/377032403 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9360011a50 con 0x7f936c0802e0 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 -- 172.21.15.90:0/377032403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 msgr2=0x7f936c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 --2- 172.21.15.90:0/377032403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93600098a0 tx=0x7f9360009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 -- 172.21.15.90:0/377032403 shutdown_connections 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 --2- 172.21.15.90:0/377032403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c10ecd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 -- 172.21.15.90:0/377032403 >> 172.21.15.90:0/377032403 conn(0x7f936c07b220 msgr2=0x7f936c07d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 -- 172.21.15.90:0/377032403 shutdown_connections 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.034+0000 7f9373400700 1 -- 172.21.15.90:0/377032403 wait complete. 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.035+0000 7f9373400700 1 Processor -- start 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.035+0000 7f9373400700 1 -- start start 2024-04-24T08:11:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9373400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9373400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1a03a0 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9370a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9370a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36266/0 (socket says 172.21.15.90:36266) 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9370a00700 1 -- 172.21.15.90:0/1793594615 learned_addr learned my addr 172.21.15.90:0/1793594615 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9370a00700 1 -- 172.21.15.90:0/1793594615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009550 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.036+0000 7f9370a00700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.037+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9360003b90 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.037+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93600248b0 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.037+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c1a05a0 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.037+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f936001a8e0 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.037+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c1a09c0 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.038+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c062380 con 0x7f936c0802e0 2024-04-24T08:11:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.038+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f9360024440 con 0x7f936c0802e0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.039+0000 7f9369600700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 0x7f935403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.039+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f936004c0a0 con 0x7f936c0802e0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.039+0000 7f936be00700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 0x7f935403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.040+0000 7f936be00700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 0x7f935403a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.046+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f93600233a0 con 0x7f936c0802e0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.334+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f936c1a3b40 con 0x7f936c0802e0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.335+0000 7f9369600700 1 -- 172.21.15.90:0/1793594615 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f93600233a0 con 0x7f936c0802e0 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.337+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 msgr2=0x7f935403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.337+0000 7f9373400700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 0x7f935403a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.337+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 msgr2=0x7f936c19fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.337+0000 7f9373400700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 shutdown_connections 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f93540383f0 0x7f935403a8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 --2- 172.21.15.90:0/1793594615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f936c0802e0 0x7f936c19fe60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 >> 172.21.15.90:0/1793594615 conn(0x7f936c07b220 msgr2=0x7f936c07c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 shutdown_connections 2024-04-24T08:11:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:25.338+0000 7f9373400700 1 -- 172.21.15.90:0/1793594615 wait complete. 2024-04-24T08:11:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-04-24T08:11:27.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:27 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2728183824' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:11:27.686 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:27.686 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:11:27.687 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:27.687 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:11:27.687 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:11:27.687 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:11:27.687 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:11:27.688 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:27.688 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:11:27.688 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:11:27.688 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T08:11:27.688 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:11:27.689 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:11:27.689 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:11:27.689 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:11:27.689 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:11:27.690 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:11:27.690 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.809+0000 7f3eafa00700 1 Processor -- start 2024-04-24T08:11:27.690 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.810+0000 7f3eafa00700 1 -- start start 2024-04-24T08:11:27.690 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.810+0000 7f3eafa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:27.691 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.810+0000 7f3eafa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8106460 con 0x7f3ea8105b10 2024-04-24T08:11:27.691 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.811+0000 7f3ead000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:27.691 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.811+0000 7f3ead000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36274/0 (socket says 172.21.15.90:36274) 2024-04-24T08:11:27.691 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.811+0000 7f3ead000700 1 -- 172.21.15.90:0/3923437810 learned_addr learned my addr 172.21.15.90:0/3923437810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:27.691 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.811+0000 7f3ead000700 1 -- 172.21.15.90:0/3923437810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea81065a0 con 0x7f3ea8105b10 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.812+0000 7f3ead000700 1 --2- 172.21.15.90:0/3923437810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0098a0 tx=0x7f3e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f774a7843d3e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.812+0000 7f3ea7400700 1 -- 172.21.15.90:0/3923437810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9c004180 con 0x7f3ea8105b10 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.812+0000 7f3ea7400700 1 -- 172.21.15.90:0/3923437810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0042e0 con 0x7f3ea8105b10 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.812+0000 7f3ea7400700 1 -- 172.21.15.90:0/3923437810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9c011a50 con 0x7f3ea8105b10 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 -- 172.21.15.90:0/3923437810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 msgr2=0x7f3ea8105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 --2- 172.21.15.90:0/3923437810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0098a0 tx=0x7f3e9c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 -- 172.21.15.90:0/3923437810 shutdown_connections 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 --2- 172.21.15.90:0/3923437810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8105f20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 -- 172.21.15.90:0/3923437810 >> 172.21.15.90:0/3923437810 conn(0x7f3ea8101160 msgr2=0x7f3ea8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 -- 172.21.15.90:0/3923437810 shutdown_connections 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.813+0000 7f3eafa00700 1 -- 172.21.15.90:0/3923437810 wait complete. 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3eafa00700 1 Processor -- start 2024-04-24T08:11:27.694 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3eafa00700 1 -- start start 2024-04-24T08:11:27.695 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3eafa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:27.695 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3eafa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8199d70 con 0x7f3ea8105b10 2024-04-24T08:11:27.695 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3ead000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:27.695 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3ead000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36284/0 (socket says 172.21.15.90:36284) 2024-04-24T08:11:27.695 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.815+0000 7f3ead000700 1 -- 172.21.15.90:0/2728183824 learned_addr learned my addr 172.21.15.90:0/2728183824 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:27.696 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.816+0000 7f3ead000700 1 -- 172.21.15.90:0/2728183824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c009550 con 0x7f3ea8105b10 2024-04-24T08:11:27.696 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.816+0000 7f3ead000700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000c00 tx=0x7f3e9c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:27.696 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.816+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9c003b90 con 0x7f3ea8105b10 2024-04-24T08:11:27.697 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.816+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0248b0 con 0x7f3ea8105b10 2024-04-24T08:11:27.697 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.817+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9c01a8e0 con 0x7f3ea8105b10 2024-04-24T08:11:27.697 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.817+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea8199f70 con 0x7f3ea8105b10 2024-04-24T08:11:27.697 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.817+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea819a390 con 0x7f3ea8105b10 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.818+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f3e9c024a20 con 0x7f3ea8105b10 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.818+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea8062380 con 0x7f3ea8105b10 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.818+0000 7f3ea5600700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 0x7f3e9003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.818+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e9c04c1b0 con 0x7f3ea8105b10 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.819+0000 7f3ea7e00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 0x7f3e9003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:27.698 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.819+0000 7f3ea7e00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 0x7f3e9003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006fd0 tx=0x7f3e94006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:26.824+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3e9c024db0 con 0x7f3ea8105b10 2024-04-24T08:11:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.037+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3ea819d1f0 con 0x7f3ea8105b10 2024-04-24T08:11:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.039+0000 7f3ea5600700 1 -- 172.21.15.90:0/2728183824 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f3e9c018350 con 0x7f3ea8105b10 2024-04-24T08:11:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 msgr2=0x7f3e9003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 0x7f3e9003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006fd0 tx=0x7f3e94006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:27.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 msgr2=0x7f3ea8199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:27.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000c00 tx=0x7f3e9c011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:27.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 shutdown_connections 2024-04-24T08:11:27.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f3e900383a0 0x7f3e9003a850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:27.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 --2- 172.21.15.90:0/2728183824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8105b10 0x7f3ea8199830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:27.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.042+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 >> 172.21.15.90:0/2728183824 conn(0x7f3ea8101160 msgr2=0x7f3ea8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:27.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.043+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 shutdown_connections 2024-04-24T08:11:27.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:27.043+0000 7f3eafa00700 1 -- 172.21.15.90:0/2728183824 wait complete. 2024-04-24T08:11:27.701 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-04-24T08:11:28.910 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:28 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1227669001' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T08:11:29.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.290+0000 7ff667400700 1 Processor -- start 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.290+0000 7ff667400700 1 -- start start 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.290+0000 7ff667400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.290+0000 7ff667400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff660106460 con 0x7ff660105b10 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.291+0000 7ff664a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.291+0000 7ff664a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36300/0 (socket says 172.21.15.90:36300) 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.291+0000 7ff664a00700 1 -- 172.21.15.90:0/1378160948 learned_addr learned my addr 172.21.15.90:0/1378160948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.291+0000 7ff664a00700 1 -- 172.21.15.90:0/1378160948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6601065a0 con 0x7ff660105b10 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.292+0000 7ff664a00700 1 --2- 172.21.15.90:0/1378160948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6540098a0 tx=0x7ff654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9455c080c50855 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.292+0000 7ff65f400700 1 -- 172.21.15.90:0/1378160948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff654004180 con 0x7ff660105b10 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.292+0000 7ff65f400700 1 -- 172.21.15.90:0/1378160948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6540042e0 con 0x7ff660105b10 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.292+0000 7ff65f400700 1 -- 172.21.15.90:0/1378160948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff654011a50 con 0x7ff660105b10 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.293+0000 7ff667400700 1 -- 172.21.15.90:0/1378160948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 msgr2=0x7ff660105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:29.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.293+0000 7ff667400700 1 --2- 172.21.15.90:0/1378160948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6540098a0 tx=0x7ff654009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.293+0000 7ff667400700 1 -- 172.21.15.90:0/1378160948 shutdown_connections 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.293+0000 7ff667400700 1 --2- 172.21.15.90:0/1378160948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff660105f20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.293+0000 7ff667400700 1 -- 172.21.15.90:0/1378160948 >> 172.21.15.90:0/1378160948 conn(0x7ff660101180 msgr2=0x7ff660103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.294+0000 7ff667400700 1 -- 172.21.15.90:0/1378160948 shutdown_connections 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.294+0000 7ff667400700 1 -- 172.21.15.90:0/1378160948 wait complete. 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff667400700 1 Processor -- start 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff667400700 1 -- start start 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff667400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff667400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66019bf80 con 0x7ff660105b10 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff664a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff664a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36316/0 (socket says 172.21.15.90:36316) 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.295+0000 7ff664a00700 1 -- 172.21.15.90:0/1227669001 learned_addr learned my addr 172.21.15.90:0/1227669001 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff664a00700 1 -- 172.21.15.90:0/1227669001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654009550 con 0x7ff660105b10 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff664a00700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff654000c00 tx=0x7ff654011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:29.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff654003b90 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6540248b0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66019c180 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.296+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff65401a8e0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.297+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66019c5a0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.298+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7ff65401aba0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.298+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660062380 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.298+0000 7ff65d600700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 0x7ff64803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.298+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff65404c250 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.298+0000 7ff65fe00700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 0x7ff64803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.299+0000 7ff65fe00700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 0x7ff64803a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.303+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff6540233a0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.574+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ff66019f740 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.629+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff654018350 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.631+0000 7ff65d600700 1 -- 172.21.15.90:0/1227669001 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7ff654024db0 con 0x7ff660105b10 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.634+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 msgr2=0x7ff64803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:29.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.634+0000 7ff667400700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 0x7ff64803a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.634+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 msgr2=0x7ff66019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.634+0000 7ff667400700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff654000c00 tx=0x7ff654011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 shutdown_connections 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7ff6480383f0 0x7ff64803a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 --2- 172.21.15.90:0/1227669001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff660105b10 0x7ff66019ba40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 >> 172.21.15.90:0/1227669001 conn(0x7ff660101180 msgr2=0x7ff660101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 shutdown_connections 2024-04-24T08:11:29.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:28.635+0000 7ff667400700 1 -- 172.21.15.90:0/1227669001 wait complete. 2024-04-24T08:11:29.867 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:29 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1227669001' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T08:11:29.867 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:29 smithi090 ceph-mon[25016]: mgrmap e5: smithi090.jsrfxo(active, since 6s) 2024-04-24T08:11:30.911 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:30 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1570807535' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "smithi090.jsrfxo", 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.019+0000 7f7763e00700 1 Processor -- start 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.019+0000 7f7763e00700 1 -- start start 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.019+0000 7f7763e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.019+0000 7f7763e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0744b0 con 0x7f775c104460 2024-04-24T08:11:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.020+0000 7f7761400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.020+0000 7f7761400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33936/0 (socket says 172.21.15.90:33936) 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.020+0000 7f7761400700 1 -- 172.21.15.90:0/400698118 learned_addr learned my addr 172.21.15.90:0/400698118 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.020+0000 7f7761400700 1 -- 172.21.15.90:0/400698118 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0745f0 con 0x7f775c104460 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.021+0000 7f7761400700 1 --2- 172.21.15.90:0/400698118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f77500098a0 tx=0x7f7750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f82f07cdd0c5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.021+0000 7f775be00700 1 -- 172.21.15.90:0/400698118 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7750004180 con 0x7f775c104460 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.021+0000 7f775be00700 1 -- 172.21.15.90:0/400698118 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77500042e0 con 0x7f775c104460 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.021+0000 7f775be00700 1 -- 172.21.15.90:0/400698118 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7750011a50 con 0x7f775c104460 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 -- 172.21.15.90:0/400698118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 msgr2=0x7f775c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 --2- 172.21.15.90:0/400698118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f77500098a0 tx=0x7f7750009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 -- 172.21.15.90:0/400698118 shutdown_connections 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 --2- 172.21.15.90:0/400698118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c104870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 -- 172.21.15.90:0/400698118 >> 172.21.15.90:0/400698118 conn(0x7f775c100070 msgr2=0x7f775c102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 -- 172.21.15.90:0/400698118 shutdown_connections 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.022+0000 7f7763e00700 1 -- 172.21.15.90:0/400698118 wait complete. 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.023+0000 7f7763e00700 1 Processor -- start 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.023+0000 7f7763e00700 1 -- start start 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.023+0000 7f7763e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.023+0000 7f7763e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19bf70 con 0x7f775c104460 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.024+0000 7f7761400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.024+0000 7f7761400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33948/0 (socket says 172.21.15.90:33948) 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.024+0000 7f7761400700 1 -- 172.21.15.90:0/1570807535 learned_addr learned my addr 172.21.15.90:0/1570807535 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.024+0000 7f7761400700 1 -- 172.21.15.90:0/1570807535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750009550 con 0x7f775c104460 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.024+0000 7f7761400700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7750000c00 tx=0x7f7750011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.025+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7750003b90 con 0x7f775c104460 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.025+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77500248b0 con 0x7f775c104460 2024-04-24T08:11:30.927 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.025+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f775001a8e0 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.025+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c19c170 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.025+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c19c590 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.026+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f775001aba0 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.026+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c195270 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.027+0000 7f775a000700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 0x7f774403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.027+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f775004c2c0 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.027+0000 7f7760a00700 1 -- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 msgr2=0x7f774403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.027+0000 7f7760a00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 0x7f774403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.032+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f77500189a0 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.228+0000 7f7760a00700 1 -- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 msgr2=0x7f774403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.228+0000 7f7760a00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 0x7f774403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.310+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f775c0623b0 con 0x7f775c104460 2024-04-24T08:11:30.928 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.310+0000 7f775a000700 1 -- 172.21.15.90:0/1570807535 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f7750018350 con 0x7f775c104460 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 msgr2=0x7f774403a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 0x7f774403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 msgr2=0x7f775c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7750000c00 tx=0x7f7750011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 shutdown_connections 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f7744038440 0x7f774403a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 --2- 172.21.15.90:0/1570807535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f775c104460 0x7f775c19ba30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.314+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 >> 172.21.15.90:0/1570807535 conn(0x7f775c100070 msgr2=0x7f775c190020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.315+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 shutdown_connections 2024-04-24T08:11:30.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:30.315+0000 7f7763e00700 1 -- 172.21.15.90:0/1570807535 wait complete. 2024-04-24T08:11:30.930 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-24T08:11:30.930 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-04-24T08:11:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: Active manager daemon smithi090.jsrfxo restarted 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: mgrmap e6: smithi090.jsrfxo(active, starting, since 0.00943589s) 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:37.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:37 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: Found migration_current of "None". Setting to last migration. 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:11:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:38 smithi090 ceph-mon[25016]: mgrmap e7: smithi090.jsrfxo(active, since 1.01596s) 2024-04-24T08:11:39.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.548+0000 7f052f000700 1 Processor -- start 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.548+0000 7f052f000700 1 -- start start 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.549+0000 7f052f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.549+0000 7f052f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528079700 con 0x7f052807acc0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.549+0000 7f0527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.549+0000 7f0527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33960/0 (socket says 172.21.15.90:33960) 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.549+0000 7f0527e00700 1 -- 172.21.15.90:0/1319024243 learned_addr learned my addr 172.21.15.90:0/1319024243 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.550+0000 7f0527e00700 1 -- 172.21.15.90:0/1319024243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528079840 con 0x7f052807acc0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.550+0000 7f0527e00700 1 --2- 172.21.15.90:0/1319024243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890a652c6ec95c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.550+0000 7f0526a00700 1 -- 172.21.15.90:0/1319024243 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0518004180 con 0x7f052807acc0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.550+0000 7f0526a00700 1 -- 172.21.15.90:0/1319024243 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05180042e0 con 0x7f052807acc0 2024-04-24T08:11:39.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.550+0000 7f0526a00700 1 -- 172.21.15.90:0/1319024243 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0518011a50 con 0x7f052807acc0 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 -- 172.21.15.90:0/1319024243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 msgr2=0x7f05280791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 --2- 172.21.15.90:0/1319024243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 -- 172.21.15.90:0/1319024243 shutdown_connections 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 --2- 172.21.15.90:0/1319024243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f05280791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 -- 172.21.15.90:0/1319024243 >> 172.21.15.90:0/1319024243 conn(0x7f0528101160 msgr2=0x7f0528103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 -- 172.21.15.90:0/1319024243 shutdown_connections 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.551+0000 7f052f000700 1 -- 172.21.15.90:0/1319024243 wait complete. 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.552+0000 7f052f000700 1 Processor -- start 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.552+0000 7f052f000700 1 -- start start 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f052f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f052f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052819c010 con 0x7f052807acc0 2024-04-24T08:11:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f0527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f0527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33970/0 (socket says 172.21.15.90:33970) 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f0527e00700 1 -- 172.21.15.90:0/1725323279 learned_addr learned my addr 172.21.15.90:0/1725323279 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f0527e00700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518009550 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.553+0000 7f0527e00700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f05180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.554+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0518003a00 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.554+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0518003b60 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.554+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f051801a800 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.554+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052819c210 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.554+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052819c630 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.555+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f051801a960 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.556+0000 7f0524c00700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.556+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f050c03b000 con 0x7f050c038440 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.556+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f051804c100 con 0x7f052807acc0 2024-04-24T08:11:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.556+0000 7f0527400700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.556+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.757+0000 7f0527400700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:31.757+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:32.158+0000 7f0527400700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:32.158+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:32.959+0000 7f0527400700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:32.959+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:34.561+0000 7f0527400700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/833694895 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:34.561+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:37.263+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7f0518024440 con 0x7f052807acc0 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:37.264+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 msgr2=0x7f050c03a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:37.264+0000 7f0524c00700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.266+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f051804d9b0 con 0x7f052807acc0 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.266+0000 7f0524c00700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 0x7f050c03e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.266+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f050c03b000 con 0x7f050c03bd10 2024-04-24T08:11:39.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.269+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 0x7f050c03e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.272+0000 7f0527400700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 0x7f050c03e0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0514003a10 tx=0x7f05140092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.273+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f050c03b000 con 0x7f050c03bd10 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.281+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f052819ce10 con 0x7f050c03bd10 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.281+0000 7f0524c00700 1 -- 172.21.15.90:0/1725323279 <== mgr.14120 v2:172.21.15.90:6800/3041763742 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f052819ce10 con 0x7f050c03bd10 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 msgr2=0x7f050c03e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 0x7f050c03e0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0514003a10 tx=0x7f05140092b0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 msgr2=0x7f052819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f05180047c0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 shutdown_connections 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052807acc0 0x7f052819bad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f050c03bd10 0x7f050c03e0f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 --2- 172.21.15.90:0/1725323279 >> [v2:172.21.15.90:6800/833694895,v1:172.21.15.90:6801/833694895] conn(0x7f050c038440 0x7f050c03a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:39.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 >> 172.21.15.90:0/1725323279 conn(0x7f0528101160 msgr2=0x7f0528102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:39.126 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 shutdown_connections 2024-04-24T08:11:39.126 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:38.282+0000 7f052f000700 1 -- 172.21.15.90:0/1725323279 wait complete. 2024-04-24T08:11:39.126 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-04-24T08:11:39.126 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: [24/Apr/2024:08:11:38] ENGINE Bus STARTING 2024-04-24T08:11:39.676 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:39 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:40.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: [24/Apr/2024:08:11:38] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:11:40.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: [24/Apr/2024:08:11:38] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:11:40.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: [24/Apr/2024:08:11:38] ENGINE Bus STARTED 2024-04-24T08:11:40.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: mgrmap e8: smithi090.jsrfxo(active, since 2s) 2024-04-24T08:11:40.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:40.706 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:40 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.737+0000 7fb933e00700 1 Processor -- start 2024-04-24T08:11:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.737+0000 7fb933e00700 1 -- start start 2024-04-24T08:11:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.738+0000 7fb933e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.738+0000 7fb933e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c079740 con 0x7fb92c07ad00 2024-04-24T08:11:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.738+0000 7fb931400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.738+0000 7fb931400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34064/0 (socket says 172.21.15.90:34064) 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.738+0000 7fb931400700 1 -- 172.21.15.90:0/3426085531 learned_addr learned my addr 172.21.15.90:0/3426085531 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.739+0000 7fb931400700 1 -- 172.21.15.90:0/3426085531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c079880 con 0x7fb92c07ad00 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.739+0000 7fb931400700 1 --2- 172.21.15.90:0/3426085531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb920009e80 tx=0x7fb92000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd05b7105057cf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.739+0000 7fb92be00700 1 -- 172.21.15.90:0/3426085531 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92000e070 con 0x7fb92c07ad00 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.739+0000 7fb92be00700 1 -- 172.21.15.90:0/3426085531 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb920003e90 con 0x7fb92c07ad00 2024-04-24T08:11:40.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.739+0000 7fb92be00700 1 -- 172.21.15.90:0/3426085531 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb920004380 con 0x7fb92c07ad00 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 -- 172.21.15.90:0/3426085531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 msgr2=0x7fb92c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 --2- 172.21.15.90:0/3426085531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb920009e80 tx=0x7fb92000b330 comp rx=0 tx=0).stop 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 -- 172.21.15.90:0/3426085531 shutdown_connections 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 --2- 172.21.15.90:0/3426085531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c079200 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 -- 172.21.15.90:0/3426085531 >> 172.21.15.90:0/3426085531 conn(0x7fb92c101160 msgr2=0x7fb92c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.740+0000 7fb933e00700 1 -- 172.21.15.90:0/3426085531 shutdown_connections 2024-04-24T08:11:40.726 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.741+0000 7fb933e00700 1 -- 172.21.15.90:0/3426085531 wait complete. 2024-04-24T08:11:40.726 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.741+0000 7fb933e00700 1 Processor -- start 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb933e00700 1 -- start start 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb933e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb933e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c19bf90 con 0x7fb92c07ad00 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb931400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb931400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34080/0 (socket says 172.21.15.90:34080) 2024-04-24T08:11:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.742+0000 7fb931400700 1 -- 172.21.15.90:0/3069422742 learned_addr learned my addr 172.21.15.90:0/3069422742 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb931400700 1 -- 172.21.15.90:0/3069422742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920009b30 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb931400700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb92000b7b0 tx=0x7fb920003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92000e040 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb920004800 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92001b710 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c19c190 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.743+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c19c610 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.744+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb920019040 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.744+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92c0623f0 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.745+0000 7fb92a000700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 0x7fb91403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.745+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb92004bce0 con 0x7fb92c07ad00 2024-04-24T08:11:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.745+0000 7fb930a00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 0x7fb91403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.746+0000 7fb930a00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 0x7fb91403a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb918006fd0 tx=0x7fb918006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.751+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb92c0623f0 con 0x7fb92c07ad00 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.982+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fb92c062380 con 0x7fb9140383b0 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:39.998+0000 7fb92a000700 1 -- 172.21.15.90:0/3069422742 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb92c062380 con 0x7fb9140383b0 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 msgr2=0x7fb91403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 0x7fb91403a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb918006fd0 tx=0x7fb918006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 msgr2=0x7fb92c19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb92000b7b0 tx=0x7fb920003d00 comp rx=0 tx=0).stop 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 shutdown_connections 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb9140383b0 0x7fb91403a860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 --2- 172.21.15.90:0/3069422742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb92c07ad00 0x7fb92c19ba50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.001+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 >> 172.21.15.90:0/3069422742 conn(0x7fb92c101160 msgr2=0x7fb92c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.002+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 shutdown_connections 2024-04-24T08:11:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:40.002+0000 7fb933e00700 1 -- 172.21.15.90:0/3069422742 wait complete. 2024-04-24T08:11:41.566 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:41 smithi090 ceph-mon[25016]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:42.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T08:11:42.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.313+0000 7fc0b0c00700 1 Processor -- start 2024-04-24T08:11:42.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.313+0000 7fc0b0c00700 1 -- start start 2024-04-24T08:11:42.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.314+0000 7fc0b0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.314+0000 7fc0b0c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac108650 con 0x7fc0ac107d00 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.314+0000 7fc0a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.314+0000 7fc0a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41442/0 (socket says 172.21.15.90:41442) 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.314+0000 7fc0a9e00700 1 -- 172.21.15.90:0/962492280 learned_addr learned my addr 172.21.15.90:0/962492280 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.315+0000 7fc0a9e00700 1 -- 172.21.15.90:0/962492280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac108790 con 0x7fc0ac107d00 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.315+0000 7fc0a9e00700 1 --2- 172.21.15.90:0/962492280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0098a0 tx=0x7fc09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5795ca3977c9418f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:42.284 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.315+0000 7fc0a9400700 1 -- 172.21.15.90:0/962492280 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc09c004180 con 0x7fc0ac107d00 2024-04-24T08:11:42.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.315+0000 7fc0a9400700 1 -- 172.21.15.90:0/962492280 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c0042e0 con 0x7fc0ac107d00 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.315+0000 7fc0a9400700 1 -- 172.21.15.90:0/962492280 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc09c011a50 con 0x7fc0ac107d00 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 -- 172.21.15.90:0/962492280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 msgr2=0x7fc0ac108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/962492280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0098a0 tx=0x7fc09c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 -- 172.21.15.90:0/962492280 shutdown_connections 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/962492280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac108110 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 -- 172.21.15.90:0/962492280 >> 172.21.15.90:0/962492280 conn(0x7fc0ac07b220 msgr2=0x7fc0ac07b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 -- 172.21.15.90:0/962492280 shutdown_connections 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.316+0000 7fc0b0c00700 1 -- 172.21.15.90:0/962492280 wait complete. 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.317+0000 7fc0b0c00700 1 Processor -- start 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.317+0000 7fc0b0c00700 1 -- start start 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0b0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0b0c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac19bfb0 con 0x7fc0ac107d00 2024-04-24T08:11:42.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41448/0 (socket says 172.21.15.90:41448) 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0a9e00700 1 -- 172.21.15.90:0/3305372982 learned_addr learned my addr 172.21.15.90:0/3305372982 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.318+0000 7fc0a9e00700 1 -- 172.21.15.90:0/3305372982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c009550 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0a9e00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc09c000c00 tx=0x7fc09c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc09c003b90 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c0248b0 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc09c01a8e0 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac19c1b0 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.319+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac19c5d0 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.320+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fc09c024a20 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.320+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac062380 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.321+0000 7fc0a2a00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 0x7fc09003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.321+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc09c04c5a0 con 0x7fc0ac107d00 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.321+0000 7fc0a3e00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 0x7fc09003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:42.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.322+0000 7fc0a3e00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 0x7fc09003a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc094006fd0 tx=0x7fc094006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.326+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc09c024db0 con 0x7fc0ac107d00 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.543+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0ac106c50 con 0x7fc090038440 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.545+0000 7fc0a2a00700 1 -- 172.21.15.90:0/3305372982 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc0ac106c50 con 0x7fc090038440 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 msgr2=0x7fc09003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 0x7fc09003a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc094006fd0 tx=0x7fc094006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 msgr2=0x7fc0ac19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc09c000c00 tx=0x7fc09c011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 shutdown_connections 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fc090038440 0x7fc09003a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 --2- 172.21.15.90:0/3305372982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0ac107d00 0x7fc0ac19ba70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 >> 172.21.15.90:0/3305372982 conn(0x7fc0ac07b220 msgr2=0x7fc0ac105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 shutdown_connections 2024-04-24T08:11:42.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:41.548+0000 7fc0b0c00700 1 -- 172.21.15.90:0/3305372982 wait complete. 2024-04-24T08:11:42.292 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-04-24T08:11:42.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:42 smithi090 ceph-mon[25016]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:44.059 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.887+0000 7fb14cc00700 1 Processor -- start 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.887+0000 7fb14cc00700 1 -- start start 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.888+0000 7fb14cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.888+0000 7fb14cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb148074510 con 0x7fb148104400 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.888+0000 7fb145e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.888+0000 7fb145e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41456/0 (socket says 172.21.15.90:41456) 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.888+0000 7fb145e00700 1 -- 172.21.15.90:0/1206725593 learned_addr learned my addr 172.21.15.90:0/1206725593 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.889+0000 7fb145e00700 1 -- 172.21.15.90:0/1206725593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148074650 con 0x7fb148104400 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.889+0000 7fb145e00700 1 --2- 172.21.15.90:0/1206725593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3fdfaf97eecf1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.889+0000 7fb145400700 1 -- 172.21.15.90:0/1206725593 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb138004180 con 0x7fb148104400 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.889+0000 7fb145400700 1 -- 172.21.15.90:0/1206725593 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1380042e0 con 0x7fb148104400 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.889+0000 7fb145400700 1 -- 172.21.15.90:0/1206725593 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb138011a50 con 0x7fb148104400 2024-04-24T08:11:44.061 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 -- 172.21.15.90:0/1206725593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 msgr2=0x7fb148104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:44.063 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 --2- 172.21.15.90:0/1206725593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 -- 172.21.15.90:0/1206725593 shutdown_connections 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 --2- 172.21.15.90:0/1206725593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb148104810 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 -- 172.21.15.90:0/1206725593 >> 172.21.15.90:0/1206725593 conn(0x7fb148100070 msgr2=0x7fb148102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 -- 172.21.15.90:0/1206725593 shutdown_connections 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.890+0000 7fb14cc00700 1 -- 172.21.15.90:0/1206725593 wait complete. 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.891+0000 7fb14cc00700 1 Processor -- start 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.891+0000 7fb14cc00700 1 -- start start 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb14cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb14cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14819bf60 con 0x7fb148104400 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb145e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb145e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41458/0 (socket says 172.21.15.90:41458) 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb145e00700 1 -- 172.21.15.90:0/2026589586 learned_addr learned my addr 172.21.15.90:0/2026589586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.892+0000 7fb145e00700 1 -- 172.21.15.90:0/2026589586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009550 con 0x7fb148104400 2024-04-24T08:11:44.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb145e00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb138000c00 tx=0x7fb138011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:44.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb138003b90 con 0x7fb148104400 2024-04-24T08:11:44.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1380248b0 con 0x7fb148104400 2024-04-24T08:11:44.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14819c160 con 0x7fb148104400 2024-04-24T08:11:44.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb13801a8e0 con 0x7fb148104400 2024-04-24T08:11:44.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.893+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14819c580 con 0x7fb148104400 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.894+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148195260 con 0x7fb148104400 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.895+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb138024a20 con 0x7fb148104400 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.895+0000 7fb13ea00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 0x7fb12c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.895+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb13804c450 con 0x7fb148104400 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.895+0000 7fb13fe00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 0x7fb12c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.896+0000 7fb13fe00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 0x7fb12c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb130006fd0 tx=0x7fb130006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:42.900+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb138018b80 con 0x7fb148104400 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.117+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb148061190 con 0x7fb12c038440 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.375+0000 7fb13ea00700 1 -- 172.21.15.90:0/2026589586 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb148061190 con 0x7fb12c038440 2024-04-24T08:11:44.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 msgr2=0x7fb12c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 0x7fb12c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb130006fd0 tx=0x7fb130006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 msgr2=0x7fb14819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb138000c00 tx=0x7fb138011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 shutdown_connections 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fb12c038440 0x7fb12c03a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 --2- 172.21.15.90:0/2026589586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb148104400 0x7fb14819ba20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 >> 172.21.15.90:0/2026589586 conn(0x7fb148100070 msgr2=0x7fb14818ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 shutdown_connections 2024-04-24T08:11:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:43.378+0000 7fb14cc00700 1 -- 172.21.15.90:0/2026589586 wait complete. 2024-04-24T08:11:44.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:44 smithi090 ceph-mon[25016]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:44.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:44 smithi090 ceph-mon[25016]: Generating ssh key... 2024-04-24T08:11:44.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:44 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:44.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:44 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:45.544 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBfZAJ6yYXnBJwi1hSs1xNa4ok8VaY7rm226Gc3B3UlYZ6Q/ZYatB2q5fkk5rMH7IV/BRhOMexJcfdkadwk7D+tWVPtVCFHHk3MqSW2DpUzBqiGTbQ/hNOPNbyskoJ7m9BuDzq+pQ1zQ/0eKHrFf1N6AkPENn54Hx9FvkJWOmyo7bsUx5veX8oECv9ChX7yEJQyyZj+sJyQZZxiM1wvfq87VsI6y4AHdzSH2f/HqFGp2ZCLUscZ2gBigydGXcJg6re0KKcVlomWNPe/JUJdGoACQCbDkT5nZY5BRoqrzu3KzRAeN0RjHAfOLhQieLEC44sIZ9UKKJqDDGQZFVEMaEHPHRxN2y5Td8rigE8z4dFcSLlZRN0btMdxfISjb4JrT/vRUta+O6RTi45A89FqVjknqyn5Lek0QIzhFov3JKnGOIPqW+jXxLbHTl0ciKXKdip/S7y77TB0jXbCeUwOcXiN6Ob5+IHyJW0QrzeqsrSjYVB5RAk1CGjrHH0TDGAgUs= ceph-10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.634+0000 7fe5e1000700 1 Processor -- start 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.634+0000 7fe5e1000700 1 -- start start 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.635+0000 7fe5e1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.635+0000 7fe5e1000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc10ca60 con 0x7fe5dc10c110 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.635+0000 7fe5d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.635+0000 7fe5d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41474/0 (socket says 172.21.15.90:41474) 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.635+0000 7fe5d9e00700 1 -- 172.21.15.90:0/1303611731 learned_addr learned my addr 172.21.15.90:0/1303611731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.636+0000 7fe5d9e00700 1 -- 172.21.15.90:0/1303611731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc10cba0 con 0x7fe5dc10c110 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.636+0000 7fe5d9e00700 1 --2- 172.21.15.90:0/1303611731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a0a7d33e5229fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.636+0000 7fe5d8a00700 1 -- 172.21.15.90:0/1303611731 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5cc004180 con 0x7fe5dc10c110 2024-04-24T08:11:45.545 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.636+0000 7fe5d8a00700 1 -- 172.21.15.90:0/1303611731 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5cc0042e0 con 0x7fe5dc10c110 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5d8a00700 1 -- 172.21.15.90:0/1303611731 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5cc011a50 con 0x7fe5dc10c110 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 -- 172.21.15.90:0/1303611731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 msgr2=0x7fe5dc10c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 --2- 172.21.15.90:0/1303611731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 -- 172.21.15.90:0/1303611731 shutdown_connections 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 --2- 172.21.15.90:0/1303611731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc10c520 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 -- 172.21.15.90:0/1303611731 >> 172.21.15.90:0/1303611731 conn(0x7fe5dc07b220 msgr2=0x7fe5dc07d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 -- 172.21.15.90:0/1303611731 shutdown_connections 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.637+0000 7fe5e1000700 1 -- 172.21.15.90:0/1303611731 wait complete. 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.638+0000 7fe5e1000700 1 Processor -- start 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5e1000700 1 -- start start 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5e1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5e1000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1a0380 con 0x7fe5dc10c110 2024-04-24T08:11:45.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41480/0 (socket says 172.21.15.90:41480) 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.639+0000 7fe5d9e00700 1 -- 172.21.15.90:0/3763817321 learned_addr learned my addr 172.21.15.90:0/3763817321 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5d9e00700 1 -- 172.21.15.90:0/3763817321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc009550 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5d9e00700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5cc003b90 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5cc0248b0 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5cc01a8e0 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc1a0580 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.640+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc1a09a0 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.642+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc062380 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.642+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fe5cc024a20 con 0x7fe5dc10c110 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.643+0000 7fe5d2a00700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 0x7fe5c003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:45.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.643+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe5cc04c660 con 0x7fe5dc10c110 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.643+0000 7fe5d9400700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 0x7fe5c003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.644+0000 7fe5d9400700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 0x7fe5c003a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.647+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe5cc018530 con 0x7fe5dc10c110 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.863+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5dc07c6d0 con 0x7fe5c00383f0 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.865+0000 7fe5d2a00700 1 -- 172.21.15.90:0/3763817321 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe5dc07c6d0 con 0x7fe5c00383f0 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 msgr2=0x7fe5c003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 0x7fe5c003a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c8006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 msgr2=0x7fe5dc19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc011ab0 comp rx=0 tx=0).stop 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 shutdown_connections 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fe5c00383f0 0x7fe5c003a8a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 --2- 172.21.15.90:0/3763817321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5dc10c110 0x7fe5dc19fe40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.868+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 >> 172.21.15.90:0/3763817321 conn(0x7fe5dc07b220 msgr2=0x7fe5dc07bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.869+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 shutdown_connections 2024-04-24T08:11:45.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:44.869+0000 7fe5e1000700 1 -- 172.21.15.90:0/3763817321 wait complete. 2024-04-24T08:11:45.549 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T08:11:45.549 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T08:11:45.549 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-04-24T08:11:46.100 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:45 smithi090 ceph-mon[25016]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:47.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:47 smithi090 ceph-mon[25016]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:48.215 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:48 smithi090 ceph-mon[25016]: Deploying cephadm binary to smithi090 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.175+0000 7f62a0400700 1 Processor -- start 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.175+0000 7f62a0400700 1 -- start start 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.175+0000 7f62a0400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.175+0000 7f62a0400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298079700 con 0x7f629807acc0 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.176+0000 7f629da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.176+0000 7f629da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41496/0 (socket says 172.21.15.90:41496) 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.176+0000 7f629da00700 1 -- 172.21.15.90:0/3280174491 learned_addr learned my addr 172.21.15.90:0/3280174491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.176+0000 7f629da00700 1 -- 172.21.15.90:0/3280174491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298079840 con 0x7f629807acc0 2024-04-24T08:11:51.207 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.176+0000 7f629da00700 1 --2- 172.21.15.90:0/3280174491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f628c009e80 tx=0x7f628c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d809200b9e0c93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.177+0000 7f6297e00700 1 -- 172.21.15.90:0/3280174491 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f628c00e070 con 0x7f629807acc0 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.177+0000 7f6297e00700 1 -- 172.21.15.90:0/3280174491 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f628c003e90 con 0x7f629807acc0 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.177+0000 7f6297e00700 1 -- 172.21.15.90:0/3280174491 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f628c004380 con 0x7f629807acc0 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.177+0000 7f62a0400700 1 -- 172.21.15.90:0/3280174491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 msgr2=0x7f62980791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.177+0000 7f62a0400700 1 --2- 172.21.15.90:0/3280174491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f628c009e80 tx=0x7f628c00b330 comp rx=0 tx=0).stop 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.178+0000 7f62a0400700 1 -- 172.21.15.90:0/3280174491 shutdown_connections 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.178+0000 7f62a0400700 1 --2- 172.21.15.90:0/3280174491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f62980791c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.178+0000 7f62a0400700 1 -- 172.21.15.90:0/3280174491 >> 172.21.15.90:0/3280174491 conn(0x7f6298101160 msgr2=0x7f6298103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.178+0000 7f62a0400700 1 -- 172.21.15.90:0/3280174491 shutdown_connections 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.178+0000 7f62a0400700 1 -- 172.21.15.90:0/3280174491 wait complete. 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f62a0400700 1 Processor -- start 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f62a0400700 1 -- start start 2024-04-24T08:11:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f62a0400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f62a0400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629819bf70 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f629da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f629da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41500/0 (socket says 172.21.15.90:41500) 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.179+0000 7f629da00700 1 -- 172.21.15.90:0/3471994609 learned_addr learned my addr 172.21.15.90:0/3471994609 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f629da00700 1 -- 172.21.15.90:0/3471994609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c009b30 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f629da00700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f628c009530 tx=0x7f628c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f628c00e040 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f628c004850 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f628c01b680 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.180+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629819c170 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.181+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629819c5f0 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.182+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f628c019040 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.182+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62980623f0 con 0x7f629807acc0 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.182+0000 7f6296000700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 0x7f628003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.182+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f628c04bba0 con 0x7f629807acc0 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.182+0000 7f629d000700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 0x7f628003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.183+0000 7f629d000700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 0x7f628003a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6284006fd0 tx=0x7f6284006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.188+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f62980623f0 con 0x7f629807acc0 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:46.407+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) v1 -- 0x7f6298062380 con 0x7f6280038440 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.456+0000 7f6296000700 1 -- 172.21.15.90:0/3471994609 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f6298062380 con 0x7f6280038440 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.460+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 msgr2=0x7f628003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.460+0000 7f62a0400700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 0x7f628003a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6284006fd0 tx=0x7f6284006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.460+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 msgr2=0x7f629819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.460+0000 7f62a0400700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f628c009530 tx=0x7f628c003d00 comp rx=0 tx=0).stop 2024-04-24T08:11:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 shutdown_connections 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6280038440 0x7f628003a8f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 --2- 172.21.15.90:0/3471994609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f629807acc0 0x7f629819ba30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 >> 172.21.15.90:0/3471994609 conn(0x7f6298101160 msgr2=0x7f6298101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 shutdown_connections 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:50.461+0000 7f62a0400700 1 -- 172.21.15.90:0/3471994609 wait complete. 2024-04-24T08:11:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying mon service with default placement... 2024-04-24T08:11:51.751 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:51 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:51.751 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:51 smithi090 ceph-mon[25016]: Added host smithi090 2024-04-24T08:11:51.751 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:51 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.816+0000 7f994fe00700 1 Processor -- start 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.816+0000 7f994fe00700 1 -- start start 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.817+0000 7f994fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.817+0000 7f994fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950108a20 con 0x7f9950074490 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.817+0000 7f994ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.817+0000 7f994ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43270/0 (socket says 172.21.15.90:43270) 2024-04-24T08:11:52.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.817+0000 7f994ea00700 1 -- 172.21.15.90:0/3469197556 learned_addr learned my addr 172.21.15.90:0/3469197556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.818+0000 7f994ea00700 1 -- 172.21.15.90:0/3469197556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950108b60 con 0x7f9950074490 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.818+0000 7f994ea00700 1 --2- 172.21.15.90:0/3469197556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9940009e80 tx=0x7f994000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9523da2e75054177 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.818+0000 7f994d600700 1 -- 172.21.15.90:0/3469197556 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f994000e070 con 0x7f9950074490 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.818+0000 7f994d600700 1 -- 172.21.15.90:0/3469197556 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9940003e90 con 0x7f9950074490 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.818+0000 7f994d600700 1 -- 172.21.15.90:0/3469197556 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9940004380 con 0x7f9950074490 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 -- 172.21.15.90:0/3469197556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 msgr2=0x7f99501084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 --2- 172.21.15.90:0/3469197556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9940009e80 tx=0x7f994000b330 comp rx=0 tx=0).stop 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 -- 172.21.15.90:0/3469197556 shutdown_connections 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 --2- 172.21.15.90:0/3469197556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9950074490 0x7f99501084e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 -- 172.21.15.90:0/3469197556 >> 172.21.15.90:0/3469197556 conn(0x7f99500ffeb0 msgr2=0x7f99501022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 -- 172.21.15.90:0/3469197556 shutdown_connections 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.819+0000 7f994fe00700 1 -- 172.21.15.90:0/3469197556 wait complete. 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.820+0000 7f994fe00700 1 Processor -- start 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.820+0000 7f994fe00700 1 -- start start 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.820+0000 7f994fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:52.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.820+0000 7f994fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019c160 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f994ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f994ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43272/0 (socket says 172.21.15.90:43272) 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f994ea00700 1 -- 172.21.15.90:0/1606195980 learned_addr learned my addr 172.21.15.90:0/1606195980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f994ea00700 1 -- 172.21.15.90:0/1606195980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009b30 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f994ea00700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9940009530 tx=0x7f9940004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f994000e040 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.822+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995019c360 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.821+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9940003c30 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.822+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f994001b740 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.822+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995019efc0 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.823+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f9940019040 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.823+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500623b0 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.823+0000 7f9947400700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 0x7f993403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.823+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f994004bd20 con 0x7f995019b810 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.823+0000 7f994e000700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 0x7f993403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:52.917 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.824+0000 7f994e000700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 0x7f993403a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f993c006fd0 tx=0x7f993c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:51.829+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9940022b90 con 0x7f995019b810 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.059+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f9950101340 con 0x7f9934038440 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.065+0000 7f9947400700 1 -- 172.21.15.90:0/1606195980 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9950101340 con 0x7f9934038440 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 msgr2=0x7f993403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 0x7f993403a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f993c006fd0 tx=0x7f993c006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 msgr2=0x7f995019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9940009530 tx=0x7f9940004920 comp rx=0 tx=0).stop 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 shutdown_connections 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f9934038440 0x7f993403a8f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 --2- 172.21.15.90:0/1606195980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f995019b810 0x7f995019bc20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 >> 172.21.15.90:0/1606195980 conn(0x7f99500ffeb0 msgr2=0x7f9950100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 shutdown_connections 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:52.068+0000 7f994fe00700 1 -- 172.21.15.90:0/1606195980 wait complete. 2024-04-24T08:11:52.918 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying mgr service with default placement... 2024-04-24T08:11:53.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:53 smithi090 ceph-mon[25016]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:53.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:53 smithi090 ceph-mon[25016]: Saving service mon spec with placement count:5 2024-04-24T08:11:53.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:53 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:54.525 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:54 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:54.525 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:54 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:54.526 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:54 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:54.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.926+0000 7fbdd6c00700 1 Processor -- start 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.926+0000 7fbdd6c00700 1 -- start start 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.926+0000 7fbdd6c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.926+0000 7fbdd6c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd0079720 con 0x7fbdd007ace0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.927+0000 7fbdcfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.927+0000 7fbdcfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43278/0 (socket says 172.21.15.90:43278) 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.927+0000 7fbdcfe00700 1 -- 172.21.15.90:0/1066179479 learned_addr learned my addr 172.21.15.90:0/1066179479 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.927+0000 7fbdcfe00700 1 -- 172.21.15.90:0/1066179479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0079860 con 0x7fbdd007ace0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.928+0000 7fbdcfe00700 1 --2- 172.21.15.90:0/1066179479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00098a0 tx=0x7fbdc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df246d17a45e4651 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.928+0000 7fbdcea00700 1 -- 172.21.15.90:0/1066179479 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc0004180 con 0x7fbdd007ace0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.928+0000 7fbdcea00700 1 -- 172.21.15.90:0/1066179479 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00042e0 con 0x7fbdd007ace0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.928+0000 7fbdcea00700 1 -- 172.21.15.90:0/1066179479 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc0011af0 con 0x7fbdd007ace0 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 -- 172.21.15.90:0/1066179479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 msgr2=0x7fbdd00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:54.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/1066179479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00098a0 tx=0x7fbdc0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 -- 172.21.15.90:0/1066179479 shutdown_connections 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/1066179479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd00791e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 -- 172.21.15.90:0/1066179479 >> 172.21.15.90:0/1066179479 conn(0x7fbdd0101160 msgr2=0x7fbdd0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 -- 172.21.15.90:0/1066179479 shutdown_connections 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.929+0000 7fbdd6c00700 1 -- 172.21.15.90:0/1066179479 wait complete. 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.930+0000 7fbdd6c00700 1 Processor -- start 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdd6c00700 1 -- start start 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdd6c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdd6c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd019c000 con 0x7fbdd007ace0 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdcfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdcfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43282/0 (socket says 172.21.15.90:43282) 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.931+0000 7fbdcfe00700 1 -- 172.21.15.90:0/3083505461 learned_addr learned my addr 172.21.15.90:0/3083505461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdcfe00700 1 -- 172.21.15.90:0/3083505461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc0009550 con 0x7fbdd007ace0 2024-04-24T08:11:54.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdcfe00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc0003a70 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0003bd0 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc0023700 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd019c200 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.932+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd019c620 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.934+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fbdc001ab70 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.934+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd0062380 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.934+0000 7fbdccc00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 0x7fbdb403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.934+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbdc004c090 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.934+0000 7fbdcf400700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 0x7fbdb403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.935+0000 7fbdcf400700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 0x7fbdb403a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc006fd0 tx=0x7fbdbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:53.939+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbdc001a430 con 0x7fbdd007ace0 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.167+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdd0102670 con 0x7fbdb4038440 2024-04-24T08:11:54.857 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.172+0000 7fbdccc00700 1 -- 172.21.15.90:0/3083505461 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbdd0102670 con 0x7fbdb4038440 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.174+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 msgr2=0x7fbdb403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.174+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 0x7fbdb403a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc006fd0 tx=0x7fbdbc006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.174+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 msgr2=0x7fbdd019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.174+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0011b50 comp rx=0 tx=0).stop 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 shutdown_connections 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbdb4038440 0x7fbdb403a8f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 --2- 172.21.15.90:0/3083505461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd007ace0 0x7fbdd019bac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 >> 172.21.15.90:0/3083505461 conn(0x7fbdd0101160 msgr2=0x7fbdd0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 shutdown_connections 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:54.175+0000 7fbdd6c00700 1 -- 172.21.15.90:0/3083505461 wait complete. 2024-04-24T08:11:54.858 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying crash service with default placement... 2024-04-24T08:11:55.648 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:55 smithi090 ceph-mon[25016]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:55.648 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:55 smithi090 ceph-mon[25016]: Saving service mgr spec with placement count:2 2024-04-24T08:11:56.747 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:56 smithi090 ceph-mon[25016]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:56.747 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:56 smithi090 ceph-mon[25016]: Saving service crash spec with placement * 2024-04-24T08:11:56.747 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:56 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:56.747 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:56 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:56.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-24T08:11:56.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.403+0000 7fa31ac00700 1 Processor -- start 2024-04-24T08:11:56.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.403+0000 7fa31ac00700 1 -- start start 2024-04-24T08:11:56.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.404+0000 7fa31ac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:56.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.404+0000 7fa31ac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa314106460 con 0x7fa314105b10 2024-04-24T08:11:56.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.404+0000 7fa313e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:56.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.404+0000 7fa313e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43296/0 (socket says 172.21.15.90:43296) 2024-04-24T08:11:56.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.404+0000 7fa313e00700 1 -- 172.21.15.90:0/2638510868 learned_addr learned my addr 172.21.15.90:0/2638510868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:56.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.405+0000 7fa313e00700 1 -- 172.21.15.90:0/2638510868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3141065a0 con 0x7fa314105b10 2024-04-24T08:11:56.760 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.405+0000 7fa313e00700 1 --2- 172.21.15.90:0/2638510868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098a0 tx=0x7fa304009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50611923ebf00974 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:56.760 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.405+0000 7fa312a00700 1 -- 172.21.15.90:0/2638510868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa304004180 con 0x7fa314105b10 2024-04-24T08:11:56.760 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.405+0000 7fa312a00700 1 -- 172.21.15.90:0/2638510868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa3040042e0 con 0x7fa314105b10 2024-04-24T08:11:56.760 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.405+0000 7fa312a00700 1 -- 172.21.15.90:0/2638510868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa304011af0 con 0x7fa314105b10 2024-04-24T08:11:56.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 -- 172.21.15.90:0/2638510868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 msgr2=0x7fa314105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:56.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2638510868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098a0 tx=0x7fa304009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:56.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 -- 172.21.15.90:0/2638510868 shutdown_connections 2024-04-24T08:11:56.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2638510868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa314105f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:56.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 -- 172.21.15.90:0/2638510868 >> 172.21.15.90:0/2638510868 conn(0x7fa314101160 msgr2=0x7fa314103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.406+0000 7fa31ac00700 1 -- 172.21.15.90:0/2638510868 shutdown_connections 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.407+0000 7fa31ac00700 1 -- 172.21.15.90:0/2638510868 wait complete. 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa31ac00700 1 Processor -- start 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa31ac00700 1 -- start start 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa31ac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa31ac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419bfb0 con 0x7fa314105b10 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa313e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa313e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43312/0 (socket says 172.21.15.90:43312) 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.408+0000 7fa313e00700 1 -- 172.21.15.90:0/2713175556 learned_addr learned my addr 172.21.15.90:0/2713175556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa313e00700 1 -- 172.21.15.90:0/2713175556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009550 con 0x7fa314105b10 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa313e00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa304011ad0 tx=0x7fa304011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa304003ab0 con 0x7fa314105b10 2024-04-24T08:11:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa304003c10 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3040198e0 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.409+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31419c1b0 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.410+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31419c5d0 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.411+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fa304019b50 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.411+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa314062380 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.411+0000 7fa310c00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 0x7fa2f803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.411+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa30404c150 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.411+0000 7fa313400700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 0x7fa2f803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.412+0000 7fa313400700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 0x7fa2f803a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa300006fd0 tx=0x7fa300006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.417+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa30401a430 con 0x7fa314105b10 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.647+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fa314102620 con 0x7fa2f8038440 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.654+0000 7fa310c00700 1 -- 172.21.15.90:0/2713175556 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa314102620 con 0x7fa2f8038440 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.657+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 msgr2=0x7fa2f803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:56.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.657+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 0x7fa2f803a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa300006fd0 tx=0x7fa300006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.657+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 msgr2=0x7fa31419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.657+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa304011ad0 tx=0x7fa304011bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 shutdown_connections 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fa2f8038440 0x7fa2f803a8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 --2- 172.21.15.90:0/2713175556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa314105b10 0x7fa31419ba70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 >> 172.21.15.90:0/2713175556 conn(0x7fa314101160 msgr2=0x7fa314101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 shutdown_connections 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:55.658+0000 7fa31ac00700 1 -- 172.21.15.90:0/2713175556 wait complete. 2024-04-24T08:11:56.769 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying ceph-exporter service with default placement... 2024-04-24T08:11:58.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-24T08:11:58.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.348+0000 7f0e45e00700 1 Processor -- start 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.348+0000 7f0e45e00700 1 -- start start 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.349+0000 7f0e45e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.349+0000 7f0e45e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40108660 con 0x7f0e40107d10 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.349+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.349+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43324/0 (socket says 172.21.15.90:43324) 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.349+0000 7f0e3ea00700 1 -- 172.21.15.90:0/904112277 learned_addr learned my addr 172.21.15.90:0/904112277 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.350+0000 7f0e3ea00700 1 -- 172.21.15.90:0/904112277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e401087a0 con 0x7f0e40107d10 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.350+0000 7f0e3ea00700 1 --2- 172.21.15.90:0/904112277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0e300098a0 tx=0x7f0e30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85b2b39717bb56a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.351+0000 7f0e3d600700 1 -- 172.21.15.90:0/904112277 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e30004180 con 0x7f0e40107d10 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.351+0000 7f0e3d600700 1 -- 172.21.15.90:0/904112277 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e300042e0 con 0x7f0e40107d10 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.351+0000 7f0e3d600700 1 -- 172.21.15.90:0/904112277 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e30011af0 con 0x7f0e40107d10 2024-04-24T08:11:58.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 -- 172.21.15.90:0/904112277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 msgr2=0x7f0e40108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 --2- 172.21.15.90:0/904112277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0e300098a0 tx=0x7f0e30009bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 -- 172.21.15.90:0/904112277 shutdown_connections 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 --2- 172.21.15.90:0/904112277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e40108120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 -- 172.21.15.90:0/904112277 >> 172.21.15.90:0/904112277 conn(0x7f0e40103570 msgr2=0x7f0e40105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 -- 172.21.15.90:0/904112277 shutdown_connections 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.352+0000 7f0e45e00700 1 -- 172.21.15.90:0/904112277 wait complete. 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.353+0000 7f0e45e00700 1 Processor -- start 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e45e00700 1 -- start start 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e45e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e45e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4019bfa0 con 0x7f0e40107d10 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43336/0 (socket says 172.21.15.90:43336) 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.354+0000 7f0e3ea00700 1 -- 172.21.15.90:0/3832389918 learned_addr learned my addr 172.21.15.90:0/3832389918 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:11:58.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.355+0000 7f0e3ea00700 1 -- 172.21.15.90:0/3832389918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30009550 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.355+0000 7f0e3ea00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e30011ad0 tx=0x7f0e30011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.355+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e30003ab0 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.355+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e30003c10 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.355+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4019c1a0 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.356+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e300198e0 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.356+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4019c5c0 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.357+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0e3001a430 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.357+0000 7f0e44a00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 0x7f0e2403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.357+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4010c1a0 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.357+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e3004c140 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.357+0000 7f0e3e000700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 0x7f0e2403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.358+0000 7f0e3e000700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 0x7f0e2403a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006fd0 tx=0x7f0e2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.363+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e30018b00 con 0x7f0e40107d10 2024-04-24T08:11:58.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.597+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e401067f0 con 0x7f0e24038440 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.606+0000 7f0e44a00700 1 -- 172.21.15.90:0/3832389918 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f0e401067f0 con 0x7f0e24038440 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 msgr2=0x7f0e2403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 0x7f0e2403a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006fd0 tx=0x7f0e2c006e40 comp rx=0 tx=0).stop 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 msgr2=0x7f0e4019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e30011ad0 tx=0x7f0e30011bb0 comp rx=0 tx=0).stop 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 shutdown_connections 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0e24038440 0x7f0e2403a8f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 --2- 172.21.15.90:0/3832389918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e40107d10 0x7f0e4019ba60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 >> 172.21.15.90:0/3832389918 conn(0x7f0e40103570 msgr2=0x7f0e40105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.609+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 shutdown_connections 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:57.610+0000 7f0e45e00700 1 -- 172.21.15.90:0/3832389918 wait complete. 2024-04-24T08:11:58.264 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying prometheus service with default placement... 2024-04-24T08:11:58.857 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:58 smithi090 ceph-mon[25016]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:11:58.858 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:58 smithi090 ceph-mon[25016]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:11:58.858 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:58 smithi090 ceph-mon[25016]: Saving service ceph-exporter spec with placement * 2024-04-24T08:11:58.858 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:58 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:11:58.858 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:11:58 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:00.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:00 smithi090 ceph-mon[25016]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:00.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:00 smithi090 ceph-mon[25016]: Saving service prometheus spec with placement count:1 2024-04-24T08:12:00.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:00 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:00.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:00 smithi090 ceph-mon[25016]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:00.708 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-24T08:12:00.708 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.856+0000 7fdc51800700 1 Processor -- start 2024-04-24T08:12:00.709 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.856+0000 7fdc51800700 1 -- start start 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.857+0000 7fdc51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.857+0000 7fdc51800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c079760 con 0x7fdc4c07ad20 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.857+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.858+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43340/0 (socket says 172.21.15.90:43340) 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.858+0000 7fdc4aa00700 1 -- 172.21.15.90:0/1555606223 learned_addr learned my addr 172.21.15.90:0/1555606223 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.858+0000 7fdc4aa00700 1 -- 172.21.15.90:0/1555606223 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c0798a0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.858+0000 7fdc4aa00700 1 --2- 172.21.15.90:0/1555606223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098a0 tx=0x7fdc3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04baeec306657f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:00.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.859+0000 7fdc49600700 1 -- 172.21.15.90:0/1555606223 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc3c004180 con 0x7fdc4c07ad20 2024-04-24T08:12:00.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.859+0000 7fdc49600700 1 -- 172.21.15.90:0/1555606223 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c0042e0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.859+0000 7fdc49600700 1 -- 172.21.15.90:0/1555606223 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc3c011af0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 -- 172.21.15.90:0/1555606223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 msgr2=0x7fdc4c079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 --2- 172.21.15.90:0/1555606223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098a0 tx=0x7fdc3c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 -- 172.21.15.90:0/1555606223 shutdown_connections 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 --2- 172.21.15.90:0/1555606223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c079220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 -- 172.21.15.90:0/1555606223 >> 172.21.15.90:0/1555606223 conn(0x7fdc4c101160 msgr2=0x7fdc4c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 -- 172.21.15.90:0/1555606223 shutdown_connections 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.860+0000 7fdc51800700 1 -- 172.21.15.90:0/1555606223 wait complete. 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.861+0000 7fdc51800700 1 Processor -- start 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc51800700 1 -- start start 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc51800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a03d0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.717 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43356/0 (socket says 172.21.15.90:43356) 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.862+0000 7fdc4aa00700 1 -- 172.21.15.90:0/3924103458 learned_addr learned my addr 172.21.15.90:0/3924103458 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc4aa00700 1 -- 172.21.15.90:0/3924103458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009550 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc4aa00700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc3c003a70 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c003bd0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc3c0238c0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c1a05d0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.863+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c1a09f0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.864+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c199650 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.865+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fdc3c023a20 con 0x7fdc4c07ad20 2024-04-24T08:12:00.718 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.865+0000 7fdc43400700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 0x7fdc3003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.865+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc3c04c1d0 con 0x7fdc4c07ad20 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.865+0000 7fdc4a000700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 0x7fdc3003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.866+0000 7fdc4a000700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 0x7fdc3003a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006fd0 tx=0x7fdc38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:58.870+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdc3c010380 con 0x7fdc4c07ad20 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.108+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc4c061190 con 0x7fdc30038440 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.116+0000 7fdc43400700 1 -- 172.21.15.90:0/3924103458 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fdc4c061190 con 0x7fdc30038440 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 msgr2=0x7fdc3003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 0x7fdc3003a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006fd0 tx=0x7fdc38006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 msgr2=0x7fdc4c19fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 shutdown_connections 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fdc30038440 0x7fdc3003a8f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 --2- 172.21.15.90:0/3924103458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c07ad20 0x7fdc4c19fe90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.119+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 >> 172.21.15.90:0/3924103458 conn(0x7fdc4c101160 msgr2=0x7fdc4c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:00.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.120+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 shutdown_connections 2024-04-24T08:12:00.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:11:59.120+0000 7fdc51800700 1 -- 172.21.15.90:0/3924103458 wait complete. 2024-04-24T08:12:00.720 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying grafana service with default placement... 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.314+0000 7fbed7e00700 1 Processor -- start 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.314+0000 7fbed7e00700 1 -- start start 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.315+0000 7fbed7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.315+0000 7fbed7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8080210 con 0x7fbed807f8c0 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.315+0000 7fbed6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.315+0000 7fbed6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54974/0 (socket says 172.21.15.90:54974) 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.315+0000 7fbed6a00700 1 -- 172.21.15.90:0/51824645 learned_addr learned my addr 172.21.15.90:0/51824645 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:02.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.316+0000 7fbed6a00700 1 -- 172.21.15.90:0/51824645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8080350 con 0x7fbed807f8c0 2024-04-24T08:12:02.229 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.316+0000 7fbed6a00700 1 --2- 172.21.15.90:0/51824645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbec80098a0 tx=0x7fbec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed34d95c9c9f9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:02.229 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed6000700 1 -- 172.21.15.90:0/51824645 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8004180 con 0x7fbed807f8c0 2024-04-24T08:12:02.229 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed6000700 1 -- 172.21.15.90:0/51824645 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbec80042e0 con 0x7fbed807f8c0 2024-04-24T08:12:02.229 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed6000700 1 -- 172.21.15.90:0/51824645 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8011af0 con 0x7fbed807f8c0 2024-04-24T08:12:02.229 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed7e00700 1 -- 172.21.15.90:0/51824645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 msgr2=0x7fbed807fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed7e00700 1 --2- 172.21.15.90:0/51824645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbec80098a0 tx=0x7fbec8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed7e00700 1 -- 172.21.15.90:0/51824645 shutdown_connections 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed7e00700 1 --2- 172.21.15.90:0/51824645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed807fcd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.317+0000 7fbed7e00700 1 -- 172.21.15.90:0/51824645 >> 172.21.15.90:0/51824645 conn(0x7fbed807afa0 msgr2=0x7fbed807d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.318+0000 7fbed7e00700 1 -- 172.21.15.90:0/51824645 shutdown_connections 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.318+0000 7fbed7e00700 1 -- 172.21.15.90:0/51824645 wait complete. 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed7e00700 1 Processor -- start 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed7e00700 1 -- start start 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed81a0240 con 0x7fbed807f8c0 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.319+0000 7fbed6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54980/0 (socket says 172.21.15.90:54980) 2024-04-24T08:12:02.230 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbed6a00700 1 -- 172.21.15.90:0/2301367126 learned_addr learned my addr 172.21.15.90:0/2301367126 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbed6a00700 1 -- 172.21.15.90:0/2301367126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009550 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbed6a00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbec8011670 tx=0x7fbec8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8003a40 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbec8003ba0 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8024790 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.320+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed81a0440 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.321+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81a0860 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.322+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fbec801a430 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.322+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed8062380 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.322+0000 7fbecf400700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 0x7fbebc03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.322+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbec804c530 con 0x7fbed807f8c0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.322+0000 7fbecfe00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 0x7fbebc03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.323+0000 7fbecfe00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 0x7fbebc03a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:02.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.328+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbec8010380 con 0x7fbed807f8c0 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.552+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fbed807d280 con 0x7fbebc038440 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.564+0000 7fbecf400700 1 -- 172.21.15.90:0/2301367126 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fbed807d280 con 0x7fbebc038440 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 msgr2=0x7fbebc03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 0x7fbebc03a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 msgr2=0x7fbed819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbec8011670 tx=0x7fbec8011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 shutdown_connections 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7fbebc038440 0x7fbebc03a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 --2- 172.21.15.90:0/2301367126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbed807f8c0 0x7fbed819fd00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 >> 172.21.15.90:0/2301367126 conn(0x7fbed807afa0 msgr2=0x7fbed807c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 shutdown_connections 2024-04-24T08:12:02.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:01.567+0000 7fbed7e00700 1 -- 172.21.15.90:0/2301367126 wait complete. 2024-04-24T08:12:02.233 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying node-exporter service with default placement... 2024-04-24T08:12:02.857 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:02 smithi090 ceph-mon[25016]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:02.857 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:02 smithi090 ceph-mon[25016]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:02.857 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:02 smithi090 ceph-mon[25016]: Saving service grafana spec with placement count:1 2024-04-24T08:12:02.858 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:02 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.856+0000 7f328fc00700 1 Processor -- start 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.856+0000 7f328fc00700 1 -- start start 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.857+0000 7f328fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.857+0000 7f328fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3288079700 con 0x7f328807acc0 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.857+0000 7f328d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:03.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.857+0000 7f328d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54994/0 (socket says 172.21.15.90:54994) 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.857+0000 7f328d200700 1 -- 172.21.15.90:0/3279082788 learned_addr learned my addr 172.21.15.90:0/3279082788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.858+0000 7f328d200700 1 -- 172.21.15.90:0/3279082788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288079840 con 0x7f328807acc0 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.858+0000 7f328d200700 1 --2- 172.21.15.90:0/3279082788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f327c0098a0 tx=0x7f327c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cedde0e35651206 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.858+0000 7f3287400700 1 -- 172.21.15.90:0/3279082788 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f327c004180 con 0x7f328807acc0 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.858+0000 7f3287400700 1 -- 172.21.15.90:0/3279082788 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f327c0042e0 con 0x7f328807acc0 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f3287400700 1 -- 172.21.15.90:0/3279082788 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f327c011af0 con 0x7f328807acc0 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 -- 172.21.15.90:0/3279082788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 msgr2=0x7f32880791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 --2- 172.21.15.90:0/3279082788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f327c0098a0 tx=0x7f327c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 -- 172.21.15.90:0/3279082788 shutdown_connections 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 --2- 172.21.15.90:0/3279082788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f32880791c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 -- 172.21.15.90:0/3279082788 >> 172.21.15.90:0/3279082788 conn(0x7f3288101160 msgr2=0x7f3288103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.859+0000 7f328fc00700 1 -- 172.21.15.90:0/3279082788 shutdown_connections 2024-04-24T08:12:03.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.860+0000 7f328fc00700 1 -- 172.21.15.90:0/3279082788 wait complete. 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.860+0000 7f328fc00700 1 Processor -- start 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328fc00700 1 -- start start 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328819c020 con 0x7f328807acc0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54998/0 (socket says 172.21.15.90:54998) 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.861+0000 7f328d200700 1 -- 172.21.15.90:0/519026577 learned_addr learned my addr 172.21.15.90:0/519026577 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f328d200700 1 -- 172.21.15.90:0/519026577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009550 con 0x7f328807acc0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f328d200700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f327c003a70 con 0x7f328807acc0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f327c003bd0 con 0x7f328807acc0 2024-04-24T08:12:03.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328819c220 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f327c0238c0 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.862+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328819c640 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.863+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f327c023b30 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.863+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288062380 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.864+0000 7f3285600700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 0x7f327003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.864+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f327c04c220 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.864+0000 7f3287e00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 0x7f327003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.865+0000 7f3287e00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 0x7f327003a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3274006fd0 tx=0x7f3274006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:02.869+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f327c010380 con 0x7f328807acc0 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.093+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3288103460 con 0x7f3270038440 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.101+0000 7f3285600700 1 -- 172.21.15.90:0/519026577 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3288103460 con 0x7f3270038440 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 msgr2=0x7f327003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 0x7f327003a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3274006fd0 tx=0x7f3274006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 msgr2=0x7f328819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:03.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 shutdown_connections 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f3270038440 0x7f327003a8f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 --2- 172.21.15.90:0/519026577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f328807acc0 0x7f328819bae0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 >> 172.21.15.90:0/519026577 conn(0x7f3288101160 msgr2=0x7f3288102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.104+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 shutdown_connections 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:03.105+0000 7f328fc00700 1 -- 172.21.15.90:0/519026577 wait complete. 2024-04-24T08:12:03.801 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying alertmanager service with default placement... 2024-04-24T08:12:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:04 smithi090 ceph-mon[25016]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:04 smithi090 ceph-mon[25016]: Saving service node-exporter spec with placement * 2024-04-24T08:12:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:04 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:04 smithi090 ceph-mon[25016]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.387+0000 7f0003e00700 1 Processor -- start 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.387+0000 7f0003e00700 1 -- start start 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.388+0000 7f0003e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.388+0000 7f0003e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc106460 con 0x7efffc105b10 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.388+0000 7f0001400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.388+0000 7f0001400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55008/0 (socket says 172.21.15.90:55008) 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.388+0000 7f0001400700 1 -- 172.21.15.90:0/1425293356 learned_addr learned my addr 172.21.15.90:0/1425293356 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:05.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.389+0000 7f0001400700 1 -- 172.21.15.90:0/1425293356 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc1065a0 con 0x7efffc105b10 2024-04-24T08:12:05.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.389+0000 7f0001400700 1 --2- 172.21.15.90:0/1425293356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efff00098a0 tx=0x7efff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b093eed63dfd8fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:05.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.389+0000 7efffbe00700 1 -- 172.21.15.90:0/1425293356 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff0004180 con 0x7efffc105b10 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.389+0000 7efffbe00700 1 -- 172.21.15.90:0/1425293356 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efff00042e0 con 0x7efffc105b10 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.389+0000 7efffbe00700 1 -- 172.21.15.90:0/1425293356 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff0011af0 con 0x7efffc105b10 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 -- 172.21.15.90:0/1425293356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 msgr2=0x7efffc105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 --2- 172.21.15.90:0/1425293356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efff00098a0 tx=0x7efff0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 -- 172.21.15.90:0/1425293356 shutdown_connections 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 --2- 172.21.15.90:0/1425293356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc105f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 -- 172.21.15.90:0/1425293356 >> 172.21.15.90:0/1425293356 conn(0x7efffc101160 msgr2=0x7efffc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 -- 172.21.15.90:0/1425293356 shutdown_connections 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.390+0000 7f0003e00700 1 -- 172.21.15.90:0/1425293356 wait complete. 2024-04-24T08:12:05.335 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.392+0000 7f0003e00700 1 Processor -- start 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.392+0000 7f0003e00700 1 -- start start 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.392+0000 7f0003e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.392+0000 7f0003e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc07b070 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.393+0000 7f0001400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.393+0000 7f0001400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55018/0 (socket says 172.21.15.90:55018) 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.393+0000 7f0001400700 1 -- 172.21.15.90:0/3977280040 learned_addr learned my addr 172.21.15.90:0/3977280040 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.393+0000 7f0001400700 1 -- 172.21.15.90:0/3977280040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0009550 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.393+0000 7f0001400700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efff0011ad0 tx=0x7efff0011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.394+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff0003ab0 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.394+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efff0003c10 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.394+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc079180 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.394+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff00198e0 con 0x7efffc105b10 2024-04-24T08:12:05.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.394+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0795a0 con 0x7efffc105b10 2024-04-24T08:12:05.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.395+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7efff001a430 con 0x7efffc105b10 2024-04-24T08:12:05.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.395+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc062380 con 0x7efffc105b10 2024-04-24T08:12:05.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.395+0000 7efffa000700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 0x7effe403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:05.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.395+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efff004c200 con 0x7efffc105b10 2024-04-24T08:12:05.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.396+0000 7f0000a00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 0x7effe403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.396+0000 7f0000a00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 0x7effe403a8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.401+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efff0018b00 con 0x7efffc105b10 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.629+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7efffc102620 con 0x7effe4038440 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.639+0000 7efffa000700 1 -- 172.21.15.90:0/3977280040 <== mgr.14120 v2:172.21.15.90:6800/3041763742 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7efffc102620 con 0x7effe4038440 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 msgr2=0x7effe403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 0x7effe403a8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 msgr2=0x7efffc07ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efff0011ad0 tx=0x7efff0011bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 shutdown_connections 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7effe4038440 0x7effe403a8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 --2- 172.21.15.90:0/3977280040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efffc105b10 0x7efffc07ab30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:05.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 >> 172.21.15.90:0/3977280040 conn(0x7efffc101160 msgr2=0x7efffc101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:05.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 shutdown_connections 2024-04-24T08:12:05.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:04.643+0000 7f0003e00700 1 -- 172.21.15.90:0/3977280040 wait complete. 2024-04-24T08:12:05.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:05 smithi090 ceph-mon[25016]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:05.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:05 smithi090 ceph-mon[25016]: Saving service alertmanager spec with placement count:1 2024-04-24T08:12:05.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:05 smithi090 ceph-mon[25016]: from='mgr.14120 172.21.15.90:0/2879485280' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:06 smithi090 ceph-mon[25016]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:06 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3719958176' entity='client.admin' 2024-04-24T08:12:06.879 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.919+0000 7f6a8f400700 1 Processor -- start 2024-04-24T08:12:06.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.919+0000 7f6a8f400700 1 -- start start 2024-04-24T08:12:06.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.919+0000 7f6a8f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:06.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.920+0000 7f6a8f400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8807a870 con 0x7f6a8807be30 2024-04-24T08:12:06.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.920+0000 7f6a8ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:06.881 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.920+0000 7f6a8ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55030/0 (socket says 172.21.15.90:55030) 2024-04-24T08:12:06.881 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.920+0000 7f6a8ca00700 1 -- 172.21.15.90:0/640561616 learned_addr learned my addr 172.21.15.90:0/640561616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.921+0000 7f6a8ca00700 1 -- 172.21.15.90:0/640561616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8807a9b0 con 0x7f6a8807be30 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.921+0000 7f6a8ca00700 1 --2- 172.21.15.90:0/640561616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098a0 tx=0x7f6a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2624a03bc725a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.921+0000 7f6a87400700 1 -- 172.21.15.90:0/640561616 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c004180 con 0x7f6a8807be30 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.921+0000 7f6a87400700 1 -- 172.21.15.90:0/640561616 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c0042e0 con 0x7f6a8807be30 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.921+0000 7f6a87400700 1 -- 172.21.15.90:0/640561616 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c011af0 con 0x7f6a8807be30 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 -- 172.21.15.90:0/640561616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 msgr2=0x7f6a8807a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 --2- 172.21.15.90:0/640561616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098a0 tx=0x7f6a7c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 -- 172.21.15.90:0/640561616 shutdown_connections 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 --2- 172.21.15.90:0/640561616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8807a330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 -- 172.21.15.90:0/640561616 >> 172.21.15.90:0/640561616 conn(0x7f6a88101180 msgr2=0x7f6a88103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 -- 172.21.15.90:0/640561616 shutdown_connections 2024-04-24T08:12:06.884 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.922+0000 7f6a8f400700 1 -- 172.21.15.90:0/640561616 wait complete. 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8f400700 1 Processor -- start 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8f400700 1 -- start start 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8f400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8819c000 con 0x7f6a8807be30 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55040/0 (socket says 172.21.15.90:55040) 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.924+0000 7f6a8ca00700 1 -- 172.21.15.90:0/3719958176 learned_addr learned my addr 172.21.15.90:0/3719958176 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a8ca00700 1 -- 172.21.15.90:0/3719958176 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009550 con 0x7f6a8807be30 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a8ca00700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006ba0 tx=0x7f6a7c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c003b00 con 0x7f6a8807be30 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c01a8e0 con 0x7f6a8807be30 2024-04-24T08:12:06.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c019990 con 0x7f6a8807be30 2024-04-24T08:12:06.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.925+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8819c200 con 0x7f6a8807be30 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.926+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8819c620 con 0x7f6a8807be30 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.927+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a88062380 con 0x7f6a8807be30 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.927+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f6a7c019c00 con 0x7f6a8807be30 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.928+0000 7f6a85600700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 0x7f6a7003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.928+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6a7c04c9e0 con 0x7f6a8807be30 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.928+0000 7f6a87e00700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 0x7f6a7003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.929+0000 7f6a87e00700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 0x7f6a7003a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a74006fd0 tx=0x7f6a74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:06.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:05.932+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6a7c023750 con 0x7f6a8807be30 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.147+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f6a8819f060 con 0x7f6a8807be30 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.152+0000 7f6a85600700 1 -- 172.21.15.90:0/3719958176 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6a7c04b0d0 con 0x7f6a8807be30 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 msgr2=0x7f6a7003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 0x7f6a7003a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a74006fd0 tx=0x7f6a74006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 msgr2=0x7f6a8819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006ba0 tx=0x7f6a7c011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 shutdown_connections 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f6a700383f0 0x7f6a7003a8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 --2- 172.21.15.90:0/3719958176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8807be30 0x7f6a8819bac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 >> 172.21.15.90:0/3719958176 conn(0x7f6a88101180 msgr2=0x7f6a88102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:06.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.155+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 shutdown_connections 2024-04-24T08:12:06.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:06.156+0000 7f6a8f400700 1 -- 172.21.15.90:0/3719958176 wait complete. 2024-04-24T08:12:08.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.493+0000 7f62ef800700 1 Processor -- start 2024-04-24T08:12:08.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.493+0000 7f62ef800700 1 -- start start 2024-04-24T08:12:08.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.494+0000 7f62ef800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:08.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.494+0000 7f62ef800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8106460 con 0x7f62e8105b10 2024-04-24T08:12:08.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.494+0000 7f62ece00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:08.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.494+0000 7f62ece00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55044/0 (socket says 172.21.15.90:55044) 2024-04-24T08:12:08.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.494+0000 7f62ece00700 1 -- 172.21.15.90:0/25128148 learned_addr learned my addr 172.21.15.90:0/25128148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:08.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.495+0000 7f62ece00700 1 -- 172.21.15.90:0/25128148 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e81065a0 con 0x7f62e8105b10 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.495+0000 7f62ece00700 1 --2- 172.21.15.90:0/25128148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098a0 tx=0x7f62d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e32e30b44fdc46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.495+0000 7f62e7e00700 1 -- 172.21.15.90:0/25128148 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62d8004180 con 0x7f62e8105b10 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.495+0000 7f62e7e00700 1 -- 172.21.15.90:0/25128148 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62d80042e0 con 0x7f62e8105b10 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.495+0000 7f62e7e00700 1 -- 172.21.15.90:0/25128148 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62d8011af0 con 0x7f62e8105b10 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 -- 172.21.15.90:0/25128148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 msgr2=0x7f62e8105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 --2- 172.21.15.90:0/25128148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098a0 tx=0x7f62d8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 -- 172.21.15.90:0/25128148 shutdown_connections 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 --2- 172.21.15.90:0/25128148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8105f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 -- 172.21.15.90:0/25128148 >> 172.21.15.90:0/25128148 conn(0x7f62e8101160 msgr2=0x7f62e8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 -- 172.21.15.90:0/25128148 shutdown_connections 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.496+0000 7f62ef800700 1 -- 172.21.15.90:0/25128148 wait complete. 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.497+0000 7f62ef800700 1 Processor -- start 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ef800700 1 -- start start 2024-04-24T08:12:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ef800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:08.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ef800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8199db0 con 0x7f62e8105b10 2024-04-24T08:12:08.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ece00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:08.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ece00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55058/0 (socket says 172.21.15.90:55058) 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.498+0000 7f62ece00700 1 -- 172.21.15.90:0/3361362913 learned_addr learned my addr 172.21.15.90:0/3361362913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62ece00700 1 -- 172.21.15.90:0/3361362913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8009550 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62ece00700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011ad0 tx=0x7f62d8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62d8003ab0 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62d8003c10 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62d80198e0 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e8199fb0 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.499+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e819a3d0 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.500+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f62d8019b50 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.500+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e8062380 con 0x7f62e8105b10 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.501+0000 7f62e6000700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 0x7f62cc03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:08.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.501+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f62d804c150 con 0x7f62e8105b10 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.501+0000 7f62dfe00700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 0x7f62cc03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.502+0000 7f62dfe00700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 0x7f62cc03a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006fd0 tx=0x7f62d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.507+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f62d801a430 con 0x7f62e8105b10 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.720+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f62e819d2a0 con 0x7f62e8105b10 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.727+0000 7f62e6000700 1 -- 172.21.15.90:0/3361362913 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f62d8018b00 con 0x7f62e8105b10 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 msgr2=0x7f62cc03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 0x7f62cc03a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006fd0 tx=0x7f62d4006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 msgr2=0x7f62e8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011ad0 tx=0x7f62d8011bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 shutdown_connections 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f62cc038440 0x7f62cc03a8f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 --2- 172.21.15.90:0/3361362913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62e8105b10 0x7f62e8199870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:08.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 >> 172.21.15.90:0/3361362913 conn(0x7f62e8101160 msgr2=0x7f62e8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:08.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 shutdown_connections 2024-04-24T08:12:08.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:07.731+0000 7f62ef800700 1 -- 172.21.15.90:0/3361362913 wait complete. 2024-04-24T08:12:08.407 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-04-24T08:12:08.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:08 smithi090 ceph-mon[25016]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:08.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:08 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3361362913' entity='client.admin' 2024-04-24T08:12:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:09 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/999606423' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T08:12:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.984+0000 7f32f5200700 1 Processor -- start 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.984+0000 7f32f5200700 1 -- start start 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.984+0000 7f32f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.985+0000 7f32f5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f010f210 con 0x7f32f00802e0 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.985+0000 7f32ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.985+0000 7f32ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55062/0 (socket says 172.21.15.90:55062) 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.985+0000 7f32ede00700 1 -- 172.21.15.90:0/1314565108 learned_addr learned my addr 172.21.15.90:0/1314565108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.986+0000 7f32ede00700 1 -- 172.21.15.90:0/1314565108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f010f350 con 0x7f32f00802e0 2024-04-24T08:12:10.522 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.986+0000 7f32ede00700 1 --2- 172.21.15.90:0/1314565108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eb0b66b4c87c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.986+0000 7f32ed400700 1 -- 172.21.15.90:0/1314565108 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0004180 con 0x7f32f00802e0 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.986+0000 7f32ed400700 1 -- 172.21.15.90:0/1314565108 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32e00042e0 con 0x7f32f00802e0 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.986+0000 7f32ed400700 1 -- 172.21.15.90:0/1314565108 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0011af0 con 0x7f32f00802e0 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 -- 172.21.15.90:0/1314565108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 msgr2=0x7f32f010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 --2- 172.21.15.90:0/1314565108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 -- 172.21.15.90:0/1314565108 shutdown_connections 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 --2- 172.21.15.90:0/1314565108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f010ecd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 -- 172.21.15.90:0/1314565108 >> 172.21.15.90:0/1314565108 conn(0x7f32f007b220 msgr2=0x7f32f007d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.987+0000 7f32f5200700 1 -- 172.21.15.90:0/1314565108 shutdown_connections 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.988+0000 7f32f5200700 1 -- 172.21.15.90:0/1314565108 wait complete. 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.989+0000 7f32f5200700 1 Processor -- start 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.989+0000 7f32f5200700 1 -- start start 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.989+0000 7f32f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.989+0000 7f32f5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f01a03b0 con 0x7f32f00802e0 2024-04-24T08:12:10.523 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55076/0 (socket says 172.21.15.90:55076) 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32ede00700 1 -- 172.21.15.90:0/999606423 learned_addr learned my addr 172.21.15.90:0/999606423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32ede00700 1 -- 172.21.15.90:0/999606423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0009550 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32ede00700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000c00 tx=0x7f32e0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.990+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0003a70 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.991+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32e0003bd0 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.991+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f01a05b0 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.991+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e00238c0 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.991+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f01a09d0 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.992+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f32e0023b80 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.992+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f0062380 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.992+0000 7f32e7400700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 0x7f32d003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.993+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f32e004c300 con 0x7f32f00802e0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.993+0000 7f32e5400700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 0x7f32d003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:10.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.993+0000 7f32e5400700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 0x7f32d003a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32d8006fd0 tx=0x7f32d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:08.997+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f32e00193c0 con 0x7f32f00802e0 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.272+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f32f01a3b50 con 0x7f32f00802e0 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.736+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f32e0019740 con 0x7f32f00802e0 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.739+0000 7f32e7400700 1 -- 172.21.15.90:0/999606423 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f32e002c9a0 con 0x7f32f00802e0 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 msgr2=0x7f32d003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 0x7f32d003a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32d8006fd0 tx=0x7f32d8006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 msgr2=0x7f32f019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000c00 tx=0x7f32e0011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 shutdown_connections 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f32d00383f0 0x7f32d003a8a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 --2- 172.21.15.90:0/999606423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f00802e0 0x7f32f019fe70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 >> 172.21.15.90:0/999606423 conn(0x7f32f007b220 msgr2=0x7f32f007c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 shutdown_connections 2024-04-24T08:12:10.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:09.743+0000 7f32f5200700 1 -- 172.21.15.90:0/999606423 wait complete. 2024-04-24T08:12:10.932 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:10 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/999606423' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T08:12:10.932 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:10 smithi090 ceph-mon[25016]: mgrmap e9: smithi090.jsrfxo(active, since 32s) 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "smithi090.jsrfxo", 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.134+0000 7f0fe1a00700 1 Processor -- start 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.134+0000 7f0fe1a00700 1 -- start start 2024-04-24T08:12:12.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.134+0000 7f0fe1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.134+0000 7f0fe1a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc079740 con 0x7f0fdc07ad00 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.135+0000 7f0fdaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.135+0000 7f0fdaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57206/0 (socket says 172.21.15.90:57206) 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.135+0000 7f0fdaa00700 1 -- 172.21.15.90:0/4104897652 learned_addr learned my addr 172.21.15.90:0/4104897652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.135+0000 7f0fdaa00700 1 -- 172.21.15.90:0/4104897652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc079880 con 0x7f0fdc07ad00 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.136+0000 7f0fdaa00700 1 --2- 172.21.15.90:0/4104897652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0098a0 tx=0x7f0fcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65bc0b28a23c2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.136+0000 7f0fd9600700 1 -- 172.21.15.90:0/4104897652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fcc004180 con 0x7f0fdc07ad00 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.136+0000 7f0fd9600700 1 -- 172.21.15.90:0/4104897652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fcc0042e0 con 0x7f0fdc07ad00 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.136+0000 7f0fd9600700 1 -- 172.21.15.90:0/4104897652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fcc011af0 con 0x7f0fdc07ad00 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 -- 172.21.15.90:0/4104897652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 msgr2=0x7f0fdc079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/4104897652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0098a0 tx=0x7f0fcc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:12.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 -- 172.21.15.90:0/4104897652 shutdown_connections 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/4104897652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc079200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 -- 172.21.15.90:0/4104897652 >> 172.21.15.90:0/4104897652 conn(0x7f0fdc101160 msgr2=0x7f0fdc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 -- 172.21.15.90:0/4104897652 shutdown_connections 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.137+0000 7f0fe1a00700 1 -- 172.21.15.90:0/4104897652 wait complete. 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.138+0000 7f0fe1a00700 1 Processor -- start 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.138+0000 7f0fe1a00700 1 -- start start 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.138+0000 7f0fe1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.138+0000 7f0fe1a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19c010 con 0x7f0fdc07ad00 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.139+0000 7f0fdaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.139+0000 7f0fdaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57212/0 (socket says 172.21.15.90:57212) 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.139+0000 7f0fdaa00700 1 -- 172.21.15.90:0/3972482941 learned_addr learned my addr 172.21.15.90:0/3972482941 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.139+0000 7f0fdaa00700 1 -- 172.21.15.90:0/3972482941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc009550 con 0x7f0fdc07ad00 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.139+0000 7f0fdaa00700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:12.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.140+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fcc003a70 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.140+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fcc003bd0 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.140+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fcc0238c0 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.140+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc19c210 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.140+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc19c630 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.141+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f0fcc023b80 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.141+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc062380 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.141+0000 7f0fd3400700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 0x7f0fc003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.141+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0fcc04c290 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.141+0000 7f0fda000700 1 -- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 msgr2=0x7f0fc003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.142+0000 7f0fda000700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 0x7f0fc003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.147+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0fcc010970 con 0x7f0fdc07ad00 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.343+0000 7f0fda000700 1 -- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 msgr2=0x7f0fc003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.343+0000 7f0fda000700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 0x7f0fc003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:12:12.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.419+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0fdc19f7b0 con 0x7f0fdc07ad00 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.420+0000 7f0fd3400700 1 -- 172.21.15.90:0/3972482941 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f0fcc02c350 con 0x7f0fdc07ad00 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 msgr2=0x7f0fc003a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 0x7f0fc003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 msgr2=0x7f0fdc19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 shutdown_connections 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f0fc0038440 0x7f0fc003a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 --2- 172.21.15.90:0/3972482941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fdc07ad00 0x7f0fdc19bad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 >> 172.21.15.90:0/3972482941 conn(0x7f0fdc101160 msgr2=0x7f0fdc102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 shutdown_connections 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:11.423+0000 7f0fe1a00700 1 -- 172.21.15.90:0/3972482941 wait complete. 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-24T08:12:12.069 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-04-24T08:12:12.362 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:12 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3972482941' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:12:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: Active manager daemon smithi090.jsrfxo restarted 2024-04-24T08:12:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:12:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T08:12:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: mgrmap e10: smithi090.jsrfxo(active, starting, since 0.0101384s) 2024-04-24T08:12:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:12:18.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:18 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: [24/Apr/2024:08:12:18] ENGINE Bus STARTING 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: [24/Apr/2024:08:12:18] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: [24/Apr/2024:08:12:18] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: [24/Apr/2024:08:12:18] ENGINE Bus STARTED 2024-04-24T08:12:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:19 smithi090 ceph-mon[25016]: mgrmap e11: smithi090.jsrfxo(active, since 1.01729s) 2024-04-24T08:12:20.734 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:20 smithi090 ceph-mon[25016]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:12:20.734 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:20 smithi090 ceph-mon[25016]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:12:20.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:12:20.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T08:12:20.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:12:20.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:12:20.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.633+0000 7f2f26c00700 1 Processor -- start 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.633+0000 7f2f26c00700 1 -- start start 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.634+0000 7f2f26c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.634+0000 7f2f26c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20079740 con 0x7f2f2007ad00 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.634+0000 7f2f1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.634+0000 7f2f1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57214/0 (socket says 172.21.15.90:57214) 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.634+0000 7f2f1fe00700 1 -- 172.21.15.90:0/1558269125 learned_addr learned my addr 172.21.15.90:0/1558269125 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:20.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.635+0000 7f2f1fe00700 1 -- 172.21.15.90:0/1558269125 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20079880 con 0x7f2f2007ad00 2024-04-24T08:12:20.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.635+0000 7f2f1fe00700 1 --2- 172.21.15.90:0/1558269125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098a0 tx=0x7f2f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1322b9de4df7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.635+0000 7f2f1ea00700 1 -- 172.21.15.90:0/1558269125 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f10004180 con 0x7f2f2007ad00 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.635+0000 7f2f1ea00700 1 -- 172.21.15.90:0/1558269125 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f100042e0 con 0x7f2f2007ad00 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.635+0000 7f2f1ea00700 1 -- 172.21.15.90:0/1558269125 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f10011af0 con 0x7f2f2007ad00 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 -- 172.21.15.90:0/1558269125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 msgr2=0x7f2f20079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 --2- 172.21.15.90:0/1558269125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098a0 tx=0x7f2f10009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 -- 172.21.15.90:0/1558269125 shutdown_connections 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 --2- 172.21.15.90:0/1558269125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f20079200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 -- 172.21.15.90:0/1558269125 >> 172.21.15.90:0/1558269125 conn(0x7f2f20101160 msgr2=0x7f2f20103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 -- 172.21.15.90:0/1558269125 shutdown_connections 2024-04-24T08:12:20.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.636+0000 7f2f26c00700 1 -- 172.21.15.90:0/1558269125 wait complete. 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.637+0000 7f2f26c00700 1 Processor -- start 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f26c00700 1 -- start start 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f26c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f26c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019c010 con 0x7f2f2007ad00 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57218/0 (socket says 172.21.15.90:57218) 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f1fe00700 1 -- 172.21.15.90:0/2089416242 learned_addr learned my addr 172.21.15.90:0/2089416242 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.638+0000 7f2f1fe00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10009550 con 0x7f2f2007ad00 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f1fe00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f10000c00 tx=0x7f2f10011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f10003a70 con 0x7f2f2007ad00 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f10003bd0 con 0x7f2f2007ad00 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f100238c0 con 0x7f2f2007ad00 2024-04-24T08:12:20.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2019c210 con 0x7f2f2007ad00 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.639+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2019c630 con 0x7f2f2007ad00 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.640+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f2f10023b30 con 0x7f2f2007ad00 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.641+0000 7f2f1cc00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.641+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f0403b050 con 0x7f2f04038490 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.641+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f1004c2f0 con 0x7f2f2007ad00 2024-04-24T08:12:20.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.641+0000 7f2f1f400700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.641+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.842+0000 7f2f1f400700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:12.842+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:13.243+0000 7f2f1f400700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:13.243+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:14.044+0000 7f2f1f400700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:14.044+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:12:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:15.646+0000 7f2f1f400700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3041763742 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:15.646+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:18.364+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7f2f1001ad00 con 0x7f2f2007ad00 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:18.364+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 msgr2=0x7f2f0403a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:18.364+0000 7f2f1cc00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.368+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f2f100193c0 con 0x7f2f2007ad00 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.368+0000 7f2f1cc00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 0x7f2f0403e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.368+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f0403b050 con 0x7f2f0403bef0 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.369+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 0x7f2f0403e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:20.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.371+0000 7f2f1f400700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 0x7f2f0403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c003a10 tx=0x7f2f0c009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.373+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2f0403b050 con 0x7f2f0403bef0 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.380+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2f2019ce10 con 0x7f2f0403bef0 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.380+0000 7f2f1cc00700 1 -- 172.21.15.90:0/2089416242 <== mgr.14164 v2:172.21.15.90:6800/3702780720 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f2f2019ce10 con 0x7f2f0403bef0 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.381+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 msgr2=0x7f2f0403e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.381+0000 7f2f26c00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 0x7f2f0403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c003a10 tx=0x7f2f0c009270 comp rx=0 tx=0).stop 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.381+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 msgr2=0x7f2f2019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.381+0000 7f2f26c00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f10000c00 tx=0x7f2f10011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 shutdown_connections 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f2f0403bef0 0x7f2f0403e2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2007ad00 0x7f2f2019bad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 --2- 172.21.15.90:0/2089416242 >> [v2:172.21.15.90:6800/3041763742,v1:172.21.15.90:6801/3041763742] conn(0x7f2f04038490 0x7f2f0403a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 >> 172.21.15.90:0/2089416242 conn(0x7f2f20101160 msgr2=0x7f2f20102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:20.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 shutdown_connections 2024-04-24T08:12:20.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:19.382+0000 7f2f26c00700 1 -- 172.21.15.90:0/2089416242 wait complete. 2024-04-24T08:12:20.767 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-04-24T08:12:20.767 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-04-24T08:12:21.779 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:21 smithi090 ceph-mon[25016]: mgrmap e12: smithi090.jsrfxo(active, since 2s) 2024-04-24T08:12:21.779 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:21 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.362+0000 7f5fb6a00700 1 Processor -- start 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.362+0000 7f5fb6a00700 1 -- start start 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.362+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.362+0000 7f5fb6a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0106460 con 0x7f5fb0105b10 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.363+0000 7f5fafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.363+0000 7f5fafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40730/0 (socket says 172.21.15.90:40730) 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.363+0000 7f5fafe00700 1 -- 172.21.15.90:0/3757881023 learned_addr learned my addr 172.21.15.90:0/3757881023 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.364+0000 7f5fafe00700 1 -- 172.21.15.90:0/3757881023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb01065a0 con 0x7f5fb0105b10 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.364+0000 7f5fafe00700 1 --2- 172.21.15.90:0/3757881023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd177107e7a13796 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.364+0000 7f5faea00700 1 -- 172.21.15.90:0/3757881023 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa0004180 con 0x7f5fb0105b10 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.364+0000 7f5faea00700 1 -- 172.21.15.90:0/3757881023 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fa00042e0 con 0x7f5fb0105b10 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.364+0000 7f5faea00700 1 -- 172.21.15.90:0/3757881023 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa0011af0 con 0x7f5fb0105b10 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3757881023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 msgr2=0x7f5fb0105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3757881023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3757881023 shutdown_connections 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3757881023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb0105f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3757881023 >> 172.21.15.90:0/3757881023 conn(0x7f5fb0101160 msgr2=0x7f5fb0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.365+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3757881023 shutdown_connections 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.366+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3757881023 wait complete. 2024-04-24T08:12:22.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.367+0000 7f5fb6a00700 1 Processor -- start 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.367+0000 7f5fb6a00700 1 -- start start 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fb6a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019bfa0 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40732/0 (socket says 172.21.15.90:40732) 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fafe00700 1 -- 172.21.15.90:0/3957506407 learned_addr learned my addr 172.21.15.90:0/3957506407 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.368+0000 7f5fafe00700 1 -- 172.21.15.90:0/3957506407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0009550 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5fafe00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011ad0 tx=0x7f5fa0011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa0003ab0 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fa0003c10 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa00198e0 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb019c1a0 con 0x7f5fb0105b10 2024-04-24T08:12:22.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.369+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb019c5c0 con 0x7f5fb0105b10 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.371+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f5fa0019a40 con 0x7f5fb0105b10 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.371+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb01958b0 con 0x7f5fb0105b10 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.371+0000 7f5facc00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 0x7f5f9403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.371+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5fa004c2d0 con 0x7f5fb0105b10 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.372+0000 7f5faf400700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 0x7f5f9403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.372+0000 7f5faf400700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 0x7f5f9403a990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006fd0 tx=0x7f5f9c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.377+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5fa001a430 con 0x7f5fb0105b10 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.604+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb0061190 con 0x7f5f940384e0 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.862+0000 7f5facc00700 1 -- 172.21.15.90:0/3957506407 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5fb0061190 con 0x7f5f940384e0 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.866+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 msgr2=0x7f5f9403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.866+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 0x7f5f9403a990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006fd0 tx=0x7f5f9c006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.866+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 msgr2=0x7f5fb019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:22.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.866+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011ad0 tx=0x7f5fa0011bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 shutdown_connections 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5f940384e0 0x7f5f9403a990 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 --2- 172.21.15.90:0/3957506407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb0105b10 0x7f5fb019ba60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 >> 172.21.15.90:0/3957506407 conn(0x7f5fb0101160 msgr2=0x7f5fb01094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 shutdown_connections 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:21.867+0000 7f5fb6a00700 1 -- 172.21.15.90:0/3957506407 wait complete. 2024-04-24T08:12:22.555 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-04-24T08:12:23.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:22 smithi090 ceph-mon[25016]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:23.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:23.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:23.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:24.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Nw0RfuHwljEGPBGRfs8YyeRmrAgwav56lh.BbYXQxHbTdi7UEl7N2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713946343, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T08:12:24.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.279+0000 7f09b5600700 1 Processor -- start 2024-04-24T08:12:24.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.280+0000 7f09b5600700 1 -- start start 2024-04-24T08:12:24.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.280+0000 7f09b5600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.280+0000 7f09b5600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0106460 con 0x7f09b0105b10 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.280+0000 7f09aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.281+0000 7f09aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40738/0 (socket says 172.21.15.90:40738) 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.281+0000 7f09aea00700 1 -- 172.21.15.90:0/3378067304 learned_addr learned my addr 172.21.15.90:0/3378067304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.281+0000 7f09aea00700 1 -- 172.21.15.90:0/3378067304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b01065a0 con 0x7f09b0105b10 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.281+0000 7f09aea00700 1 --2- 172.21.15.90:0/3378067304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098a0 tx=0x7f09a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488be829103f67dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.281+0000 7f09ad600700 1 -- 172.21.15.90:0/3378067304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09a0004180 con 0x7f09b0105b10 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09ad600700 1 -- 172.21.15.90:0/3378067304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09a00042e0 con 0x7f09b0105b10 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09ad600700 1 -- 172.21.15.90:0/3378067304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09a0011af0 con 0x7f09b0105b10 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09b5600700 1 -- 172.21.15.90:0/3378067304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 msgr2=0x7f09b0105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09b5600700 1 --2- 172.21.15.90:0/3378067304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098a0 tx=0x7f09a0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:24.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09b5600700 1 -- 172.21.15.90:0/3378067304 shutdown_connections 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09b5600700 1 --2- 172.21.15.90:0/3378067304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b0105f20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.282+0000 7f09b5600700 1 -- 172.21.15.90:0/3378067304 >> 172.21.15.90:0/3378067304 conn(0x7f09b0101160 msgr2=0x7f09b0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.283+0000 7f09b5600700 1 -- 172.21.15.90:0/3378067304 shutdown_connections 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.283+0000 7f09b5600700 1 -- 172.21.15.90:0/3378067304 wait complete. 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.284+0000 7f09b5600700 1 Processor -- start 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.284+0000 7f09b5600700 1 -- start start 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09b5600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09b5600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0199de0 con 0x7f09b0105b10 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40742/0 (socket says 172.21.15.90:40742) 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09aea00700 1 -- 172.21.15.90:0/1602350739 learned_addr learned my addr 172.21.15.90:0/1602350739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.285+0000 7f09aea00700 1 -- 172.21.15.90:0/1602350739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009550 con 0x7f09b0105b10 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09aea00700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006ba0 tx=0x7f09a0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09a0003b00 con 0x7f09b0105b10 2024-04-24T08:12:24.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09a001a8e0 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0199fe0 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09a00199c0 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.286+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b019a400 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.287+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b0062380 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.288+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f09a0019c30 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.288+0000 7f09a7400700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 0x7f099403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.288+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09a004c720 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.289+0000 7f09ae000700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 0x7f099403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.289+0000 7f09ae000700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 0x7f099403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f099c006fd0 tx=0x7f099c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.294+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f09a001a430 con 0x7f09b0105b10 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.518+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f09b0109fb0 con 0x7f0994038440 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.751+0000 7f09a7400700 1 -- 172.21.15.90:0/1602350739 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f09b0109fb0 con 0x7f0994038440 2024-04-24T08:12:24.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 msgr2=0x7f099403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:24.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 0x7f099403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f099c006fd0 tx=0x7f099c006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:24.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 msgr2=0x7f09b01998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:24.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006ba0 tx=0x7f09a0011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 shutdown_connections 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f0994038440 0x7f099403a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 --2- 172.21.15.90:0/1602350739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b0105b10 0x7f09b01998a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 >> 172.21.15.90:0/1602350739 conn(0x7f09b0101160 msgr2=0x7f09b0102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 shutdown_connections 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:23.754+0000 7f09b5600700 1 -- 172.21.15.90:0/1602350739 wait complete. 2024-04-24T08:12:24.706 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-04-24T08:12:24.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:24 smithi090 ceph-mon[25016]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:24.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:24 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:25.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:25 smithi090 ceph-mon[25016]: mgrmap e13: smithi090.jsrfxo(active, since 6s) 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.754+0000 7efd54200700 1 Processor -- start 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.754+0000 7efd54200700 1 -- start start 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.755+0000 7efd54200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.755+0000 7efd54200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c105380 con 0x7efd4c104a30 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.755+0000 7efd51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.755+0000 7efd51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40752/0 (socket says 172.21.15.90:40752) 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.755+0000 7efd51800700 1 -- 172.21.15.90:0/978350591 learned_addr learned my addr 172.21.15.90:0/978350591 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.756+0000 7efd51800700 1 -- 172.21.15.90:0/978350591 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c1054c0 con 0x7efd4c104a30 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.757+0000 7efd51800700 1 --2- 172.21.15.90:0/978350591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd400098a0 tx=0x7efd40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41e63773db1ff0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.758+0000 7efd4be00700 1 -- 172.21.15.90:0/978350591 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd40004180 con 0x7efd4c104a30 2024-04-24T08:12:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.758+0000 7efd4be00700 1 -- 172.21.15.90:0/978350591 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd400042e0 con 0x7efd4c104a30 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.758+0000 7efd4be00700 1 -- 172.21.15.90:0/978350591 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd40011af0 con 0x7efd4c104a30 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 -- 172.21.15.90:0/978350591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 msgr2=0x7efd4c104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 --2- 172.21.15.90:0/978350591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd400098a0 tx=0x7efd40009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 -- 172.21.15.90:0/978350591 shutdown_connections 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 --2- 172.21.15.90:0/978350591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c104e40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 -- 172.21.15.90:0/978350591 >> 172.21.15.90:0/978350591 conn(0x7efd4c100060 msgr2=0x7efd4c1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 -- 172.21.15.90:0/978350591 shutdown_connections 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.759+0000 7efd54200700 1 -- 172.21.15.90:0/978350591 wait complete. 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.760+0000 7efd54200700 1 Processor -- start 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd54200700 1 -- start start 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd54200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd54200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c190150 con 0x7efd4c104a30 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:26.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd51800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40762/0 (socket says 172.21.15.90:40762) 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.761+0000 7efd51800700 1 -- 172.21.15.90:0/2144529240 learned_addr learned my addr 172.21.15.90:0/2144529240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd51800700 1 -- 172.21.15.90:0/2144529240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40009550 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd51800700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd40011670 tx=0x7efd40011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd40003a40 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd40003ba0 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd40024790 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.762+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c18e260 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.763+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c18e680 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.764+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7efd4001a430 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.764+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c062380 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.764+0000 7efd4a000700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 0x7efd3403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.764+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd4004c820 con 0x7efd4c104a30 2024-04-24T08:12:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.765+0000 7efd50e00700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 0x7efd3403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.765+0000 7efd50e00700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 0x7efd3403a990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd38006fd0 tx=0x7efd38006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.769+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd40010380 con 0x7efd4c104a30 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.985+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7efd4c18f050 con 0x7efd4c104a30 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.986+0000 7efd4a000700 1 -- 172.21.15.90:0/2144529240 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7efd40018b40 con 0x7efd4c104a30 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 msgr2=0x7efd3403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 0x7efd3403a990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd38006fd0 tx=0x7efd38006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 msgr2=0x7efd4c18fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd40011670 tx=0x7efd40011b50 comp rx=0 tx=0).stop 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 shutdown_connections 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7efd340384e0 0x7efd3403a990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 --2- 172.21.15.90:0/2144529240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd4c104a30 0x7efd4c18fc10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.988+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 >> 172.21.15.90:0/2144529240 conn(0x7efd4c100060 msgr2=0x7efd4c079950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:26.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.989+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 shutdown_connections 2024-04-24T08:12:26.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:25.989+0000 7efd54200700 1 -- 172.21.15.90:0/2144529240 wait complete. 2024-04-24T08:12:26.667 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:12:26.667 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T08:12:26.686 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:12:26.686 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T08:12:26.686 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T08:12:26.686 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T08:12:26.690 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-04-24T08:12:26.690 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:26.690 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-04-24T08:12:26.690 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-04-24T08:12:26.691 INFO:teuthology.orchestra.run.smithi090.stdout: Password: uau1dtr5zc 2024-04-24T08:12:26.691 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:26.691 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config directory 2024-04-24T08:12:26.692 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-04-24T08:12:26.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:26 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2144529240' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.304+0000 7fee57e00700 1 Processor -- start 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.304+0000 7fee57e00700 1 -- start start 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.304+0000 7fee57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.304+0000 7fee57e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58080210 con 0x7fee5807f8c0 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.305+0000 7fee56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:28.239 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.305+0000 7fee56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40774/0 (socket says 172.21.15.90:40774) 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.305+0000 7fee56a00700 1 -- 172.21.15.90:0/2220777840 learned_addr learned my addr 172.21.15.90:0/2220777840 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.305+0000 7fee56a00700 1 -- 172.21.15.90:0/2220777840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58080350 con 0x7fee5807f8c0 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.306+0000 7fee56a00700 1 --2- 172.21.15.90:0/2220777840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee480098a0 tx=0x7fee48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900650295cdac7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.306+0000 7fee55600700 1 -- 172.21.15.90:0/2220777840 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee48004180 con 0x7fee5807f8c0 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.306+0000 7fee55600700 1 -- 172.21.15.90:0/2220777840 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee480042e0 con 0x7fee5807f8c0 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.306+0000 7fee55600700 1 -- 172.21.15.90:0/2220777840 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee48011af0 con 0x7fee5807f8c0 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 -- 172.21.15.90:0/2220777840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 msgr2=0x7fee5807fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 --2- 172.21.15.90:0/2220777840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee480098a0 tx=0x7fee48009bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 -- 172.21.15.90:0/2220777840 shutdown_connections 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 --2- 172.21.15.90:0/2220777840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5807fcd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 -- 172.21.15.90:0/2220777840 >> 172.21.15.90:0/2220777840 conn(0x7fee5807afe0 msgr2=0x7fee5807d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 -- 172.21.15.90:0/2220777840 shutdown_connections 2024-04-24T08:12:28.240 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.307+0000 7fee57e00700 1 -- 172.21.15.90:0/2220777840 wait complete. 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.308+0000 7fee57e00700 1 Processor -- start 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee57e00700 1 -- start start 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee57e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee581a01d0 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40782/0 (socket says 172.21.15.90:40782) 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.309+0000 7fee56a00700 1 -- 172.21.15.90:0/2254801963 learned_addr learned my addr 172.21.15.90:0/2254801963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee56a00700 1 -- 172.21.15.90:0/2254801963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009550 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee56a00700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee48011ad0 tx=0x7fee48011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee48003ab0 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee48003c10 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee480198e0 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581a03d0 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.310+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a07f0 con 0x7fee5807f8c0 2024-04-24T08:12:28.241 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.312+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fee48019b50 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.312+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58062380 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.312+0000 7fee4f400700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 0x7fee3c03a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.312+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee4804c360 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.312+0000 7fee56000700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 0x7fee3c03a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.313+0000 7fee56000700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 0x7fee3c03a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee44006fd0 tx=0x7fee44006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.318+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fee4801a430 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.531+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fee581a3640 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.532+0000 7fee4f400700 1 -- 172.21.15.90:0/2254801963 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fee48010380 con 0x7fee5807f8c0 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.535+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 msgr2=0x7fee3c03a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.535+0000 7fee57e00700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 0x7fee3c03a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee44006fd0 tx=0x7fee44006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.535+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 msgr2=0x7fee5819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:28.242 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.535+0000 7fee57e00700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee48011ad0 tx=0x7fee48011bb0 comp rx=0 tx=0).stop 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.535+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 shutdown_connections 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.536+0000 7fee57e00700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fee3c038530 0x7fee3c03a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.536+0000 7fee57e00700 1 --2- 172.21.15.90:0/2254801963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee5807f8c0 0x7fee5819fc90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.536+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 >> 172.21.15.90:0/2254801963 conn(0x7fee5807afe0 msgr2=0x7fee5807c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.536+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 shutdown_connections 2024-04-24T08:12:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:27.536+0000 7fee57e00700 1 -- 172.21.15.90:0/2254801963 wait complete. 2024-04-24T08:12:29.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.853+0000 7fe293e00700 1 Processor -- start 2024-04-24T08:12:29.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.853+0000 7fe293e00700 1 -- start start 2024-04-24T08:12:29.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.854+0000 7fe293e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.854+0000 7fe293e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0744b0 con 0x7fe28c104460 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.854+0000 7fe291400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.855+0000 7fe291400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40786/0 (socket says 172.21.15.90:40786) 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.855+0000 7fe291400700 1 -- 172.21.15.90:0/1300490212 learned_addr learned my addr 172.21.15.90:0/1300490212 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.855+0000 7fe291400700 1 -- 172.21.15.90:0/1300490212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0745f0 con 0x7fe28c104460 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.855+0000 7fe291400700 1 --2- 172.21.15.90:0/1300490212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e80 tx=0x7fe28400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b546653eec2b4858 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe28be00700 1 -- 172.21.15.90:0/1300490212 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe28400e070 con 0x7fe28c104460 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe28be00700 1 -- 172.21.15.90:0/1300490212 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe28400b8c0 con 0x7fe28c104460 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe28be00700 1 -- 172.21.15.90:0/1300490212 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2840044d0 con 0x7fe28c104460 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe293e00700 1 -- 172.21.15.90:0/1300490212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 msgr2=0x7fe28c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe293e00700 1 --2- 172.21.15.90:0/1300490212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e80 tx=0x7fe28400b330 comp rx=0 tx=0).stop 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe293e00700 1 -- 172.21.15.90:0/1300490212 shutdown_connections 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe293e00700 1 --2- 172.21.15.90:0/1300490212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c104870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:29.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.856+0000 7fe293e00700 1 -- 172.21.15.90:0/1300490212 >> 172.21.15.90:0/1300490212 conn(0x7fe28c100070 msgr2=0x7fe28c102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:29.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.857+0000 7fe293e00700 1 -- 172.21.15.90:0/1300490212 shutdown_connections 2024-04-24T08:12:29.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.857+0000 7fe293e00700 1 -- 172.21.15.90:0/1300490212 wait complete. 2024-04-24T08:12:29.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe293e00700 1 Processor -- start 2024-04-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe293e00700 1 -- start start 2024-04-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe293e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe293e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19bfa0 con 0x7fe28c104460 2024-04-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe291400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe291400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40798/0 (socket says 172.21.15.90:40798) 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.858+0000 7fe291400700 1 -- 172.21.15.90:0/4067638426 learned_addr learned my addr 172.21.15.90:0/4067638426 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe291400700 1 -- 172.21.15.90:0/4067638426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284009b30 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe291400700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe284004940 tx=0x7fe284004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe28400e040 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe28400bc40 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2840126e0 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.859+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c19c1a0 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.860+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c19c620 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.861+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0623f0 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.861+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fe284019040 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.861+0000 7fe28a000700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 0x7fe27003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.861+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe28404bf00 con 0x7fe28c104460 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.861+0000 7fe290a00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 0x7fe27003a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.862+0000 7fe290a00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 0x7fe27003a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe278006fd0 tx=0x7fe278006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:29.823 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:28.866+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe28c0623f0 con 0x7fe28c104460 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.194+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe28c195270 con 0x7fe28c104460 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.199+0000 7fe28a000700 1 -- 172.21.15.90:0/4067638426 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe28401f020 con 0x7fe28c104460 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.201+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 msgr2=0x7fe27003a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.201+0000 7fe293e00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 0x7fe27003a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe278006fd0 tx=0x7fe278006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 msgr2=0x7fe28c19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe284004940 tx=0x7fe284004a20 comp rx=0 tx=0).stop 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 shutdown_connections 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe270038530 0x7fe27003a9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 --2- 172.21.15.90:0/4067638426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe28c104460 0x7fe28c19ba60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 >> 172.21.15.90:0/4067638426 conn(0x7fe28c100070 msgr2=0x7fe28c190020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 shutdown_connections 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T08:12:29.202+0000 7fe293e00700 1 -- 172.21.15.90:0/4067638426 wait complete. 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-04-24T08:12:29.915 INFO:tasks.cephadm:Fetching config... 2024-04-24T08:12:29.916 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:12:29.916 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T08:12:29.963 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T08:12:29.963 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:12:29.963 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T08:12:30.047 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T08:12:30.047 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:12:30.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/keyring of=/dev/stdout 2024-04-24T08:12:30.157 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T08:12:30.157 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:12:30.157 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T08:12:30.250 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T08:12:30.251 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBfZAJ6yYXnBJwi1hSs1xNa4ok8VaY7rm226Gc3B3UlYZ6Q/ZYatB2q5fkk5rMH7IV/BRhOMexJcfdkadwk7D+tWVPtVCFHHk3MqSW2DpUzBqiGTbQ/hNOPNbyskoJ7m9BuDzq+pQ1zQ/0eKHrFf1N6AkPENn54Hx9FvkJWOmyo7bsUx5veX8oECv9ChX7yEJQyyZj+sJyQZZxiM1wvfq87VsI6y4AHdzSH2f/HqFGp2ZCLUscZ2gBigydGXcJg6re0KKcVlomWNPe/JUJdGoACQCbDkT5nZY5BRoqrzu3KzRAeN0RjHAfOLhQieLEC44sIZ9UKKJqDDGQZFVEMaEHPHRxN2y5Td8rigE8z4dFcSLlZRN0btMdxfISjb4JrT/vRUta+O6RTi45A89FqVjknqyn5Lek0QIzhFov3JKnGOIPqW+jXxLbHTl0ciKXKdip/S7y77TB0jXbCeUwOcXiN6Ob5+IHyJW0QrzeqsrSjYVB5RAk1CGjrHH0TDGAgUs= ceph-10d437de-0212-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:12:30.369 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBfZAJ6yYXnBJwi1hSs1xNa4ok8VaY7rm226Gc3B3UlYZ6Q/ZYatB2q5fkk5rMH7IV/BRhOMexJcfdkadwk7D+tWVPtVCFHHk3MqSW2DpUzBqiGTbQ/hNOPNbyskoJ7m9BuDzq+pQ1zQ/0eKHrFf1N6AkPENn54Hx9FvkJWOmyo7bsUx5veX8oECv9ChX7yEJQyyZj+sJyQZZxiM1wvfq87VsI6y4AHdzSH2f/HqFGp2ZCLUscZ2gBigydGXcJg6re0KKcVlomWNPe/JUJdGoACQCbDkT5nZY5BRoqrzu3KzRAeN0RjHAfOLhQieLEC44sIZ9UKKJqDDGQZFVEMaEHPHRxN2y5Td8rigE8z4dFcSLlZRN0btMdxfISjb4JrT/vRUta+O6RTi45A89FqVjknqyn5Lek0QIzhFov3JKnGOIPqW+jXxLbHTl0ciKXKdip/S7y77TB0jXbCeUwOcXiN6Ob5+IHyJW0QrzeqsrSjYVB5RAk1CGjrHH0TDGAgUs= ceph-10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:12:30.399 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:30 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/4067638426' entity='client.admin' 2024-04-24T08:12:30.415 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBfZAJ6yYXnBJwi1hSs1xNa4ok8VaY7rm226Gc3B3UlYZ6Q/ZYatB2q5fkk5rMH7IV/BRhOMexJcfdkadwk7D+tWVPtVCFHHk3MqSW2DpUzBqiGTbQ/hNOPNbyskoJ7m9BuDzq+pQ1zQ/0eKHrFf1N6AkPENn54Hx9FvkJWOmyo7bsUx5veX8oECv9ChX7yEJQyyZj+sJyQZZxiM1wvfq87VsI6y4AHdzSH2f/HqFGp2ZCLUscZ2gBigydGXcJg6re0KKcVlomWNPe/JUJdGoACQCbDkT5nZY5BRoqrzu3KzRAeN0RjHAfOLhQieLEC44sIZ9UKKJqDDGQZFVEMaEHPHRxN2y5Td8rigE8z4dFcSLlZRN0btMdxfISjb4JrT/vRUta+O6RTi45A89FqVjknqyn5Lek0QIzhFov3JKnGOIPqW+jXxLbHTl0ciKXKdip/S7y77TB0jXbCeUwOcXiN6Ob5+IHyJW0QrzeqsrSjYVB5RAk1CGjrHH0TDGAgUs= ceph-10d437de-0212-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:12:30.540 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBfZAJ6yYXnBJwi1hSs1xNa4ok8VaY7rm226Gc3B3UlYZ6Q/ZYatB2q5fkk5rMH7IV/BRhOMexJcfdkadwk7D+tWVPtVCFHHk3MqSW2DpUzBqiGTbQ/hNOPNbyskoJ7m9BuDzq+pQ1zQ/0eKHrFf1N6AkPENn54Hx9FvkJWOmyo7bsUx5veX8oECv9ChX7yEJQyyZj+sJyQZZxiM1wvfq87VsI6y4AHdzSH2f/HqFGp2ZCLUscZ2gBigydGXcJg6re0KKcVlomWNPe/JUJdGoACQCbDkT5nZY5BRoqrzu3KzRAeN0RjHAfOLhQieLEC44sIZ9UKKJqDDGQZFVEMaEHPHRxN2y5Td8rigE8z4dFcSLlZRN0btMdxfISjb4JrT/vRUta+O6RTi45A89FqVjknqyn5Lek0QIzhFov3JKnGOIPqW+jXxLbHTl0ciKXKdip/S7y77TB0jXbCeUwOcXiN6Ob5+IHyJW0QrzeqsrSjYVB5RAk1CGjrHH0TDGAgUs= ceph-10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:12:30.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T08:12:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:12:33.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.509+0000 7f5ce2a00700 1 -- 172.21.15.90:0/84281209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 msgr2=0x7f5cdc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.509+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/84281209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc100230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009910 tx=0x7f5ccc009c20 comp rx=0 tx=0).stop 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.510+0000 7f5ce2a00700 1 -- 172.21.15.90:0/84281209 shutdown_connections 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.510+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/84281209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc100230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.510+0000 7f5ce2a00700 1 -- 172.21.15.90:0/84281209 >> 172.21.15.90:0/84281209 conn(0x7f5cdc0fb3d0 msgr2=0x7f5cdc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.510+0000 7f5ce2a00700 1 -- 172.21.15.90:0/84281209 shutdown_connections 2024-04-24T08:12:33.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.510+0000 7f5ce2a00700 1 -- 172.21.15.90:0/84281209 wait complete. 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.511+0000 7f5ce2a00700 1 Processor -- start 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.511+0000 7f5ce2a00700 1 -- start start 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.511+0000 7f5ce2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.511+0000 7f5ce2a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc197690 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.511+0000 7f5cdbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.512+0000 7f5cdbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33642/0 (socket says 172.21.15.90:33642) 2024-04-24T08:12:33.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.512+0000 7f5cdbe00700 1 -- 172.21.15.90:0/1224540900 learned_addr learned my addr 172.21.15.90:0/1224540900 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:33.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.512+0000 7f5cdbe00700 1 -- 172.21.15.90:0/1224540900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0095f0 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.512+0000 7f5cdbe00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0092e0 tx=0x7f5ccc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:33.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.512+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ccc01d070 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.513+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ccc00be10 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.513+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc197890 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.513+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ccc00f460 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.513+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc197d30 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.514+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f5ccc022470 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.514+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc190e80 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.515+0000 7f5cd8c00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 0x7f5cc003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:33.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.515+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ccc04c360 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.515+0000 7f5cdb400700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 0x7f5cc003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:33.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.515+0000 7f5cdb400700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 0x7f5cc003a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.520+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5ccc00b960 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.740+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5cdc1008b0 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.745+0000 7f5cd8c00700 1 -- 172.21.15.90:0/1224540900 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5cdc1008b0 con 0x7f5cdc0ffe20 2024-04-24T08:12:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 msgr2=0x7f5cc003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 0x7f5cc003a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 msgr2=0x7f5cdc197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0092e0 tx=0x7f5ccc009e60 comp rx=0 tx=0).stop 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 shutdown_connections 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f5cc00384e0 0x7f5cc003a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 --2- 172.21.15.90:0/1224540900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cdc0ffe20 0x7f5cdc197150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 >> 172.21.15.90:0/1224540900 conn(0x7f5cdc0fb3d0 msgr2=0x7f5cdc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 shutdown_connections 2024-04-24T08:12:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:33.748+0000 7f5ce2a00700 1 -- 172.21.15.90:0/1224540900 wait complete. 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:12:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:33 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1224540900' entity='client.admin' 2024-04-24T08:12:34.520 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T08:12:34.520 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T08:12:35.147 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:34 smithi090 ceph-mon[25016]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:12:35.330 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:12:38.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.302+0000 7fe505000700 1 -- 172.21.15.90:0/3163723404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 msgr2=0x7fe500100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:38.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.302+0000 7fe505000700 1 --2- 172.21.15.90:0/3163723404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500100230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009910 tx=0x7fe4f0009c20 comp rx=0 tx=0).stop 2024-04-24T08:12:38.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.303+0000 7fe505000700 1 -- 172.21.15.90:0/3163723404 shutdown_connections 2024-04-24T08:12:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.303+0000 7fe505000700 1 --2- 172.21.15.90:0/3163723404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500100230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.303+0000 7fe505000700 1 -- 172.21.15.90:0/3163723404 >> 172.21.15.90:0/3163723404 conn(0x7fe5000fb3d0 msgr2=0x7fe5000fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.303+0000 7fe505000700 1 -- 172.21.15.90:0/3163723404 shutdown_connections 2024-04-24T08:12:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.303+0000 7fe505000700 1 -- 172.21.15.90:0/3163723404 wait complete. 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.304+0000 7fe505000700 1 Processor -- start 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe505000700 1 -- start start 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe505000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe505000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe500075030 con 0x7fe5000ffe20 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe4fde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe4fde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33662/0 (socket says 172.21.15.90:33662) 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.305+0000 7fe4fde00700 1 -- 172.21.15.90:0/3430076382 learned_addr learned my addr 172.21.15.90:0/3430076382 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe4fde00700 1 -- 172.21.15.90:0/3430076382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00095f0 con 0x7fe5000ffe20 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe4fde00700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00092e0 tx=0x7fe4f0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4f001d070 con 0x7fe5000ffe20 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe500073140 con 0x7fe5000ffe20 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe4f000be10 con 0x7fe5000ffe20 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4f000f460 con 0x7fe5000ffe20 2024-04-24T08:12:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.306+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5000735e0 con 0x7fe5000ffe20 2024-04-24T08:12:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.308+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe500062380 con 0x7fe5000ffe20 2024-04-24T08:12:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.309+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fe4f0022470 con 0x7fe5000ffe20 2024-04-24T08:12:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.309+0000 7fe4f6a00700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 0x7fe4e403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.309+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe4f004c370 con 0x7fe5000ffe20 2024-04-24T08:12:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.310+0000 7fe4fd400700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 0x7fe4e403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.310+0000 7fe4fd400700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 0x7fe4e403a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec006fd0 tx=0x7fe4ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:38.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.316+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe4f000b960 con 0x7fe5000ffe20 2024-04-24T08:12:38.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.543+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe500073820 con 0x7fe4e4038490 2024-04-24T08:12:38.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.552+0000 7fe4f6a00700 1 -- 172.21.15.90:0/3430076382 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe500073820 con 0x7fe4e4038490 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 msgr2=0x7fe4e403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 0x7fe4e403a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec006fd0 tx=0x7fe4ec006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 msgr2=0x7fe500074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00092e0 tx=0x7fe4f0009e60 comp rx=0 tx=0).stop 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 shutdown_connections 2024-04-24T08:12:38.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fe4e4038490 0x7fe4e403a940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:38.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 --2- 172.21.15.90:0/3430076382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5000ffe20 0x7fe500074af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:38.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.555+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 >> 172.21.15.90:0/3430076382 conn(0x7fe5000fb3d0 msgr2=0x7fe5000fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:38.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.556+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 shutdown_connections 2024-04-24T08:12:38.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:38.556+0000 7fe505000700 1 -- 172.21.15.90:0/3430076382 wait complete. 2024-04-24T08:12:39.497 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-04-24T08:12:39.497 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:12:39.497 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-04-24T08:12:39.554 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:12:39.554 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:12:39.651 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-04-24T08:12:39.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch host add smithi149 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:39.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:12:39.706 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T08:12:39.706 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:39 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:12:40.549 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:12:40.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:40 smithi090 ceph-mon[25016]: Deploying daemon crash.smithi090 on smithi090 2024-04-24T08:12:41.644 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:41 smithi090 ceph-mon[25016]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:43.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.030+0000 7faf47600700 1 -- 172.21.15.90:0/1086497092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 msgr2=0x7faf40100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.030+0000 7faf47600700 1 --2- 172.21.15.90:0/1086497092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40100230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf34009910 tx=0x7faf34009c20 comp rx=0 tx=0).stop 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.031+0000 7faf47600700 1 -- 172.21.15.90:0/1086497092 shutdown_connections 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.031+0000 7faf47600700 1 --2- 172.21.15.90:0/1086497092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40100230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.031+0000 7faf47600700 1 -- 172.21.15.90:0/1086497092 >> 172.21.15.90:0/1086497092 conn(0x7faf400fb3d0 msgr2=0x7faf400fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.031+0000 7faf47600700 1 -- 172.21.15.90:0/1086497092 shutdown_connections 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.031+0000 7faf47600700 1 -- 172.21.15.90:0/1086497092 wait complete. 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.032+0000 7faf47600700 1 Processor -- start 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.032+0000 7faf47600700 1 -- start start 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf47600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf47600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401954c0 con 0x7faf400ffe20 2024-04-24T08:12:43.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf44c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf44c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50982/0 (socket says 172.21.15.90:50982) 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf44c00700 1 -- 172.21.15.90:0/3765432227 learned_addr learned my addr 172.21.15.90:0/3765432227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.033+0000 7faf44c00700 1 -- 172.21.15.90:0/3765432227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf340095f0 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf44c00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf340092e0 tx=0x7faf34009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf3401d070 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faf3400be10 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf3400f460 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf47600700 1 -- 172.21.15.90:0/3765432227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401956c0 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.034+0000 7faf47600700 1 -- 172.21.15.90:0/3765432227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf40195b60 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.035+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7faf34022470 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.035+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1c0052f0 con 0x7faf400ffe20 2024-04-24T08:12:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.036+0000 7faf3d600700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 0x7faf2803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:43.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.036+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faf3404c3c0 con 0x7faf400ffe20 2024-04-24T08:12:43.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.036+0000 7faf3fe00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 0x7faf2803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:43.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.037+0000 7faf3fe00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 0x7faf2803a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006fd0 tx=0x7faf2c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:43.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.042+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faf3400b960 con 0x7faf400ffe20 2024-04-24T08:12:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:43.262+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7faf1c000bc0 con 0x7faf28038490 2024-04-24T08:12:44.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:43 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:44.683 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:44 smithi090 ceph-mon[25016]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:12:44.683 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:44 smithi090 ceph-mon[25016]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-24T08:12:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:45 smithi090 ceph-mon[25016]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:45 smithi090 ceph-mon[25016]: Deploying cephadm binary to smithi149 2024-04-24T08:12:47.422 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:47 smithi090 ceph-mon[25016]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:47.437 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-04-24T08:12:47.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.433+0000 7faf3d600700 1 -- 172.21.15.90:0/3765432227 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7faf1c000bc0 con 0x7faf28038490 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 msgr2=0x7faf2803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 0x7faf2803a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006fd0 tx=0x7faf2c006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 msgr2=0x7faf40194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf340092e0 tx=0x7faf34009e60 comp rx=0 tx=0).stop 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 shutdown_connections 2024-04-24T08:12:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7faf28038490 0x7faf2803a940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 --2- 172.21.15.90:0/3765432227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf400ffe20 0x7faf40194f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.437+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 >> 172.21.15.90:0/3765432227 conn(0x7faf400fb3d0 msgr2=0x7faf400fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.438+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 shutdown_connections 2024-04-24T08:12:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:47.438+0000 7faf32a00700 1 -- 172.21.15.90:0/3765432227 wait complete. 2024-04-24T08:12:48.687 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:48 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:48.687 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:48 smithi090 ceph-mon[25016]: Added host smithi149 2024-04-24T08:12:49.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T08:12:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:12:49.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:49 smithi090 ceph-mon[25016]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:49.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:49 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:49.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:49 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:49.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:49 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:49.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:49 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:50.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:50 smithi090 ceph-mon[25016]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-24T08:12:51.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 -- 172.21.15.90:0/1926580832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 msgr2=0x7f83880731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 --2- 172.21.15.90:0/1926580832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f83880731e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8378009910 tx=0x7f8378009c20 comp rx=0 tx=0).stop 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 -- 172.21.15.90:0/1926580832 shutdown_connections 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 --2- 172.21.15.90:0/1926580832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f83880731e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 -- 172.21.15.90:0/1926580832 >> 172.21.15.90:0/1926580832 conn(0x7f83880fb3d0 msgr2=0x7f83880fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 -- 172.21.15.90:0/1926580832 shutdown_connections 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.505+0000 7f838d000700 1 -- 172.21.15.90:0/1926580832 wait complete. 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.506+0000 7f838d000700 1 Processor -- start 2024-04-24T08:12:51.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f838d000700 1 -- start start 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f838d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f838d000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83881976b0 con 0x7f8388074d80 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f8385e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f8385e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42342/0 (socket says 172.21.15.90:42342) 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.507+0000 7f8385e00700 1 -- 172.21.15.90:0/67257507 learned_addr learned my addr 172.21.15.90:0/67257507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:51.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f8385e00700 1 -- 172.21.15.90:0/67257507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780095f0 con 0x7f8388074d80 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f8385e00700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83780092e0 tx=0x7f8378009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837801d070 con 0x7f8388074d80 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f837800be10 con 0x7f8388074d80 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837800f460 con 0x7f8388074d80 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.508+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83881978b0 con 0x7f8388074d80 2024-04-24T08:12:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.509+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8388197d50 con 0x7f8388074d80 2024-04-24T08:12:51.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.510+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8388190d00 con 0x7f8388074d80 2024-04-24T08:12:51.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.510+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8378022470 con 0x7f8388074d80 2024-04-24T08:12:51.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.510+0000 7f837ea00700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 0x7f836c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:51.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.510+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f837804c510 con 0x7f8388074d80 2024-04-24T08:12:51.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.511+0000 7f8385400700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 0x7f836c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:51.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.512+0000 7f8385400700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 0x7f836c03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8374006fd0 tx=0x7f8374006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:51.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.516+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f837800b960 con 0x7f8388074d80 2024-04-24T08:12:51.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.746+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8388061190 con 0x7f836c0384e0 2024-04-24T08:12:51.748 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:51 smithi090 ceph-mon[25016]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:51.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.749+0000 7f837ea00700 1 -- 172.21.15.90:0/67257507 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f8388061190 con 0x7f836c0384e0 2024-04-24T08:12:51.750 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:12:51.750 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-04-24T08:12:51.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.752+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 msgr2=0x7f836c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.752+0000 7f838d000700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 0x7f836c03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8374006fd0 tx=0x7f8374006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.752+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 msgr2=0x7f8388197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.752+0000 7f838d000700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83780092e0 tx=0x7f8378009e60 comp rx=0 tx=0).stop 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 shutdown_connections 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f836c0384e0 0x7f836c03a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:51.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 --2- 172.21.15.90:0/67257507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8388074d80 0x7f8388197170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:51.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 >> 172.21.15.90:0/67257507 conn(0x7f83880fb3d0 msgr2=0x7f83880fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:51.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 shutdown_connections 2024-04-24T08:12:51.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:51.753+0000 7f838d000700 1 -- 172.21.15.90:0/67257507 wait complete. 2024-04-24T08:12:52.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:52 smithi090 ceph-mon[25016]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:12:52.725 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T08:12:52.725 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T08:12:53.112 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:12:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:53 smithi090 ceph-mon[25016]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:53 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:12:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:55 smithi090 ceph-mon[25016]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:57.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:57 smithi090 ceph-mon[25016]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.341+0000 7fa5bfe00700 1 -- 172.21.15.90:0/3605490637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 msgr2=0x7fa5c0102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.341+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/3605490637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0102260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009910 tx=0x7fa5b0009c20 comp rx=0 tx=0).stop 2024-04-24T08:12:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.342+0000 7fa5bfe00700 1 -- 172.21.15.90:0/3605490637 shutdown_connections 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.342+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/3605490637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0102260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.342+0000 7fa5bfe00700 1 -- 172.21.15.90:0/3605490637 >> 172.21.15.90:0/3605490637 conn(0x7fa5c00fd530 msgr2=0x7fa5c00ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.342+0000 7fa5bfe00700 1 -- 172.21.15.90:0/3605490637 shutdown_connections 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.342+0000 7fa5bfe00700 1 -- 172.21.15.90:0/3605490637 wait complete. 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bfe00700 1 Processor -- start 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bfe00700 1 -- start start 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bfe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0197540 con 0x7fa5c0101e50 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42348/0 (socket says 172.21.15.90:42348) 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.343+0000 7fa5bea00700 1 -- 172.21.15.90:0/1710954738 learned_addr learned my addr 172.21.15.90:0/1710954738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5bea00700 1 -- 172.21.15.90:0/1710954738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00095f0 con 0x7fa5c0101e50 2024-04-24T08:12:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5bea00700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005b40 tx=0x7fa5b0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b001d070 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa5b000be10 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c0197740 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b000f460 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.344+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c0197be0 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.346+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fa5b00227e0 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.346+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c0190ab0 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.346+0000 7fa5b7400700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 0x7fa5a403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.346+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa5b004c2b0 con 0x7fa5c0101e50 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.346+0000 7fa5be000700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 0x7fa5a403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:12:57.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.347+0000 7fa5be000700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 0x7fa5a403a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac006fd0 tx=0x7fa5ac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:12:57.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.352+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa5b002b3b0 con 0x7fa5c0101e50 2024-04-24T08:12:57.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:57.575+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa5c002cc30 con 0x7fa5c0101e50 2024-04-24T08:12:58.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.106+0000 7fa5b7400700 1 -- 172.21.15.90:0/1710954738 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa5b0027030 con 0x7fa5c0101e50 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 msgr2=0x7fa5a403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 0x7fa5a403a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac006fd0 tx=0x7fa5ac006e40 comp rx=0 tx=0).stop 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 msgr2=0x7fa5c0197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005b40 tx=0x7fa5b0004740 comp rx=0 tx=0).stop 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 shutdown_connections 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fa5a40384e0 0x7fa5a403a990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 --2- 172.21.15.90:0/1710954738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5c0101e50 0x7fa5c0197000 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:12:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.109+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 >> 172.21.15.90:0/1710954738 conn(0x7fa5c00fd530 msgr2=0x7fa5c00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:12:58.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.110+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 shutdown_connections 2024-04-24T08:12:58.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:12:58.110+0000 7fa5bfe00700 1 -- 172.21.15.90:0/1710954738 wait complete. 2024-04-24T08:12:58.111 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-04-24T08:12:58.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1710954738' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:12:58.899 INFO:tasks.cephadm:Adding mon.smithi090 on smithi090 2024-04-24T08:12:58.899 INFO:tasks.cephadm:Adding mon.smithi149 on smithi149 2024-04-24T08:12:58.900 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi090:172.21.15.90=smithi090;smithi149:172.21.15.149=smithi149' 2024-04-24T08:12:59.287 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:12:59.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:59 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1710954738' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:12:59.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:59 smithi090 ceph-mon[25016]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:12:59.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:12:59 smithi090 ceph-mon[25016]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:12:59.393 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:01.643 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:01 smithi090 ceph-mon[25016]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.028+0000 7f3e40a00700 1 -- 172.21.15.149:0/2237587590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 msgr2=0x7f3e3c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.028+0000 7f3e40a00700 1 --2- 172.21.15.149:0/2237587590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c102450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009910 tx=0x7f3e2c009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.029+0000 7f3e40a00700 1 -- 172.21.15.149:0/2237587590 shutdown_connections 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.029+0000 7f3e40a00700 1 --2- 172.21.15.149:0/2237587590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c102450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.029+0000 7f3e40a00700 1 -- 172.21.15.149:0/2237587590 >> 172.21.15.149:0/2237587590 conn(0x7f3e3c0fd700 msgr2=0x7f3e3c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.029+0000 7f3e40a00700 1 -- 172.21.15.149:0/2237587590 shutdown_connections 2024-04-24T08:13:02.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.029+0000 7f3e40a00700 1 -- 172.21.15.149:0/2237587590 wait complete. 2024-04-24T08:13:02.032 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.030+0000 7f3e40a00700 1 Processor -- start 2024-04-24T08:13:02.032 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.030+0000 7f3e40a00700 1 -- start start 2024-04-24T08:13:02.032 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.030+0000 7f3e40a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:02.032 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.030+0000 7f3e40a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c197710 con 0x7f3e3c102040 2024-04-24T08:13:02.032 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.031+0000 7f3e39e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:02.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.031+0000 7f3e39e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:45410/0 (socket says 172.21.15.149:45410) 2024-04-24T08:13:02.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.031+0000 7f3e39e00700 1 -- 172.21.15.149:0/3323183868 learned_addr learned my addr 172.21.15.149:0/3323183868 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:02.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.032+0000 7f3e39e00700 1 -- 172.21.15.149:0/3323183868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0095f0 con 0x7f3e3c102040 2024-04-24T08:13:02.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.032+0000 7f3e39e00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005950 tx=0x7f3e2c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:02.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.032+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c01d070 con 0x7f3e3c102040 2024-04-24T08:13:02.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.033+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c197910 con 0x7f3e3c102040 2024-04-24T08:13:02.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.033+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e2c00be10 con 0x7f3e3c102040 2024-04-24T08:13:02.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.033+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c00f460 con 0x7f3e3c102040 2024-04-24T08:13:02.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.033+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c197db0 con 0x7f3e3c102040 2024-04-24T08:13:02.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.034+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c190d20 con 0x7f3e3c102040 2024-04-24T08:13:02.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.034+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f3e2c0227c0 con 0x7f3e3c102040 2024-04-24T08:13:02.036 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.035+0000 7f3e32a00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 0x7f3e2003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:02.036 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.035+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e2c04c4d0 con 0x7f3e3c102040 2024-04-24T08:13:02.036 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.036+0000 7f3e39400700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 0x7f3e2003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:02.038 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.037+0000 7f3e39400700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 0x7f3e2003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e28006fd0 tx=0x7f3e28006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:02.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.047+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3e2c02bba0 con 0x7f3e3c102040 2024-04-24T08:13:02.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.277+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 --> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi090:172.21.15.90=smithi090;smithi149:172.21.15.149=smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e3c061300 con 0x7f3e20038490 2024-04-24T08:13:02.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.286+0000 7f3e32a00700 1 -- 172.21.15.149:0/3323183868 <== mgr.14164 v2:172.21.15.90:6800/3702780720 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3e3c061300 con 0x7f3e20038490 2024-04-24T08:13:02.288 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 msgr2=0x7f3e2003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 0x7f3e2003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e28006fd0 tx=0x7f3e28006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 msgr2=0x7f3e3c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005950 tx=0x7f3e2c009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 shutdown_connections 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f3e20038490 0x7f3e2003a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 --2- 172.21.15.149:0/3323183868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c102040 0x7f3e3c1971d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:02.291 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.289+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 >> 172.21.15.149:0/3323183868 conn(0x7f3e3c0fd700 msgr2=0x7f3e3c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:02.292 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.290+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 shutdown_connections 2024-04-24T08:13:02.292 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:02.290+0000 7f3e40a00700 1 -- 172.21.15.149:0/3323183868 wait complete. 2024-04-24T08:13:03.081 DEBUG:teuthology.orchestra.run.smithi149:mon.smithi149> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi149.service 2024-04-24T08:13:03.087 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:03.087 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:03.475 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi090:172.21.15.90=smithi090;smithi149:172.21.15.149=smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: Saving service mon spec with placement smithi090:172.21.15.90=smithi090;smithi149:172.21.15.149=smithi149;count:2 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T08:13:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:03.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:03 smithi090 ceph-mon[25016]: Deploying daemon grafana.smithi090 on smithi090 2024-04-24T08:13:03.577 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:04.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:04 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:05.615 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 -- 172.21.15.149:0/3877262765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 msgr2=0x7f40380731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:05.615 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 --2- 172.21.15.149:0/3877262765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f40380731e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4028009910 tx=0x7f4028009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:05.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 -- 172.21.15.149:0/3877262765 shutdown_connections 2024-04-24T08:13:05.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 --2- 172.21.15.149:0/3877262765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f40380731e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:05.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 -- 172.21.15.149:0/3877262765 >> 172.21.15.149:0/3877262765 conn(0x7f40380fb1e0 msgr2=0x7f40380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:05.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 -- 172.21.15.149:0/3877262765 shutdown_connections 2024-04-24T08:13:05.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.614+0000 7f403d600700 1 -- 172.21.15.149:0/3877262765 wait complete. 2024-04-24T08:13:05.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.615+0000 7f403d600700 1 Processor -- start 2024-04-24T08:13:05.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.615+0000 7f403d600700 1 -- start start 2024-04-24T08:13:05.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.616+0000 7f403d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.616+0000 7f403d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038197510 con 0x7f4038074d80 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.616+0000 7f4037e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.617+0000 7f4037e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:45426/0 (socket says 172.21.15.149:45426) 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.617+0000 7f4037e00700 1 -- 172.21.15.149:0/4078178244 learned_addr learned my addr 172.21.15.149:0/4078178244 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.617+0000 7f4037e00700 1 -- 172.21.15.149:0/4078178244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40280095f0 con 0x7f4038074d80 2024-04-24T08:13:05.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f4037e00700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4028005b40 tx=0x7f4028004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:05.619 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f402801d070 con 0x7f4038074d80 2024-04-24T08:13:05.619 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4038197710 con 0x7f4038074d80 2024-04-24T08:13:05.619 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f402800be10 con 0x7f4038074d80 2024-04-24T08:13:05.619 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f402800f460 con 0x7f4038074d80 2024-04-24T08:13:05.619 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.618+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4038197bb0 con 0x7f4038074d80 2024-04-24T08:13:05.620 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.619+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4038190b30 con 0x7f4038074d80 2024-04-24T08:13:05.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.620+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f40280227e0 con 0x7f4038074d80 2024-04-24T08:13:05.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.620+0000 7f4034c00700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 0x7f401c03a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:05.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.620+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f402804c450 con 0x7f4038074d80 2024-04-24T08:13:05.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.621+0000 7f4037400700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 0x7f401c03a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:05.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.622+0000 7f4037400700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 0x7f401c03a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4024006fd0 tx=0x7f4024006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:05.636 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.635+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f402802ba00 con 0x7f4038074d80 2024-04-24T08:13:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:05 smithi090 ceph-mon[25016]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:05.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.935+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f403802cc30 con 0x7f4038074d80 2024-04-24T08:13:05.938 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.937+0000 7f4034c00700 1 -- 172.21.15.149:0/4078178244 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4028027070 con 0x7f4038074d80 2024-04-24T08:13:05.938 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:05.938 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 msgr2=0x7f401c03a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 0x7f401c03a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4024006fd0 tx=0x7f4024006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 msgr2=0x7f4038196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4028005b40 tx=0x7f4028004740 comp rx=0 tx=0).stop 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 shutdown_connections 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f401c038530 0x7f401c03a9e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:05.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 --2- 172.21.15.149:0/4078178244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4038074d80 0x7f4038196fd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:05.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 >> 172.21.15.149:0/4078178244 conn(0x7f40380fb1e0 msgr2=0x7f40380fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:05.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 shutdown_connections 2024-04-24T08:13:05.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:05.940+0000 7f403d600700 1 -- 172.21.15.149:0/4078178244 wait complete. 2024-04-24T08:13:05.942 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:06.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:06 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/4078178244' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:07.689 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:07.689 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:07 smithi090 ceph-mon[25016]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:08.066 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:08.162 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:09 smithi090 ceph-mon[25016]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:10.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.181+0000 7f46c1800700 1 -- 172.21.15.149:0/4132285986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 msgr2=0x7f46bc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:10.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.181+0000 7f46c1800700 1 --2- 172.21.15.149:0/4132285986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc102c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009910 tx=0x7f46ac009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:10.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.181+0000 7f46c1800700 1 -- 172.21.15.149:0/4132285986 shutdown_connections 2024-04-24T08:13:10.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.181+0000 7f46c1800700 1 --2- 172.21.15.149:0/4132285986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc102c90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.181+0000 7f46c1800700 1 -- 172.21.15.149:0/4132285986 >> 172.21.15.149:0/4132285986 conn(0x7f46bc0fa2b0 msgr2=0x7f46bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.182+0000 7f46c1800700 1 -- 172.21.15.149:0/4132285986 shutdown_connections 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.182+0000 7f46c1800700 1 -- 172.21.15.149:0/4132285986 wait complete. 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.183+0000 7f46c1800700 1 Processor -- start 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.183+0000 7f46c1800700 1 -- start start 2024-04-24T08:13:10.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.184+0000 7f46c1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:10.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.184+0000 7f46c1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc1976c0 con 0x7f46bc1008b0 2024-04-24T08:13:10.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.185+0000 7f46baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:10.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.185+0000 7f46baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:53386/0 (socket says 172.21.15.149:53386) 2024-04-24T08:13:10.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.185+0000 7f46baa00700 1 -- 172.21.15.149:0/3932320181 learned_addr learned my addr 172.21.15.149:0/3932320181 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:10.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.185+0000 7f46baa00700 1 -- 172.21.15.149:0/3932320181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0095f0 con 0x7f46bc1008b0 2024-04-24T08:13:10.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46baa00700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0092e0 tx=0x7f46ac009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:10.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46ac01d070 con 0x7f46bc1008b0 2024-04-24T08:13:10.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc1978c0 con 0x7f46bc1008b0 2024-04-24T08:13:10.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f46ac00be10 con 0x7f46bc1008b0 2024-04-24T08:13:10.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46ac00f460 con 0x7f46bc1008b0 2024-04-24T08:13:10.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.186+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc197d60 con 0x7f46bc1008b0 2024-04-24T08:13:10.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.187+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46bc190f00 con 0x7f46bc1008b0 2024-04-24T08:13:10.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.188+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f46ac022470 con 0x7f46bc1008b0 2024-04-24T08:13:10.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.188+0000 7f46b3400700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 0x7f46a003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:10.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.188+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f46ac04c370 con 0x7f46bc1008b0 2024-04-24T08:13:10.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.189+0000 7f46ba000700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 0x7f46a003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:10.191 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.190+0000 7f46ba000700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 0x7f46a003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006fd0 tx=0x7f46a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:10.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.204+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f46ac00b960 con 0x7f46bc1008b0 2024-04-24T08:13:10.497 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.496+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f46bc02cc30 con 0x7f46bc1008b0 2024-04-24T08:13:10.498 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.497+0000 7f46b3400700 1 -- 172.21.15.149:0/3932320181 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f46ac027070 con 0x7f46bc1008b0 2024-04-24T08:13:10.498 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:10.498 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.499+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 msgr2=0x7f46a003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 0x7f46a003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006fd0 tx=0x7f46a8006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 msgr2=0x7f46bc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0092e0 tx=0x7f46ac009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 shutdown_connections 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f46a0038490 0x7f46a003a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:10.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 --2- 172.21.15.149:0/3932320181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46bc1008b0 0x7f46bc197180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:10.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 >> 172.21.15.149:0/3932320181 conn(0x7f46bc0fa2b0 msgr2=0x7f46bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:10.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 shutdown_connections 2024-04-24T08:13:10.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:10.500+0000 7f46c1800700 1 -- 172.21.15.149:0/3932320181 wait complete. 2024-04-24T08:13:10.502 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:11 smithi090 ceph-mon[25016]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:11 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/3932320181' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:12.144 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:12.144 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:12.521 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:12.620 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:13.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:13 smithi090 ceph-mon[25016]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:14.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 -- 172.21.15.149:0/2956301618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 msgr2=0x7fbd1c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:14.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 --2- 172.21.15.149:0/2956301618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c073180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009910 tx=0x7fbd10009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 -- 172.21.15.149:0/2956301618 shutdown_connections 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 --2- 172.21.15.149:0/2956301618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c073180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 -- 172.21.15.149:0/2956301618 >> 172.21.15.149:0/2956301618 conn(0x7fbd1c0fb3c0 msgr2=0x7fbd1c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 -- 172.21.15.149:0/2956301618 shutdown_connections 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.774+0000 7fbd24000700 1 -- 172.21.15.149:0/2956301618 wait complete. 2024-04-24T08:13:14.776 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.775+0000 7fbd24000700 1 Processor -- start 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.775+0000 7fbd24000700 1 -- start start 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.775+0000 7fbd24000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.776+0000 7fbd24000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c1976f0 con 0x7fbd1c074d20 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.776+0000 7fbd21600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.776+0000 7fbd21600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:53396/0 (socket says 172.21.15.149:53396) 2024-04-24T08:13:14.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.776+0000 7fbd21600700 1 -- 172.21.15.149:0/2283585586 learned_addr learned my addr 172.21.15.149:0/2283585586 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:14.778 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.777+0000 7fbd21600700 1 -- 172.21.15.149:0/2283585586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100095f0 con 0x7fbd1c074d20 2024-04-24T08:13:14.778 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.777+0000 7fbd21600700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005b40 tx=0x7fbd10004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:14.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.778+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd1001d070 con 0x7fbd1c074d20 2024-04-24T08:13:14.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.778+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c1978f0 con 0x7fbd1c074d20 2024-04-24T08:13:14.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.778+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd1000be10 con 0x7fbd1c074d20 2024-04-24T08:13:14.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.778+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd1000f460 con 0x7fbd1c074d20 2024-04-24T08:13:14.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.778+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c197d90 con 0x7fbd1c074d20 2024-04-24T08:13:14.780 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.779+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c190d00 con 0x7fbd1c074d20 2024-04-24T08:13:14.780 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.780+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fbd100227e0 con 0x7fbd1c074d20 2024-04-24T08:13:14.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.780+0000 7fbd1a000700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 0x7fbd0403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:14.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.780+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd1004cd80 con 0x7fbd1c074d20 2024-04-24T08:13:14.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.781+0000 7fbd20c00700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 0x7fbd0403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:14.783 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.782+0000 7fbd20c00700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 0x7fbd0403a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:14.799 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:14.798+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbd1002bba0 con 0x7fbd1c074d20 2024-04-24T08:13:15.091 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.090+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd1c02cc30 con 0x7fbd1c074d20 2024-04-24T08:13:15.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.091+0000 7fbd1a000700 1 -- 172.21.15.149:0/2283585586 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbd10027020 con 0x7fbd1c074d20 2024-04-24T08:13:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:15.092 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 msgr2=0x7fbd0403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 0x7fbd0403a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 msgr2=0x7fbd1c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005b40 tx=0x7fbd10004740 comp rx=0 tx=0).stop 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 shutdown_connections 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fbd04038490 0x7fbd0403a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 --2- 172.21.15.149:0/2283585586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd1c074d20 0x7fbd1c1971b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:15.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 >> 172.21.15.149:0/2283585586 conn(0x7fbd1c0fb3c0 msgr2=0x7fbd1c0fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:15.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 shutdown_connections 2024-04-24T08:13:15.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:15.094+0000 7fbd24000700 1 -- 172.21.15.149:0/2283585586 wait complete. 2024-04-24T08:13:15.097 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:15.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:15 smithi090 ceph-mon[25016]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:15.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:15 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/2283585586' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:16.849 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:16.849 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:17.230 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:17.332 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:17 smithi090 ceph-mon[25016]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.419+0000 7fd9f3400700 1 -- 172.21.15.149:0/2945030798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 msgr2=0x7fd9ec102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.419+0000 7fd9f3400700 1 --2- 172.21.15.149:0/2945030798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec102c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009910 tx=0x7fd9e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.420+0000 7fd9f3400700 1 -- 172.21.15.149:0/2945030798 shutdown_connections 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.420+0000 7fd9f3400700 1 --2- 172.21.15.149:0/2945030798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec102c90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.420+0000 7fd9f3400700 1 -- 172.21.15.149:0/2945030798 >> 172.21.15.149:0/2945030798 conn(0x7fd9ec0fa2b0 msgr2=0x7fd9ec0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.420+0000 7fd9f3400700 1 -- 172.21.15.149:0/2945030798 shutdown_connections 2024-04-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.420+0000 7fd9f3400700 1 -- 172.21.15.149:0/2945030798 wait complete. 2024-04-24T08:13:19.423 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.421+0000 7fd9f3400700 1 Processor -- start 2024-04-24T08:13:19.423 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.422+0000 7fd9f3400700 1 -- start start 2024-04-24T08:13:19.423 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.422+0000 7fd9f3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:19.423 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.422+0000 7fd9f3400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1976c0 con 0x7fd9ec1008b0 2024-04-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.423+0000 7fd9f0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.423+0000 7fd9f0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:57172/0 (socket says 172.21.15.149:57172) 2024-04-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.423+0000 7fd9f0a00700 1 -- 172.21.15.149:0/3100050173 learned_addr learned my addr 172.21.15.149:0/3100050173 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.423+0000 7fd9f0a00700 1 -- 172.21.15.149:0/3100050173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00095f0 con 0x7fd9ec1008b0 2024-04-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9f0a00700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00092e0 tx=0x7fd9e0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e001d070 con 0x7fd9ec1008b0 2024-04-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec1978c0 con 0x7fd9ec1008b0 2024-04-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec197d60 con 0x7fd9ec1008b0 2024-04-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd9e000be10 con 0x7fd9ec1008b0 2024-04-24T08:13:19.426 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.424+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9e000f460 con 0x7fd9ec1008b0 2024-04-24T08:13:19.426 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.425+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec190f00 con 0x7fd9ec1008b0 2024-04-24T08:13:19.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.426+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd9e0022470 con 0x7fd9ec1008b0 2024-04-24T08:13:19.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.427+0000 7fd9e9600700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 0x7fd9d403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:19.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.427+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd9e004c3f0 con 0x7fd9ec1008b0 2024-04-24T08:13:19.428 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.427+0000 7fd9ebe00700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 0x7fd9d403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:19.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.428+0000 7fd9ebe00700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 0x7fd9d403a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:19.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.442+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd9e000b960 con 0x7fd9ec1008b0 2024-04-24T08:13:19.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:19 smithi090 ceph-mon[25016]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:19.740 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.739+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9ec02cc30 con 0x7fd9ec1008b0 2024-04-24T08:13:19.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.742+0000 7fd9e9600700 1 -- 172.21.15.149:0/3100050173 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd9e0027030 con 0x7fd9ec1008b0 2024-04-24T08:13:19.742 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:19.743 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:19.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 msgr2=0x7fd9d403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:19.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 0x7fd9d403a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:19.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 msgr2=0x7fd9ec197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00092e0 tx=0x7fd9e0009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 shutdown_connections 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fd9d40384e0 0x7fd9d403a990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 --2- 172.21.15.149:0/3100050173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1008b0 0x7fd9ec197180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 >> 172.21.15.149:0/3100050173 conn(0x7fd9ec0fa2b0 msgr2=0x7fd9ec0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 shutdown_connections 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:19.744+0000 7fd9f3400700 1 -- 172.21.15.149:0/3100050173 wait complete. 2024-04-24T08:13:19.746 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:20.651 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:20 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/3100050173' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:21.537 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:21 smithi090 ceph-mon[25016]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:21.552 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:21.553 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:21.927 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:22.036 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:22.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:22.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:22.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:22.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:22.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:22.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:22 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:23.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:23 smithi090 ceph-mon[25016]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-24T08:13:23.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:23 smithi090 ceph-mon[25016]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:23.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:23 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:24.176 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 -- 172.21.15.149:0/839704371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01062b0 msgr2=0x7f34e01066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:24.176 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 --2- 172.21.15.149:0/839704371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01062b0 0x7f34e01066c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009910 tx=0x7f34d0009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:24.176 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 -- 172.21.15.149:0/839704371 shutdown_connections 2024-04-24T08:13:24.176 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 --2- 172.21.15.149:0/839704371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01062b0 0x7f34e01066c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:24.176 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 -- 172.21.15.149:0/839704371 >> 172.21.15.149:0/839704371 conn(0x7f34e0075410 msgr2=0x7f34e0077840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 -- 172.21.15.149:0/839704371 shutdown_connections 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.175+0000 7f34e5400700 1 -- 172.21.15.149:0/839704371 wait complete. 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.177+0000 7f34e5400700 1 Processor -- start 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.177+0000 7f34e5400700 1 -- start start 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.177+0000 7f34e5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.177+0000 7f34e5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e019bf40 con 0x7f34e019b5f0 2024-04-24T08:13:24.178 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.178+0000 7f34dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:24.178 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.178+0000 7f34dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:57196/0 (socket says 172.21.15.149:57196) 2024-04-24T08:13:24.178 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.178+0000 7f34dfe00700 1 -- 172.21.15.149:0/2238595790 learned_addr learned my addr 172.21.15.149:0/2238595790 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:24.178 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.179+0000 7f34dfe00700 1 -- 172.21.15.149:0/2238595790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00095f0 con 0x7f34e019b5f0 2024-04-24T08:13:24.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.179+0000 7f34dfe00700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005950 tx=0x7f34d0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:24.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.180+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34d001d070 con 0x7f34e019b5f0 2024-04-24T08:13:24.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.180+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f34d000be10 con 0x7f34e019b5f0 2024-04-24T08:13:24.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.180+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34d0022e20 con 0x7f34e019b5f0 2024-04-24T08:13:24.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.180+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e019c140 con 0x7f34e019b5f0 2024-04-24T08:13:24.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.180+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e019eda0 con 0x7f34e019b5f0 2024-04-24T08:13:24.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.181+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e0194f40 con 0x7f34e019b5f0 2024-04-24T08:13:24.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.181+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f34d0022700 con 0x7f34e019b5f0 2024-04-24T08:13:24.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.182+0000 7f34dcc00700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 0x7f34c403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:24.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.182+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34d004c2f0 con 0x7f34e019b5f0 2024-04-24T08:13:24.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.182+0000 7f34df400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 0x7f34c403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:24.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.184+0000 7f34df400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 0x7f34c403a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:24.197 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.197+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f34d002b980 con 0x7f34e019b5f0 2024-04-24T08:13:24.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.494+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f34e002cc30 con 0x7f34e019b5f0 2024-04-24T08:13:24.496 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.495+0000 7f34dcc00700 1 -- 172.21.15.149:0/2238595790 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f34d0027070 con 0x7f34e019b5f0 2024-04-24T08:13:24.497 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:24.497 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:24.499 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 msgr2=0x7f34c403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 0x7f34c403a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 msgr2=0x7f34e019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005950 tx=0x7f34d0009f10 comp rx=0 tx=0).stop 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 shutdown_connections 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f34c4038490 0x7f34c403a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 --2- 172.21.15.149:0/2238595790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e019b5f0 0x7f34e019ba00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.498+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 >> 172.21.15.149:0/2238595790 conn(0x7f34e0075410 msgr2=0x7f34e0077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.499+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 shutdown_connections 2024-04-24T08:13:24.500 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:24.499+0000 7f34e5400700 1 -- 172.21.15.149:0/2238595790 wait complete. 2024-04-24T08:13:24.501 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:25.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:24 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/2238595790' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:25 smithi090 ceph-mon[25016]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:26.128 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:26.129 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:26.513 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:26.613 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:27.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:27 smithi090 ceph-mon[25016]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:28.630 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 -- 172.21.15.149:0/1510006527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 msgr2=0x7fc138073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 --2- 172.21.15.149:0/1510006527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc138073160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc128009910 tx=0x7fc128009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 -- 172.21.15.149:0/1510006527 shutdown_connections 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 --2- 172.21.15.149:0/1510006527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc138073160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 -- 172.21.15.149:0/1510006527 >> 172.21.15.149:0/1510006527 conn(0x7fc1380fb3d0 msgr2=0x7fc1380fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 -- 172.21.15.149:0/1510006527 shutdown_connections 2024-04-24T08:13:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.629+0000 7fc13e400700 1 -- 172.21.15.149:0/1510006527 wait complete. 2024-04-24T08:13:28.632 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.630+0000 7fc13e400700 1 Processor -- start 2024-04-24T08:13:28.632 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.631+0000 7fc13e400700 1 -- start start 2024-04-24T08:13:28.632 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.631+0000 7fc13e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:28.632 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.631+0000 7fc13e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1381976e0 con 0x7fc138074d00 2024-04-24T08:13:28.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.632+0000 7fc137400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:28.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.632+0000 7fc137400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:33252/0 (socket says 172.21.15.149:33252) 2024-04-24T08:13:28.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.632+0000 7fc137400700 1 -- 172.21.15.149:0/3424877407 learned_addr learned my addr 172.21.15.149:0/3424877407 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:28.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.632+0000 7fc137400700 1 -- 172.21.15.149:0/3424877407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280095f0 con 0x7fc138074d00 2024-04-24T08:13:28.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc137400700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc128005b40 tx=0x7fc128004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc12801d070 con 0x7fc138074d00 2024-04-24T08:13:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc12800be10 con 0x7fc138074d00 2024-04-24T08:13:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1381978e0 con 0x7fc138074d00 2024-04-24T08:13:28.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc12800f460 con 0x7fc138074d00 2024-04-24T08:13:28.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.633+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc138197d80 con 0x7fc138074d00 2024-04-24T08:13:28.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.634+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc138190d00 con 0x7fc138074d00 2024-04-24T08:13:28.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.635+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc1280227e0 con 0x7fc138074d00 2024-04-24T08:13:28.636 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.635+0000 7fc12fe00700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 0x7fc11c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:28.636 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.635+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc12804c430 con 0x7fc138074d00 2024-04-24T08:13:28.636 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.636+0000 7fc136a00700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 0x7fc11c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:28.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.637+0000 7fc136a00700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 0x7fc11c03a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:28.650 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.649+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc12802bba0 con 0x7fc138074d00 2024-04-24T08:13:28.948 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.946+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc13802cc30 con 0x7fc138074d00 2024-04-24T08:13:28.948 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.947+0000 7fc12fe00700 1 -- 172.21.15.149:0/3424877407 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc128027070 con 0x7fc138074d00 2024-04-24T08:13:28.949 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:28.949 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:28.951 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 msgr2=0x7fc11c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:28.951 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 0x7fc11c03a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 msgr2=0x7fc1381971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc128005b40 tx=0x7fc128004740 comp rx=0 tx=0).stop 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 shutdown_connections 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fc11c038490 0x7fc11c03a940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 --2- 172.21.15.149:0/3424877407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc138074d00 0x7fc1381971a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 >> 172.21.15.149:0/3424877407 conn(0x7fc1380fb3d0 msgr2=0x7fc1380fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 shutdown_connections 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:28.950+0000 7fc13e400700 1 -- 172.21.15.149:0/3424877407 wait complete. 2024-04-24T08:13:28.952 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:29 smithi090 ceph-mon[25016]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:29 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/3424877407' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:30.686 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:30.686 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:31.025 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:31.136 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:31 smithi090 ceph-mon[25016]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:33.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.297+0000 7f8a58a00700 1 -- 172.21.15.149:0/3099881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 msgr2=0x7f8a54100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:33.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.297+0000 7f8a58a00700 1 --2- 172.21.15.149:0/3099881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54100240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009910 tx=0x7f8a44009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:33.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.297+0000 7f8a58a00700 1 -- 172.21.15.149:0/3099881795 shutdown_connections 2024-04-24T08:13:33.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.297+0000 7f8a58a00700 1 --2- 172.21.15.149:0/3099881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54100240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:33.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.297+0000 7f8a58a00700 1 -- 172.21.15.149:0/3099881795 >> 172.21.15.149:0/3099881795 conn(0x7f8a540fb3c0 msgr2=0x7f8a540fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.298+0000 7f8a58a00700 1 -- 172.21.15.149:0/3099881795 shutdown_connections 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.298+0000 7f8a58a00700 1 -- 172.21.15.149:0/3099881795 wait complete. 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.299+0000 7f8a58a00700 1 Processor -- start 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.299+0000 7f8a58a00700 1 -- start start 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.299+0000 7f8a58a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:33.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.299+0000 7f8a58a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a541976d0 con 0x7f8a540ffe30 2024-04-24T08:13:33.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.300+0000 7f8a51e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:33.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.300+0000 7f8a51e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:33262/0 (socket says 172.21.15.149:33262) 2024-04-24T08:13:33.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.300+0000 7f8a51e00700 1 -- 172.21.15.149:0/2503610308 learned_addr learned my addr 172.21.15.149:0/2503610308 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:33.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.301+0000 7f8a51e00700 1 -- 172.21.15.149:0/2503610308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440095f0 con 0x7f8a540ffe30 2024-04-24T08:13:33.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.301+0000 7f8a51e00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005b40 tx=0x7f8a44004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:33.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.302+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a4401d070 con 0x7f8a540ffe30 2024-04-24T08:13:33.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.302+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a541978d0 con 0x7f8a540ffe30 2024-04-24T08:13:33.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.302+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8a4400be10 con 0x7f8a540ffe30 2024-04-24T08:13:33.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.302+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a54197d70 con 0x7f8a540ffe30 2024-04-24T08:13:33.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.302+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a4400f460 con 0x7f8a540ffe30 2024-04-24T08:13:33.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.303+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a54190e90 con 0x7f8a540ffe30 2024-04-24T08:13:33.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.303+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8a440227c0 con 0x7f8a540ffe30 2024-04-24T08:13:33.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.304+0000 7f8a4aa00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 0x7f8a3803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:33.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.304+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a4404c160 con 0x7f8a540ffe30 2024-04-24T08:13:33.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.305+0000 7f8a51400700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 0x7f8a3803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:33.307 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.306+0000 7f8a51400700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 0x7f8a3803a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:33.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.319+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8a44022470 con 0x7f8a540ffe30 2024-04-24T08:13:33.616 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.615+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8a541008c0 con 0x7f8a540ffe30 2024-04-24T08:13:33.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.616+0000 7f8a4aa00700 1 -- 172.21.15.149:0/2503610308 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8a541008c0 con 0x7f8a540ffe30 2024-04-24T08:13:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:33.618 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 msgr2=0x7f8a3803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 0x7f8a3803a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 msgr2=0x7f8a54197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005b40 tx=0x7f8a44004740 comp rx=0 tx=0).stop 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 shutdown_connections 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8a380384e0 0x7f8a3803a990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 --2- 172.21.15.149:0/2503610308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a540ffe30 0x7f8a54197190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 >> 172.21.15.149:0/2503610308 conn(0x7f8a540fb3c0 msgr2=0x7f8a540fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:33.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 shutdown_connections 2024-04-24T08:13:33.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:33.620+0000 7f8a58a00700 1 -- 172.21.15.149:0/2503610308 wait complete. 2024-04-24T08:13:33.622 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:33.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:33 smithi090 ceph-mon[25016]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:34.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:34 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/2503610308' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:35.507 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:35.507 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:35.761 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:35 smithi090 ceph-mon[25016]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:35.884 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:35.979 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:37.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:37 smithi090 ceph-mon[25016]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:38.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 -- 172.21.15.149:0/2850838302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 msgr2=0x7fb484102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 --2- 172.21.15.149:0/2850838302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484102280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb478009910 tx=0x7fb478009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 -- 172.21.15.149:0/2850838302 shutdown_connections 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 --2- 172.21.15.149:0/2850838302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484102280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 -- 172.21.15.149:0/2850838302 >> 172.21.15.149:0/2850838302 conn(0x7fb4840fd530 msgr2=0x7fb4840ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 -- 172.21.15.149:0/2850838302 shutdown_connections 2024-04-24T08:13:38.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.165+0000 7fb48a200700 1 -- 172.21.15.149:0/2850838302 wait complete. 2024-04-24T08:13:38.168 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.166+0000 7fb48a200700 1 Processor -- start 2024-04-24T08:13:38.168 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.167+0000 7fb48a200700 1 -- start start 2024-04-24T08:13:38.168 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.167+0000 7fb48a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:38.168 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.167+0000 7fb48a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb484197540 con 0x7fb484101e70 2024-04-24T08:13:38.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.168+0000 7fb488e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:38.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.168+0000 7fb488e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:46914/0 (socket says 172.21.15.149:46914) 2024-04-24T08:13:38.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.168+0000 7fb488e00700 1 -- 172.21.15.149:0/327702181 learned_addr learned my addr 172.21.15.149:0/327702181 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:38.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.168+0000 7fb488e00700 1 -- 172.21.15.149:0/327702181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780095f0 con 0x7fb484101e70 2024-04-24T08:13:38.170 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.169+0000 7fb488e00700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4780092e0 tx=0x7fb478009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:38.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.170+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47801d070 con 0x7fb484101e70 2024-04-24T08:13:38.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.170+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb484197740 con 0x7fb484101e70 2024-04-24T08:13:38.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.170+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb47800be10 con 0x7fb484101e70 2024-04-24T08:13:38.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.170+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484197be0 con 0x7fb484101e70 2024-04-24T08:13:38.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.170+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47800f460 con 0x7fb484101e70 2024-04-24T08:13:38.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.171+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb484190ad0 con 0x7fb484101e70 2024-04-24T08:13:38.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.172+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fb478022470 con 0x7fb484101e70 2024-04-24T08:13:38.173 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.172+0000 7fb481600700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 0x7fb46c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:38.173 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.172+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb47804c5b0 con 0x7fb484101e70 2024-04-24T08:13:38.174 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.173+0000 7fb483e00700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 0x7fb46c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:38.175 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.174+0000 7fb483e00700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 0x7fb46c03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:38.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.188+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb478022790 con 0x7fb484101e70 2024-04-24T08:13:38.487 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.486+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb48402cc30 con 0x7fb484101e70 2024-04-24T08:13:38.488 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.487+0000 7fb481600700 1 -- 172.21.15.149:0/327702181 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb478027020 con 0x7fb484101e70 2024-04-24T08:13:38.488 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:38.488 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:38.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 msgr2=0x7fb46c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:38.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 0x7fb46c03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 msgr2=0x7fb484197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4780092e0 tx=0x7fb478009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 shutdown_connections 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7fb46c038490 0x7fb46c03a940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 --2- 172.21.15.149:0/327702181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb484101e70 0x7fb484197000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 >> 172.21.15.149:0/327702181 conn(0x7fb4840fd530 msgr2=0x7fb4840ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.490+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 shutdown_connections 2024-04-24T08:13:38.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:38.491+0000 7fb48a200700 1 -- 172.21.15.149:0/327702181 wait complete. 2024-04-24T08:13:38.493 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:39.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:39 smithi090 ceph-mon[25016]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:39.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:39 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/327702181' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:40.539 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:40.540 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:40.907 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:41.003 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:41.648 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:41 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:41.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:41 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:41.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:41 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:41.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:41 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-24T08:13:41.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:41 smithi090 ceph-mon[25016]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:13:42.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:42 smithi090 ceph-mon[25016]: from='mgr.14164 172.21.15.90:0/709923027' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-24T08:13:42.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:42 smithi090 ceph-mon[25016]: mgrmap e14: smithi090.jsrfxo(active, since 83s) 2024-04-24T08:13:43.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 -- 172.21.15.149:0/490144113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 msgr2=0x7f900c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:43.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 --2- 172.21.15.149:0/490144113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c102c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009910 tx=0x7f8ffc009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 -- 172.21.15.149:0/490144113 shutdown_connections 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 --2- 172.21.15.149:0/490144113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c102c90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 -- 172.21.15.149:0/490144113 >> 172.21.15.149:0/490144113 conn(0x7f900c0fa2b0 msgr2=0x7f900c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 -- 172.21.15.149:0/490144113 shutdown_connections 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.052+0000 7f9012000700 1 -- 172.21.15.149:0/490144113 wait complete. 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.053+0000 7f9012000700 1 Processor -- start 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.053+0000 7f9012000700 1 -- start start 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.053+0000 7f9012000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:43.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.053+0000 7f9012000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c18f200 con 0x7f900c1008b0 2024-04-24T08:13:43.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.054+0000 7f900b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:43.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.054+0000 7f900b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:46934/0 (socket says 172.21.15.149:46934) 2024-04-24T08:13:43.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.054+0000 7f900b400700 1 -- 172.21.15.149:0/1963128907 learned_addr learned my addr 172.21.15.149:0/1963128907 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.054+0000 7f900b400700 1 -- 172.21.15.149:0/1963128907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0095f0 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f900b400700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0092e0 tx=0x7f8ffc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ffc01d070 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc00be10 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c18f400 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ffc00f460 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.055+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c18b970 con 0x7f900c1008b0 2024-04-24T08:13:43.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.056+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c062380 con 0x7f900c1008b0 2024-04-24T08:13:43.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.057+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f8ffc00b960 con 0x7f900c1008b0 2024-04-24T08:13:43.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.057+0000 7f9008c00700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 0x7f8ff003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:43.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.057+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ffc04c530 con 0x7f900c1008b0 2024-04-24T08:13:43.059 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.057+0000 7f9003e00700 1 -- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 msgr2=0x7f8ff003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3702780720 2024-04-24T08:13:43.059 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.057+0000 7f9003e00700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 0x7f8ff003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:13:43.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.065+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8ffc00f5c0 con 0x7f900c1008b0 2024-04-24T08:13:43.259 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.258+0000 7f9003e00700 1 -- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 msgr2=0x7f8ff003a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3702780720 2024-04-24T08:13:43.259 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.258+0000 7f9003e00700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 0x7f8ff003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:13:43.362 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.360+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f900c190710 con 0x7f900c1008b0 2024-04-24T08:13:43.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.362+0000 7f9008c00700 1 -- 172.21.15.149:0/1963128907 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8ffc027070 con 0x7f900c1008b0 2024-04-24T08:13:43.363 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:43.363 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:43.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 msgr2=0x7f8ff003a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:13:43.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 0x7f8ff003a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:43.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 msgr2=0x7f900c18ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:43.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0092e0 tx=0x7f8ffc009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 shutdown_connections 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f8ff0038530 0x7f8ff003a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 --2- 172.21.15.149:0/1963128907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f900c1008b0 0x7f900c18ecc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 >> 172.21.15.149:0/1963128907 conn(0x7f900c0fa2b0 msgr2=0x7f900c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 shutdown_connections 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:43.365+0000 7f9012000700 1 -- 172.21.15.149:0/1963128907 wait complete. 2024-04-24T08:13:43.367 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:43 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1963128907' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:45.051 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:45.051 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:45.424 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:45.551 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:47.683 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 -- 172.21.15.149:0/2367372931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 msgr2=0x7f0290106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:47.683 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 --2- 172.21.15.149:0/2367372931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f0290106850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0280009910 tx=0x7f0280009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 -- 172.21.15.149:0/2367372931 shutdown_connections 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 --2- 172.21.15.149:0/2367372931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f0290106850 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 -- 172.21.15.149:0/2367372931 >> 172.21.15.149:0/2367372931 conn(0x7f02900756e0 msgr2=0x7f0290077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 -- 172.21.15.149:0/2367372931 shutdown_connections 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.682+0000 7f0296800700 1 -- 172.21.15.149:0/2367372931 wait complete. 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.683+0000 7f0296800700 1 Processor -- start 2024-04-24T08:13:47.684 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.684+0000 7f0296800700 1 -- start start 2024-04-24T08:13:47.685 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.684+0000 7f0296800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:47.685 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.684+0000 7f0296800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019bae0 con 0x7f0290106440 2024-04-24T08:13:47.685 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.684+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:47.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.685+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:51716/0 (socket says 172.21.15.149:51716) 2024-04-24T08:13:47.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.685+0000 7f028f400700 1 -- 172.21.15.149:0/680394808 learned_addr learned my addr 172.21.15.149:0/680394808 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:47.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.685+0000 7f028f400700 1 -- 172.21.15.149:0/680394808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800095f0 con 0x7f0290106440 2024-04-24T08:13:47.687 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.686+0000 7f028f400700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0280005950 tx=0x7f0280009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:47.687 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.686+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f028001d070 con 0x7f0290106440 2024-04-24T08:13:47.687 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.686+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f028000be10 con 0x7f0290106440 2024-04-24T08:13:47.687 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.686+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029019bce0 con 0x7f0290106440 2024-04-24T08:13:47.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.686+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0280022ea0 con 0x7f0290106440 2024-04-24T08:13:47.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.687+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029019c180 con 0x7f0290106440 2024-04-24T08:13:47.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.688+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0290195110 con 0x7f0290106440 2024-04-24T08:13:47.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.688+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f028003d020 con 0x7f0290106440 2024-04-24T08:13:47.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.689+0000 7f0287e00700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 0x7f027403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:47.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.689+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f028004c750 con 0x7f0290106440 2024-04-24T08:13:47.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.689+0000 7f028ea00700 1 -- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 msgr2=0x7f027403a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3702780720 2024-04-24T08:13:47.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.689+0000 7f028ea00700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 0x7f027403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:13:47.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.703+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f028002b3f0 con 0x7f0290106440 2024-04-24T08:13:47.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.890+0000 7f028ea00700 1 -- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 msgr2=0x7f027403a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3702780720 2024-04-24T08:13:47.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:47.890+0000 7f028ea00700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 0x7f027403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:13:48.005 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.003+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f029002cc30 con 0x7f0290106440 2024-04-24T08:13:48.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.005+0000 7f0287e00700 1 -- 172.21.15.149:0/680394808 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0280027070 con 0x7f0290106440 2024-04-24T08:13:48.006 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:48.006 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:48.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 msgr2=0x7f027403a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:13:48.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 0x7f027403a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 msgr2=0x7f029019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0280005950 tx=0x7f0280009e60 comp rx=0 tx=0).stop 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 shutdown_connections 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:6800/3702780720,v1:172.21.15.90:6801/3702780720] conn(0x7f02740384e0 0x7f027403a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 --2- 172.21.15.149:0/680394808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0290106440 0x7f029019b5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 >> 172.21.15.149:0/680394808 conn(0x7f02900756e0 msgr2=0x7f02900773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 shutdown_connections 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:48.008+0000 7f0296800700 1 -- 172.21.15.149:0/680394808 wait complete. 2024-04-24T08:13:48.010 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:48.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:48 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/680394808' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:49.739 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:49.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:50.121 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:50.164 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: Active manager daemon smithi090.jsrfxo restarted 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: mgrmap e15: smithi090.jsrfxo(active, starting, since 0.00517737s) 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:13:50.165 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:13:50.235 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:51.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: [24/Apr/2024:08:13:50] ENGINE Bus STARTING 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: [24/Apr/2024:08:13:50] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:13:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:50 smithi090 ceph-mon[25016]: mgrmap e16: smithi090.jsrfxo(active, since 1.01426s) 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.330+0000 7f20a3e00700 1 -- 172.21.15.149:0/846954213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 msgr2=0x7f209c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.330+0000 7f20a3e00700 1 --2- 172.21.15.149:0/846954213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c102430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2090009910 tx=0x7f2090009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.331+0000 7f20a3e00700 1 -- 172.21.15.149:0/846954213 shutdown_connections 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.331+0000 7f20a3e00700 1 --2- 172.21.15.149:0/846954213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c102430 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.331+0000 7f20a3e00700 1 -- 172.21.15.149:0/846954213 >> 172.21.15.149:0/846954213 conn(0x7f209c0fd700 msgr2=0x7f209c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.331+0000 7f20a3e00700 1 -- 172.21.15.149:0/846954213 shutdown_connections 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.331+0000 7f20a3e00700 1 -- 172.21.15.149:0/846954213 wait complete. 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.332+0000 7f20a3e00700 1 Processor -- start 2024-04-24T08:13:52.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.332+0000 7f20a3e00700 1 -- start start 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.332+0000 7f20a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.332+0000 7f20a3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1976f0 con 0x7f209c102020 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.333+0000 7f20a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.333+0000 7f20a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:51730/0 (socket says 172.21.15.149:51730) 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.333+0000 7f20a1400700 1 -- 172.21.15.149:0/3622047407 learned_addr learned my addr 172.21.15.149:0/3622047407 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:52.334 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.333+0000 7f20a1400700 1 -- 172.21.15.149:0/3622047407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900095f0 con 0x7f209c102020 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f20a1400700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2090005b40 tx=0x7f2090004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f209001d070 con 0x7f209c102020 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1978f0 con 0x7f209c102020 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c197d90 con 0x7f209c102020 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f209000be10 con 0x7f209c102020 2024-04-24T08:13:52.336 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.334+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f209000f460 con 0x7f209c102020 2024-04-24T08:13:52.337 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.336+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c190d00 con 0x7f209c102020 2024-04-24T08:13:52.338 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.336+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f20900227c0 con 0x7f209c102020 2024-04-24T08:13:52.338 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.336+0000 7f209a000700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 0x7f208403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:52.338 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.336+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f209004c5a0 con 0x7f209c102020 2024-04-24T08:13:52.338 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.337+0000 7f20a0a00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 0x7f208403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:52.339 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.337+0000 7f20a0a00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 0x7f208403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2088006fd0 tx=0x7f2088006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:52.352 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.351+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f209000b960 con 0x7f209c102020 2024-04-24T08:13:52.540 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:52 smithi090 ceph-mon[25016]: [24/Apr/2024:08:13:50] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:13:52.540 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:52 smithi090 ceph-mon[25016]: [24/Apr/2024:08:13:50] ENGINE Bus STARTED 2024-04-24T08:13:52.540 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:52 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:52.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.652+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f209c02cc30 con 0x7f209c102020 2024-04-24T08:13:52.654 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.653+0000 7f209a000700 1 -- 172.21.15.149:0/3622047407 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2090027020 con 0x7f209c102020 2024-04-24T08:13:52.655 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:52.655 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:52.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 msgr2=0x7f208403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:52.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 0x7f208403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2088006fd0 tx=0x7f2088006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:52.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 msgr2=0x7f209c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:52.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2090005b40 tx=0x7f2090004740 comp rx=0 tx=0).stop 2024-04-24T08:13:52.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 shutdown_connections 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f2084038580 0x7f208403aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 --2- 172.21.15.149:0/3622047407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f209c102020 0x7f209c1971b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 >> 172.21.15.149:0/3622047407 conn(0x7f209c0fd700 msgr2=0x7f209c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 shutdown_connections 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:52.657+0000 7f20a3e00700 1 -- 172.21.15.149:0/3622047407 wait complete. 2024-04-24T08:13:52.659 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:53 smithi090 ceph-mon[25016]: mgrmap e17: smithi090.jsrfxo(active, since 2s) 2024-04-24T08:13:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:53 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:53 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/3622047407' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:13:54.373 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:13:54.373 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:13:54.740 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:54.844 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:13:55.288 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:55 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:57.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:57.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.854+0000 7fb106a00700 1 -- 172.21.15.149:0/764967033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0f80055b0 con 0x7fb1080ffe20 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 -- 172.21.15.149:0/764967033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 msgr2=0x7fb108100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 --2- 172.21.15.149:0/764967033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108100230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009910 tx=0x7fb0f8009c20 comp rx=0 tx=0).stop 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 -- 172.21.15.149:0/764967033 shutdown_connections 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 --2- 172.21.15.149:0/764967033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108100230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 -- 172.21.15.149:0/764967033 >> 172.21.15.149:0/764967033 conn(0x7fb1080fb3d0 msgr2=0x7fb1080fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 -- 172.21.15.149:0/764967033 shutdown_connections 2024-04-24T08:13:57.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.855+0000 7fb10ea00700 1 -- 172.21.15.149:0/764967033 wait complete. 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.856+0000 7fb10ea00700 1 Processor -- start 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.856+0000 7fb10ea00700 1 -- start start 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.857+0000 7fb10ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.857+0000 7fb10ea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1081976a0 con 0x7fb1080ffe20 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.857+0000 7fb107e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.857+0000 7fb107e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:56064/0 (socket says 172.21.15.149:56064) 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.857+0000 7fb107e00700 1 -- 172.21.15.149:0/1466598826 learned_addr learned my addr 172.21.15.149:0/1466598826 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.858+0000 7fb107e00700 1 -- 172.21.15.149:0/1466598826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80095f0 con 0x7fb1080ffe20 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.858+0000 7fb107e00700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8005b40 tx=0x7fb0f8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.859+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0f801d070 con 0x7fb1080ffe20 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.859+0000 7fb10ea00700 1 -- 172.21.15.149:0/1466598826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1081978a0 con 0x7fb1080ffe20 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.859+0000 7fb10ea00700 1 -- 172.21.15.149:0/1466598826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108197d40 con 0x7fb1080ffe20 2024-04-24T08:13:57.956 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.859+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0f800be10 con 0x7fb1080ffe20 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.859+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0f800f460 con 0x7fb1080ffe20 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.860+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e00052f0 con 0x7fb1080ffe20 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.861+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fb0f80227c0 con 0x7fb1080ffe20 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.861+0000 7fb104c00700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 0x7fb0ec03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.861+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb0f804c370 con 0x7fb1080ffe20 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.862+0000 7fb107400700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 0x7fb0ec03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.863+0000 7fb107400700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 0x7fb0ec03aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006fd0 tx=0x7fb0f4006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:13:57.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:57.876+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb0f8022470 con 0x7fb1080ffe20 2024-04-24T08:13:58.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.180+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb0e0005160 con 0x7fb1080ffe20 2024-04-24T08:13:58.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.182+0000 7fb104c00700 1 -- 172.21.15.149:0/1466598826 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb0f8027070 con 0x7fb1080ffe20 2024-04-24T08:13:58.183 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:13:58.183 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:13:58.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 msgr2=0x7fb0ec03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:58.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 0x7fb0ec03aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006fd0 tx=0x7fb0f4006e40 comp rx=0 tx=0).stop 2024-04-24T08:13:58.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 msgr2=0x7fb108197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:13:58.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8005b40 tx=0x7fb0f8004740 comp rx=0 tx=0).stop 2024-04-24T08:13:58.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 shutdown_connections 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb0ec0385d0 0x7fb0ec03aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 --2- 172.21.15.149:0/1466598826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1080ffe20 0x7fb108197160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 >> 172.21.15.149:0/1466598826 conn(0x7fb1080fb3d0 msgr2=0x7fb1080fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 shutdown_connections 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:13:58.185+0000 7fb0fe000700 1 -- 172.21.15.149:0/1466598826 wait complete. 2024-04-24T08:13:58.187 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:13:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:13:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:13:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:13:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1466598826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:00.256 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:00.256 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:00.608 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:14:00.714 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:14:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:14:01.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:01.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:14:01.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:14:01.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:00 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:14:02.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:02 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:02.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:02 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:02.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:02 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:02.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:02 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:02.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 -- 172.21.15.149:0/254496375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 msgr2=0x7f5fd8100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:02.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/254496375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8100290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009910 tx=0x7f5fc8009c20 comp rx=0 tx=0).stop 2024-04-24T08:14:02.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 -- 172.21.15.149:0/254496375 shutdown_connections 2024-04-24T08:14:02.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/254496375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8100290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:02.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 -- 172.21.15.149:0/254496375 >> 172.21.15.149:0/254496375 conn(0x7f5fd80fb780 msgr2=0x7f5fd80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:02.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 -- 172.21.15.149:0/254496375 shutdown_connections 2024-04-24T08:14:02.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.976+0000 7f5fdcc00700 1 -- 172.21.15.149:0/254496375 wait complete. 2024-04-24T08:14:02.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.977+0000 7f5fdcc00700 1 Processor -- start 2024-04-24T08:14:02.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.977+0000 7f5fdcc00700 1 -- start start 2024-04-24T08:14:02.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.978+0000 7f5fdcc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:02.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.978+0000 7f5fdcc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8197940 con 0x7f5fd80ffec0 2024-04-24T08:14:02.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.978+0000 7f5fd7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:02.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.979+0000 7f5fd7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:56076/0 (socket says 172.21.15.149:56076) 2024-04-24T08:14:02.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.979+0000 7f5fd7400700 1 -- 172.21.15.149:0/1350541956 learned_addr learned my addr 172.21.15.149:0/1350541956 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:02.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.979+0000 7f5fd7400700 1 -- 172.21.15.149:0/1350541956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80095f0 con 0x7f5fd80ffec0 2024-04-24T08:14:02.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.980+0000 7f5fd7400700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005b40 tx=0x7f5fc8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:02.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.980+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fc801d070 con 0x7f5fd80ffec0 2024-04-24T08:14:02.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.980+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd8197b40 con 0x7f5fd80ffec0 2024-04-24T08:14:02.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.980+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8197f00 con 0x7f5fd80ffec0 2024-04-24T08:14:02.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.981+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fc80044d0 con 0x7f5fd80ffec0 2024-04-24T08:14:02.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.981+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fc8003880 con 0x7f5fd80ffec0 2024-04-24T08:14:02.982 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.981+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb8005320 con 0x7f5fd80ffec0 2024-04-24T08:14:02.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.982+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f5fc8003b30 con 0x7f5fd80ffec0 2024-04-24T08:14:02.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.982+0000 7f5fcfe00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 0x7f5fbc03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:02.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.982+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5fc804c3c0 con 0x7f5fd80ffec0 2024-04-24T08:14:02.984 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.983+0000 7f5fd6a00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 0x7f5fbc03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:02.985 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.984+0000 7f5fd6a00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 0x7f5fbc03aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:02.999 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:02.998+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5fc802a540 con 0x7f5fd80ffec0 2024-04-24T08:14:03.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.326+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5fb8005190 con 0x7f5fd80ffec0 2024-04-24T08:14:03.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.327+0000 7f5fcfe00700 1 -- 172.21.15.149:0/1350541956 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5fc802a360 con 0x7f5fd80ffec0 2024-04-24T08:14:03.328 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:03.328 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 msgr2=0x7f5fbc03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 0x7f5fbc03aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4006e40 comp rx=0 tx=0).stop 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 msgr2=0x7f5fd8197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005b40 tx=0x7f5fc8004c80 comp rx=0 tx=0).stop 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 shutdown_connections 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f5fbc0385d0 0x7f5fbc03aa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 --2- 172.21.15.149:0/1350541956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fd80ffec0 0x7f5fd8197400 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 >> 172.21.15.149:0/1350541956 conn(0x7f5fd80fb780 msgr2=0x7f5fd8102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:03.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 shutdown_connections 2024-04-24T08:14:03.332 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:03.330+0000 7f5fdcc00700 1 -- 172.21.15.149:0/1350541956 wait complete. 2024-04-24T08:14:03.332 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:03.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:03 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1350541956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:04 smithi090 ceph-mon[25016]: Deploying daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:14:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:06.245 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:06.245 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:06.734 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:08.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:08.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:08.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:08 smithi090 ceph-mon[25016]: Deploying daemon crash.smithi149 on smithi149 2024-04-24T08:14:09.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 -- 172.21.15.149:0/409639157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 msgr2=0x7fec4c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 --2- 172.21.15.149:0/409639157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c101290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fec40009910 tx=0x7fec40009c20 comp rx=0 tx=0).stop 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 -- 172.21.15.149:0/409639157 shutdown_connections 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 --2- 172.21.15.149:0/409639157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c101290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 -- 172.21.15.149:0/409639157 >> 172.21.15.149:0/409639157 conn(0x7fec4c0fa850 msgr2=0x7fec4c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 -- 172.21.15.149:0/409639157 shutdown_connections 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.893+0000 7fec54400700 1 -- 172.21.15.149:0/409639157 wait complete. 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.895+0000 7fec54400700 1 Processor -- start 2024-04-24T08:14:09.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.895+0000 7fec54400700 1 -- start start 2024-04-24T08:14:09.896 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.895+0000 7fec54400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:09.896 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.895+0000 7fec54400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c19be90 con 0x7fec4c100ec0 2024-04-24T08:14:09.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.896+0000 7fec51a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:09.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.896+0000 7fec51a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:49950/0 (socket says 172.21.15.149:49950) 2024-04-24T08:14:09.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.896+0000 7fec51a00700 1 -- 172.21.15.149:0/1497818647 learned_addr learned my addr 172.21.15.149:0/1497818647 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:09.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.897+0000 7fec51a00700 1 -- 172.21.15.149:0/1497818647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400095f0 con 0x7fec4c100ec0 2024-04-24T08:14:09.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.897+0000 7fec51a00700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec400092e0 tx=0x7fec40004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:09.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.898+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec4001d070 con 0x7fec4c100ec0 2024-04-24T08:14:09.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.898+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c19c090 con 0x7fec4c100ec0 2024-04-24T08:14:09.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.898+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec4000be10 con 0x7fec4c100ec0 2024-04-24T08:14:09.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.898+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec40022e10 con 0x7fec4c100ec0 2024-04-24T08:14:09.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.898+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c19c4b0 con 0x7fec4c100ec0 2024-04-24T08:14:09.900 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.899+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c062380 con 0x7fec4c100ec0 2024-04-24T08:14:09.900 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.900+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fec40022470 con 0x7fec4c100ec0 2024-04-24T08:14:09.901 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.900+0000 7fec4a000700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 0x7fec3403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:09.901 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.900+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fec4004c4f0 con 0x7fec4c100ec0 2024-04-24T08:14:09.902 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.901+0000 7fec51000700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 0x7fec3403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:09.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.902+0000 7fec51000700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 0x7fec3403a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:09.916 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:09.915+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fec4002a540 con 0x7fec4c100ec0 2024-04-24T08:14:10.215 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.214+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fec4c1a0430 con 0x7fec4c100ec0 2024-04-24T08:14:10.216 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.215+0000 7fec4a000700 1 -- 172.21.15.149:0/1497818647 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fec40027030 con 0x7fec4c100ec0 2024-04-24T08:14:10.216 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:10.217 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:14:10.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 msgr2=0x7fec3403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 0x7fec3403a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38006e40 comp rx=0 tx=0).stop 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 msgr2=0x7fec4c19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec400092e0 tx=0x7fec40004c80 comp rx=0 tx=0).stop 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 shutdown_connections 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fec34038530 0x7fec3403a9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 --2- 172.21.15.149:0/1497818647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c100ec0 0x7fec4c19b950 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.218+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 >> 172.21.15.149:0/1497818647 conn(0x7fec4c0fa850 msgr2=0x7fec4c106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.219+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 shutdown_connections 2024-04-24T08:14:10.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:10.219+0000 7fec54400700 1 -- 172.21.15.149:0/1497818647 wait complete. 2024-04-24T08:14:10.221 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:14:11.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:10 smithi090 ceph-mon[25016]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:11.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:10 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:11.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:10 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1497818647' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:12.257 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:12.258 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:12.622 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:13 smithi090 ceph-mon[25016]: Deploying daemon node-exporter.smithi149 on smithi149 2024-04-24T08:14:14.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.741+0000 7f18bbe00700 1 -- 172.21.15.149:0/3529059416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 msgr2=0x7f18b40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.741+0000 7f18bbe00700 1 --2- 172.21.15.149:0/3529059416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b40734c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b0a0 tx=0x7f18a800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.742+0000 7f18bbe00700 1 -- 172.21.15.149:0/3529059416 shutdown_connections 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.742+0000 7f18bbe00700 1 --2- 172.21.15.149:0/3529059416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b40734c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.742+0000 7f18bbe00700 1 -- 172.21.15.149:0/3529059416 >> 172.21.15.149:0/3529059416 conn(0x7f18b40fb970 msgr2=0x7f18b40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.742+0000 7f18bbe00700 1 -- 172.21.15.149:0/3529059416 shutdown_connections 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.742+0000 7f18bbe00700 1 -- 172.21.15.149:0/3529059416 wait complete. 2024-04-24T08:14:14.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.743+0000 7f18bbe00700 1 Processor -- start 2024-04-24T08:14:14.744 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.743+0000 7f18bbe00700 1 -- start start 2024-04-24T08:14:14.744 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.743+0000 7f18bbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:14.744 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.743+0000 7f18bbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41a1c50 con 0x7f18b40730f0 2024-04-24T08:14:14.744 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.744+0000 7f18b9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:14.744 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.744+0000 7f18b9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:49996/0 (socket says 172.21.15.149:49996) 2024-04-24T08:14:14.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.744+0000 7f18b9400700 1 -- 172.21.15.149:0/1584877229 learned_addr learned my addr 172.21.15.149:0/1584877229 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:14.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.745+0000 7f18b9400700 1 -- 172.21.15.149:0/1584877229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009d00 con 0x7f18b40730f0 2024-04-24T08:14:14.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.745+0000 7f18b9400700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18a80062a0 tx=0x7f18a80097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:14.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.746+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a800e040 con 0x7f18b40730f0 2024-04-24T08:14:14.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.746+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b41a1df0 con 0x7f18b40730f0 2024-04-24T08:14:14.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.746+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f18a8007c70 con 0x7f18b40730f0 2024-04-24T08:14:14.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.746+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a801d440 con 0x7f18b40730f0 2024-04-24T08:14:14.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.746+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b41a2270 con 0x7f18b40730f0 2024-04-24T08:14:14.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.747+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b4062380 con 0x7f18b40730f0 2024-04-24T08:14:14.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.748+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f18a8004680 con 0x7f18b40730f0 2024-04-24T08:14:14.748 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.748+0000 7f18b2000700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 0x7f189c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:14.748 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.748+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f18a804c080 con 0x7f18b40730f0 2024-04-24T08:14:14.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.749+0000 7f18b8a00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 0x7f189c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:14.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.750+0000 7f18b8a00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 0x7f189c03aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006fd0 tx=0x7f18a0006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:14.763 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:14.763+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f18a807e720 con 0x7f18b40730f0 2024-04-24T08:14:15.069 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.068+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f18b419cca0 con 0x7f18b40730f0 2024-04-24T08:14:15.070 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.070+0000 7f18b2000700 1 -- 172.21.15.149:0/1584877229 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f18a8011570 con 0x7f18b40730f0 2024-04-24T08:14:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:15.070 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.072+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 msgr2=0x7f189c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.072+0000 7f18bbe00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 0x7f189c03aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006fd0 tx=0x7f18a0006e40 comp rx=0 tx=0).stop 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.072+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 msgr2=0x7f18b41a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.072+0000 7f18bbe00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18a80062a0 tx=0x7f18a80097f0 comp rx=0 tx=0).stop 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 shutdown_connections 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f189c038580 0x7f189c03aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 --2- 172.21.15.149:0/1584877229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b40730f0 0x7f18b41a1710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 >> 172.21.15.149:0/1584877229 conn(0x7f18b40fb970 msgr2=0x7f18b40fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 shutdown_connections 2024-04-24T08:14:15.073 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:15.073+0000 7f18bbe00700 1 -- 172.21.15.149:0/1584877229 wait complete. 2024-04-24T08:14:15.074 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:14:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:15 smithi090 ceph-mon[25016]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:15 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:16 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1584877229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:17.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:17 smithi090 ceph-mon[25016]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:17.691 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:17.692 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:18.155 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:19 smithi090 ceph-mon[25016]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 -- 172.21.15.149:0/4019496225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 msgr2=0x7ff02c102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 --2- 172.21.15.149:0/4019496225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c102650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009910 tx=0x7ff01c009c20 comp rx=0 tx=0).stop 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 -- 172.21.15.149:0/4019496225 shutdown_connections 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 --2- 172.21.15.149:0/4019496225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c102650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 -- 172.21.15.149:0/4019496225 >> 172.21.15.149:0/4019496225 conn(0x7ff02c0fdb60 msgr2=0x7ff02c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:20.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 -- 172.21.15.149:0/4019496225 shutdown_connections 2024-04-24T08:14:20.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.511+0000 7ff032400700 1 -- 172.21.15.149:0/4019496225 wait complete. 2024-04-24T08:14:20.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.512+0000 7ff032400700 1 Processor -- start 2024-04-24T08:14:20.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.512+0000 7ff032400700 1 -- start start 2024-04-24T08:14:20.514 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.513+0000 7ff032400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:20.514 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.513+0000 7ff032400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c197a10 con 0x7ff02c102280 2024-04-24T08:14:20.514 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.514+0000 7ff02b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:20.514 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.514+0000 7ff02b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:35784/0 (socket says 172.21.15.149:35784) 2024-04-24T08:14:20.515 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.514+0000 7ff02b400700 1 -- 172.21.15.149:0/1758284325 learned_addr learned my addr 172.21.15.149:0/1758284325 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:20.515 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.514+0000 7ff02b400700 1 -- 172.21.15.149:0/1758284325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c0095f0 con 0x7ff02c102280 2024-04-24T08:14:20.515 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff02b400700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0092e0 tx=0x7ff01c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:20.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff01c01d070 con 0x7ff02c102280 2024-04-24T08:14:20.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c197c10 con 0x7ff02c102280 2024-04-24T08:14:20.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff01c00be10 con 0x7ff02c102280 2024-04-24T08:14:20.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff01c022e10 con 0x7ff02c102280 2024-04-24T08:14:20.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.515+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c198030 con 0x7ff02c102280 2024-04-24T08:14:20.517 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.516+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c062380 con 0x7ff02c102280 2024-04-24T08:14:20.517 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.517+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7ff01c022470 con 0x7ff02c102280 2024-04-24T08:14:20.518 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.517+0000 7ff023e00700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 0x7ff01003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:20.518 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.517+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff01c04c540 con 0x7ff02c102280 2024-04-24T08:14:20.518 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.518+0000 7ff02aa00700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 0x7ff01003a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:20.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.519+0000 7ff02aa00700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 0x7ff01003a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff014006fd0 tx=0x7ff014006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:20.533 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.532+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff01c02a540 con 0x7ff02c102280 2024-04-24T08:14:20.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:20.840 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.839+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff02c10a020 con 0x7ff02c102280 2024-04-24T08:14:20.841 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.840+0000 7ff023e00700 1 -- 172.21.15.149:0/1758284325 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff01c027030 con 0x7ff02c102280 2024-04-24T08:14:20.841 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:20.841 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 msgr2=0x7ff01003a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 0x7ff01003a9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff014006fd0 tx=0x7ff014006e40 comp rx=0 tx=0).stop 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 msgr2=0x7ff02c1974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0092e0 tx=0x7ff01c004c80 comp rx=0 tx=0).stop 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 shutdown_connections 2024-04-24T08:14:20.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff010038530 0x7ff01003a9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:20.845 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 --2- 172.21.15.149:0/1758284325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff02c102280 0x7ff02c1974d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:20.845 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 >> 172.21.15.149:0/1758284325 conn(0x7ff02c0fdb60 msgr2=0x7ff02c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:20.845 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 shutdown_connections 2024-04-24T08:14:20.845 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:20.843+0000 7ff032400700 1 -- 172.21.15.149:0/1758284325 wait complete. 2024-04-24T08:14:20.845 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:14:21.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:21 smithi090 ceph-mon[25016]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:21.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:21 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1758284325' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:22.913 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:22.913 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:23.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:23 smithi090 ceph-mon[25016]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:23.883 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:25.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:14:25.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:25.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:24 smithi090 ceph-mon[25016]: Deploying daemon mon.smithi149 on smithi149 2024-04-24T08:14:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 -- 172.21.15.149:0/941787575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 msgr2=0x7f115c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 --2- 172.21.15.149:0/941787575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c075650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f114c009910 tx=0x7f114c009c20 comp rx=0 tx=0).stop 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 -- 172.21.15.149:0/941787575 shutdown_connections 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 --2- 172.21.15.149:0/941787575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c075650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 -- 172.21.15.149:0/941787575 >> 172.21.15.149:0/941787575 conn(0x7f115c0fd9c0 msgr2=0x7f115c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.115+0000 7f115b200700 1 -- 172.21.15.149:0/941787575 shutdown_connections 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.116+0000 7f115b200700 1 -- 172.21.15.149:0/941787575 wait complete. 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.117+0000 7f115b200700 1 Processor -- start 2024-04-24T08:14:27.179 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.117+0000 7f115b200700 1 -- start start 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.117+0000 7f115b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.117+0000 7f115b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a1b00 con 0x7f115c075280 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.118+0000 7f1159e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.118+0000 7f1159e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:59398/0 (socket says 172.21.15.149:59398) 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.118+0000 7f1159e00700 1 -- 172.21.15.149:0/1647014713 learned_addr learned my addr 172.21.15.149:0/1647014713 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:27.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.119+0000 7f1159e00700 1 -- 172.21.15.149:0/1647014713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c0095f0 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.119+0000 7f1159e00700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f114c0092e0 tx=0x7f114c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.125+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c01d070 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.125+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f114c00be10 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.125+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1a1d00 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.125+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c022e10 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.125+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a2120 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.126+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c062380 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.126+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f114c022470 con 0x7f115c075280 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.127+0000 7f1152a00700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 0x7f114003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:27.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.127+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f114c04cb10 con 0x7f115c075280 2024-04-24T08:14:27.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.128+0000 7f1159400700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 0x7f114003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:27.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.129+0000 7f1159400700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 0x7f114003aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:27.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.141+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f114c02b590 con 0x7f115c075280 2024-04-24T08:14:27.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.450+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f115c10d890 con 0x7f115c075280 2024-04-24T08:14:27.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.452+0000 7f1152a00700 1 -- 172.21.15.149:0/1647014713 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f114c034300 con 0x7f115c075280 2024-04-24T08:14:27.453 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:27.453 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:10:48.372320Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 msgr2=0x7f114003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 0x7f114003aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).stop 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 msgr2=0x7f115c1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f114c0092e0 tx=0x7f114c004c80 comp rx=0 tx=0).stop 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 shutdown_connections 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.454+0000 7f115b200700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1140038580 0x7f114003aa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.455+0000 7f115b200700 1 --2- 172.21.15.149:0/1647014713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075280 0x7f115c1a15c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.455+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 >> 172.21.15.149:0/1647014713 conn(0x7f115c0fd9c0 msgr2=0x7f115c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:27.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.455+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 shutdown_connections 2024-04-24T08:14:27.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:27.455+0000 7f115b200700 1 -- 172.21.15.149:0/1647014713 wait complete. 2024-04-24T08:14:27.457 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T08:14:27.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:27 smithi090 ceph-mon[25016]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:28.458 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:28 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1647014713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:29.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:29 smithi090 ceph-mon[25016]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:29.670 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T08:14:29.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:14:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:29 smithi149 podman[31224]: 2024-04-24 08:14:29.856131119 +0000 UTC m=+0.162447170 container create db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, RELEASE=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T08:14:30.303 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 podman[31224]: 2024-04-24 08:14:30.077696433 +0000 UTC m=+0.384012550 container init db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD) 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 podman[31224]: 2024-04-24 08:14:30.088458177 +0000 UTC m=+0.394774229 container start db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux ) 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pidfile_write: ignore empty --pid-file 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: load: jerasure load: lrc 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Git sha 0 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Compile date 2023-08-03 19:21:13 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: DB SUMMARY 2024-04-24T08:14:30.450 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: DB Session ID: 9XPIBX07MNU9723H4RMY 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi149/store.db dir, Total Num: 0, files: 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi149/store.db: 000004.log size: 511 ; 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.env: 0x55fd1bfbc720 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:14:30.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.info_log: 0x55fd3cca32c0 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.statistics: (nil) 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.use_fsync: 0 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:14:30.452 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.db_log_dir: 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.wal_dir: 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:14:30.453 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.write_buffer_manager: 0x55fd3bf143c0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.unordered_write: 0 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:14:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:14:30.455 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.row_cache: None 2024-04-24T08:14:30.455 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.wal_filter: None 2024-04-24T08:14:30.455 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:14:30.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:14:30.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:14:30.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:14:30.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.wal_compression: 0 2024-04-24T08:14:30.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:14:30.457 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_open_files: -1 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:14:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Compression algorithms supported: 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kZSTD supported: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:14:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000005 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.merge_operator: 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_filter: None 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd3cca33c0) 2024-04-24T08:14:30.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_type: 0 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_index_type: 0 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_shortening: 1 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: checksum: 4 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: no_block_cache: 0 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache: 0x55fd3bfb5350 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_options: 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: capacity : 536870912 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_shard_bits : 4 2024-04-24T08:14:30.461 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: strict_capacity_limit : 0 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_compressed: (nil) 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: persistent_cache: (nil) 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size: 4096 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size_deviation: 10 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_restart_interval: 16 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_block_restart_interval: 1 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: metadata_block_size: 4096 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: partition_filters: 0 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: use_delta_encoding: 1 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: filter_policy: bloomfilter 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: whole_key_filtering: 1 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: verify_compression: 0 2024-04-24T08:14:30.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: format_version: 5 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_index_compression: 1 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_align: 0 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_auto_readahead_size: 262144 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: prepopulate_block_cache: 0 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression: NoCompression 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:14:30.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.num_levels: 7 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:14:30.464 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:14:30.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:14:30.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.table_properties_collectors: 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:14:30.467 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.ttl: 2592000 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:14:30.468 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T08:14:30.469 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 65e150f3-c259-4a1b-abcb-8c7e876726d3 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946470139483, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946470140555, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713946470, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "65e150f3-c259-4a1b-abcb-8c7e876726d3", "db_session_id": "9XPIBX07MNU9723H4RMY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713946470140794, "job": 1, "event": "recovery_finished"} 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi149/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fd3c052000 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: DB pointer 0x55fd3c03e000 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** DB Stats ** 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:14:30.470 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:14:30.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache BinnedLRUCache@0x55fd3bfb5350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149 does not exist in monmap, will attempt to join an existing cluster 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: starting mon.smithi149 rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi149 fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(???) e0 preinit fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:14:30.473 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 bash[31224]: db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).mds e1 new map 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).mds e1 print_map 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: e1 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: legacy client fscid: -1 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: No filesystems configured 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T08:14:30.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: [24/Apr/2024:08:13:50] ENGINE Bus STARTING 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: [24/Apr/2024:08:13:50] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:14:30.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mgrmap e16: smithi090.jsrfxo(active, since 1.01426s) 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: [24/Apr/2024:08:13:50] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: [24/Apr/2024:08:13:50] ENGINE Bus STARTED 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mgrmap e17: smithi090.jsrfxo(active, since 2s) 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/3622047407' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:14:30.476 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1466598826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:14:30.477 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1350541956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Deploying daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:14:30.479 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Deploying daemon crash.smithi149 on smithi149 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1497818647' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Deploying daemon node-exporter.smithi149 on smithi149 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1584877229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1758284325' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: Deploying daemon mon.smithi149 on smithi149 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1647014713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: mon.smithi149@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expand map: {default=false} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta from 'false' to 'false' 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expanded map: {default=false} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expand map: {default=info} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta from 'info' to 'info' 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expanded map: {default=info} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expand map: {default=daemon} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expanded map: {default=daemon} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expand map: {default=debug} 2024-04-24T08:14:30.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta from 'debug' to 'debug' 2024-04-24T08:14:30.484 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 ceph-mon[31282]: expand_channel_meta expanded map: {default=debug} 2024-04-24T08:14:30.484 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:30 smithi149 systemd[1]: Started Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.423+0000 7f22d1e00700 1 -- 172.21.15.149:0/3803731178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 msgr2=0x7f22b0005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.423+0000 7f22d1e00700 1 --2- 172.21.15.149:0/3803731178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22b0005a40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22cc073020 tx=0x7f22bc00feb0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.424+0000 7f22d1e00700 1 -- 172.21.15.149:0/3803731178 shutdown_connections 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.424+0000 7f22d1e00700 1 --2- 172.21.15.149:0/3803731178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22b0005a40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.424+0000 7f22d1e00700 1 -- 172.21.15.149:0/3803731178 >> 172.21.15.149:0/3803731178 conn(0x7f22cc0fba50 msgr2=0x7f22cc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.424+0000 7f22d1e00700 1 -- 172.21.15.149:0/3803731178 shutdown_connections 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.424+0000 7f22d1e00700 1 -- 172.21.15.149:0/3803731178 wait complete. 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.426+0000 7f22d1e00700 1 Processor -- start 2024-04-24T08:14:35.429 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22d1e00700 1 -- start start 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22d1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22d1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 0x7f22cc19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22d1e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc1a2410 con 0x7f22b00035c0 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22d1e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc19bed0 con 0x7f22cc1a1e50 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22caa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.427+0000 7f22caa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:59488/0 (socket says 172.21.15.149:59488) 2024-04-24T08:14:35.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22caa00700 1 -- 172.21.15.149:0/2065671936 learned_addr learned my addr 172.21.15.149:0/2065671936 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:14:35.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22ca000700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 0x7f22cc19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:35.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22caa00700 1 -- 172.21.15.149:0/2065671936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 msgr2=0x7f22cc19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:35.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22caa00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 0x7f22cc19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22caa00700 1 -- 172.21.15.149:0/2065671936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc00f900 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22ca000700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 0x7f22cc19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.428+0000 7f22caa00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0059d0 tx=0x7f22bc011590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.429+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22bc013b30 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.429+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22bc0044e0 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.429+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22bc011cd0 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.429+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc19c150 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.429+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc19c6a0 con 0x7f22b00035c0 2024-04-24T08:14:35.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.431+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc062380 con 0x7f22b00035c0 2024-04-24T08:14:35.433 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.431+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f22bc003680 con 0x7f22b00035c0 2024-04-24T08:14:35.433 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.433+0000 7f22d0a00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 0x7f22ac06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:35.434 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.433+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f22bc08d3e0 con 0x7f22b00035c0 2024-04-24T08:14:35.434 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.433+0000 7f22ca000700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 0x7f22ac06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:35.434 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.434+0000 7f22ca000700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 0x7f22ac06ec00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f22b80079b0 tx=0x7f22b8008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:35.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.439+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f22bc05c670 con 0x7f22b00035c0 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: mon.smithi090 calling monitor election 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: mon.smithi149 calling monitor election 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:14:35.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: monmap e2: 2 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: fsmap 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: mgrmap e17: smithi090.jsrfxo(active, since 45s) 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: overall HEALTH_OK 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:35.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:14:35.546 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:35.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.741+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f22cc1a6840 con 0x7f22b00035c0 2024-04-24T08:14:35.742 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: mon.smithi090 calling monitor election 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: mon.smithi149 calling monitor election 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:35.743 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: monmap e2: 2 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: fsmap 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: mgrmap e17: smithi090.jsrfxo(active, since 45s) 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: overall HEALTH_OK 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2942568790' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:14:35.744 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:35.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.742+0000 7f22d0a00700 1 -- 172.21.15.149:0/2065671936 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f22bc031020 con 0x7f22b00035c0 2024-04-24T08:14:35.745 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T08:14:35.745 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","modified":"2024-04-24T08:14:30.195856Z","created":"2024-04-24T08:10:48.372320Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi149","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 msgr2=0x7f22ac06ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 0x7f22ac06ec00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f22b80079b0 tx=0x7f22b8008040 comp rx=0 tx=0).stop 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 msgr2=0x7f22cc1a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0059d0 tx=0x7f22bc011590 comp rx=0 tx=0).stop 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 shutdown_connections 2024-04-24T08:14:35.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f22ac06c750 0x7f22ac06ec00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22b00035c0 0x7f22cc1a1910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 --2- 172.21.15.149:0/2065671936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22cc1a1e50 0x7f22cc19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 >> 172.21.15.149:0/2065671936 conn(0x7f22cc0fba50 msgr2=0x7f22cc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 shutdown_connections 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:14:35.745+0000 7f22d1e00700 1 -- 172.21.15.149:0/2065671936 wait complete. 2024-04-24T08:14:35.747 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: mgrmap e18: smithi090.jsrfxo(active, since 45s), standbys: smithi149.vavftl 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.465 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:14:36.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/2065671936' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:36.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:36.508 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T08:14:36.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: mgrmap e18: smithi090.jsrfxo(active, since 45s), standbys: smithi149.vavftl 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/2065671936' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:14:36.590 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:14:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:14:37.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:37 smithi149 ceph-mon[31282]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:37 smithi090 ceph-mon[25016]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.081+0000 7fea4b000700 1 -- 172.21.15.90:0/3285546330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 msgr2=0x7fea440734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.081+0000 7fea4b000700 1 --2- 172.21.15.90:0/3285546330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea440734c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea34009910 tx=0x7fea34009c20 comp rx=0 tx=0).stop 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.081+0000 7fea4b000700 1 -- 172.21.15.90:0/3285546330 shutdown_connections 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.082+0000 7fea4b000700 1 --2- 172.21.15.90:0/3285546330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea440734c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.082+0000 7fea4b000700 1 -- 172.21.15.90:0/3285546330 >> 172.21.15.90:0/3285546330 conn(0x7fea440fb970 msgr2=0x7fea440fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.082+0000 7fea4b000700 1 -- 172.21.15.90:0/3285546330 shutdown_connections 2024-04-24T08:14:39.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.082+0000 7fea4b000700 1 -- 172.21.15.90:0/3285546330 wait complete. 2024-04-24T08:14:39.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.083+0000 7fea4b000700 1 Processor -- start 2024-04-24T08:14:39.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.083+0000 7fea4b000700 1 -- start start 2024-04-24T08:14:39.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea4b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:39.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea4b000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:39.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea4b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea441a25a0 con 0x7fea440730f0 2024-04-24T08:14:39.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea4b000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4419bff0 con 0x7fea441a2000 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52224/0 (socket says 172.21.15.90:52224) 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea43e00700 1 -- 172.21.15.90:0/1573519331 learned_addr learned my addr 172.21.15.90:0/1573519331 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.084+0000 7fea43400700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43400700 1 -- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 msgr2=0x7fea4419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43400700 1 -- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 msgr2=0x7fea4419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43400700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43e00700 1 -- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 msgr2=0x7fea4419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43400700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43e00700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43e00700 1 -- 172.21.15.90:0/1573519331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340095f0 con 0x7fea440730f0 2024-04-24T08:14:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea43e00700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea34006010 tx=0x7fea3400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:39.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.085+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3401d070 con 0x7fea440730f0 2024-04-24T08:14:39.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.086+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea34005140 con 0x7fea440730f0 2024-04-24T08:14:39.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.086+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4419c220 con 0x7fea440730f0 2024-04-24T08:14:39.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.086+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea34017440 con 0x7fea440730f0 2024-04-24T08:14:39.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.086+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4419c710 con 0x7fea440730f0 2024-04-24T08:14:39.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.087+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea44062380 con 0x7fea440730f0 2024-04-24T08:14:39.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.089+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7fea34003a60 con 0x7fea440730f0 2024-04-24T08:14:39.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.090+0000 7fea40c00700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 0x7fea2806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:39.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.090+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fea3408bba0 con 0x7fea440730f0 2024-04-24T08:14:39.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.091+0000 7fea43400700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 0x7fea2806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:39.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.092+0000 7fea43400700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 0x7fea2806ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea30009850 tx=0x7fea30008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.095+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fea340572c0 con 0x7fea440730f0 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.323+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fea4419cfe0 con 0x7fea440730f0 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.324+0000 7fea40c00700 1 -- 172.21.15.90:0/1573519331 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7fea34026090 con 0x7fea440730f0 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:14:39.326 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-04-24T08:14:39.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 msgr2=0x7fea2806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 0x7fea2806ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea30009850 tx=0x7fea30008040 comp rx=0 tx=0).stop 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 msgr2=0x7fea441a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea34006010 tx=0x7fea3400bda0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 shutdown_connections 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fea2806c760 0x7fea2806ec10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea440730f0 0x7fea441a1ac0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 --2- 172.21.15.90:0/1573519331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea441a2000 0x7fea4419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 >> 172.21.15.90:0/1573519331 conn(0x7fea440fb970 msgr2=0x7fea440fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:39.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 shutdown_connections 2024-04-24T08:14:39.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:39.327+0000 7fea4b000700 1 -- 172.21.15.90:0/1573519331 wait complete. 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:14:39.686 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:39 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1573519331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:14:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:39 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1573519331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:40.103 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T08:14:40.103 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:14:40.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:14:40.173 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:14:40.173 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:40.282 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:14:40.282 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:14:40.358 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:14:40.358 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:14:40.476 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T08:14:40.477 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:14:40.477 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:14:40.518 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:14:40.518 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:40.596 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 08:13:39.295698462 +0000 2024-04-24T08:14:40.597 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 08:07:06.436860208 +0000 2024-04-24T08:14:40.597 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 08:07:06.436860208 +0000 2024-04-24T08:14:40.597 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 08:07:06.436860208 +0000 2024-04-24T08:14:40.597 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:14:40.655 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:40 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:14:40.655 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:40 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:14:40.656 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.656 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.656 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T08:14:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T08:14:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000400034 s, 1.3 MB/s 2024-04-24T08:14:40.702 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:14:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:40 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:14:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:40 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:14:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:40.789 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 08:13:39.296698447 +0000 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 08:07:07.021851675 +0000 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 08:07:07.021851675 +0000 2024-04-24T08:14:40.872 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 08:07:07.021851675 +0000 2024-04-24T08:14:40.872 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:14:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T08:14:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T08:14:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000338487 s, 1.5 MB/s 2024-04-24T08:14:40.981 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:14:41.029 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 08:13:39.297698432 +0000 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 08:07:07.625842864 +0000 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 08:07:07.625842864 +0000 2024-04-24T08:14:41.071 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 08:07:07.625842864 +0000 2024-04-24T08:14:41.072 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:14:41.185 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T08:14:41.186 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T08:14:41.186 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000313542 s, 1.6 MB/s 2024-04-24T08:14:41.189 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:14:41.230 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:41.309 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 08:13:39.297698432 +0000 2024-04-24T08:14:41.310 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 08:07:08.265833528 +0000 2024-04-24T08:14:41.310 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 08:07:08.265833528 +0000 2024-04-24T08:14:41.310 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 08:07:08.265833528 +0000 2024-04-24T08:14:41.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:14:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T08:14:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T08:14:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000469751 s, 1.1 MB/s 2024-04-24T08:14:41.413 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:14:41.458 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:14:41.459 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:14:41.517 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:14:41.517 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-04-24T08:14:41.571 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:14:41.571 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:41.571 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T08:14:41.571 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:41.572 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:41.572 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 08:14:29.346725862 +0000 2024-04-24T08:14:41.572 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 08:07:06.383816377 +0000 2024-04-24T08:14:41.572 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 08:07:06.383816377 +0000 2024-04-24T08:14:41.572 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 08:07:06.383816377 +0000 2024-04-24T08:14:41.572 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:14:41.653 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T08:14:41.653 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T08:14:41.653 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000486851 s, 1.1 MB/s 2024-04-24T08:14:41.657 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:14:41.711 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-04-24T08:14:41.759 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:14:41.759 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 08:14:29.347725843 +0000 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 08:07:07.005804279 +0000 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 08:07:07.005804279 +0000 2024-04-24T08:14:41.760 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 08:07:07.005804279 +0000 2024-04-24T08:14:41.760 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:14:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:14:41.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:14:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:41.839 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T08:14:41.839 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T08:14:41.839 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000375241 s, 1.4 MB/s 2024-04-24T08:14:41.843 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:14:41.941 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 08:14:29.347725843 +0000 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 08:07:07.642791889 +0000 2024-04-24T08:14:41.989 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 08:07:07.642791889 +0000 2024-04-24T08:14:41.990 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 08:07:07.642791889 +0000 2024-04-24T08:14:41.990 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:14:42.059 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T08:14:42.060 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T08:14:42.060 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000514801 s, 995 kB/s 2024-04-24T08:14:42.062 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:14:42.113 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 08:14:29.348725823 +0000 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 08:07:08.261779849 +0000 2024-04-24T08:14:42.161 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 08:07:08.261779849 +0000 2024-04-24T08:14:42.162 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 08:07:08.261779849 +0000 2024-04-24T08:14:42.162 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:14:42.283 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T08:14:42.284 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T08:14:42.284 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000403442 s, 1.3 MB/s 2024-04-24T08:14:42.287 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:14:42.344 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-04-24T08:14:42.344 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:14:42.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:42 smithi149 ceph-mon[31282]: Reconfiguring mon.smithi090 (unknown last config time)... 2024-04-24T08:14:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:42 smithi149 ceph-mon[31282]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-24T08:14:42.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:42 smithi090 ceph-mon[25016]: Reconfiguring mon.smithi090 (unknown last config time)... 2024-04-24T08:14:42.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:42 smithi090 ceph-mon[25016]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-24T08:14:42.848 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:14:43.713 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:14:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:14:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:44.561 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:44 smithi090 ceph-mon[25016]: Reconfiguring mgr.smithi090.jsrfxo (unknown last config time)... 2024-04-24T08:14:44.561 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:44 smithi090 ceph-mon[25016]: Reconfiguring daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:14:44.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:44 smithi149 ceph-mon[31282]: Reconfiguring mgr.smithi090.jsrfxo (unknown last config time)... 2024-04-24T08:14:44.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:44 smithi149 ceph-mon[31282]: Reconfiguring daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0347774 s, 302 MB/s 2024-04-24T08:14:45.580 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T08:14:45.581 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:45 smithi090 ceph-mon[25016]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:45.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:45 smithi149 ceph-mon[31282]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:46.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-04-24T08:14:47.101 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:47.101 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:47.101 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:47.102 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-24T08:14:47.102 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:14:47.102 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:47.102 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:46 smithi090 ceph-mon[25016]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:14:47.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:47.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:47.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:47.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-24T08:14:47.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:14:47.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:47.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:46 smithi149 ceph-mon[31282]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:14:47.970 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:49.157 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:49 smithi090 ceph-mon[25016]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:49 smithi149 ceph-mon[31282]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-24T08:14:50.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:50.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 -- 172.21.15.90:0/281963959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 msgr2=0x7f12c8073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:50.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 --2- 172.21.15.90:0/281963959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8073c20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009940 tx=0x7f12b0009c50 comp rx=0 tx=0).stop 2024-04-24T08:14:50.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 -- 172.21.15.90:0/281963959 shutdown_connections 2024-04-24T08:14:50.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 --2- 172.21.15.90:0/281963959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8073c20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 --2- 172.21.15.90:0/281963959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12c8074d80 0x7f12c80731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 -- 172.21.15.90:0/281963959 >> 172.21.15.90:0/281963959 conn(0x7f12c80fb7f0 msgr2=0x7f12c80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 -- 172.21.15.90:0/281963959 shutdown_connections 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.016+0000 7f12ce600700 1 -- 172.21.15.90:0/281963959 wait complete. 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 Processor -- start 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 -- start start 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12c8074d80 0x7f12c81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81987e0 con 0x7f12c80737b0 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.018+0000 7f12ce600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8198950 con 0x7f12c8074d80 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54204/0 (socket says 172.21.15.90:54204) 2024-04-24T08:14:50.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 -- 172.21.15.90:0/3385232014 learned_addr learned my addr 172.21.15.90:0/3385232014 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 -- 172.21.15.90:0/3385232014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12c8074d80 msgr2=0x7f12c81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12c8074d80 0x7f12c81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 -- 172.21.15.90:0/3385232014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b00095f0 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.019+0000 7f12c7400700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c8a0 tx=0x7f12b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.020+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b800cea0 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.020+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f12b8004830 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.020+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b80055b0 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.020+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c819d3e0 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.020+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c819d930 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.021+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c810af80 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.022+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f12b8007ad0 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.024+0000 7f12c4c00700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 0x7f12ac06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.024+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f12b808a420 con 0x7f12c80737b0 2024-04-24T08:14:50.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.024+0000 7f12bea00700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 0x7f12ac06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:14:50.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.025+0000 7f12bea00700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 0x7f12ac06eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12b0006010 tx=0x7f12b0000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:14:50.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.028+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f12b8055c60 con 0x7f12c80737b0 2024-04-24T08:14:50.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:14:50.257+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f12c8107740 con 0x7f12ac06c610 2024-04-24T08:14:50.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:14:51.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:51.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:51.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:52.412 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:52 smithi090 ceph-mon[25016]: Reconfiguring alertmanager.smithi090 (dependencies changed)... 2024-04-24T08:14:52.412 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:52 smithi090 ceph-mon[25016]: Reconfiguring daemon alertmanager.smithi090 on smithi090 2024-04-24T08:14:52.500 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:52 smithi149 ceph-mon[31282]: Reconfiguring alertmanager.smithi090 (dependencies changed)... 2024-04-24T08:14:52.500 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:52 smithi149 ceph-mon[31282]: Reconfiguring daemon alertmanager.smithi090 on smithi090 2024-04-24T08:14:53.502 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:53 smithi090 ceph-mon[25016]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:53.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:53 smithi149 ceph-mon[31282]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:55 smithi090 ceph-mon[25016]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:55 smithi149 ceph-mon[31282]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:57.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:57 smithi149 ceph-mon[31282]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:57 smithi090 ceph-mon[25016]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:59 smithi149 ceph-mon[31282]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:59 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:59 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:59 smithi149 ceph-mon[31282]: Reconfiguring grafana.smithi090 (dependencies changed)... 2024-04-24T08:14:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:14:59 smithi149 ceph-mon[31282]: Reconfiguring daemon grafana.smithi090 on smithi090 2024-04-24T08:14:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:59 smithi090 ceph-mon[25016]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:14:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:59 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:59 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:14:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:59 smithi090 ceph-mon[25016]: Reconfiguring grafana.smithi090 (dependencies changed)... 2024-04-24T08:14:59.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:14:59 smithi090 ceph-mon[25016]: Reconfiguring daemon grafana.smithi090 on smithi090 2024-04-24T08:15:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:00 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2790232035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46ccda0-8129-4110-8a35-c132eb64e5ae"}]: dispatch 2024-04-24T08:15:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:00 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2790232035' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f46ccda0-8129-4110-8a35-c132eb64e5ae"}]': finished 2024-04-24T08:15:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:00 smithi149 ceph-mon[31282]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:15:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:00 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:00 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2790232035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46ccda0-8129-4110-8a35-c132eb64e5ae"}]: dispatch 2024-04-24T08:15:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:00 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2790232035' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f46ccda0-8129-4110-8a35-c132eb64e5ae"}]': finished 2024-04-24T08:15:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:00 smithi090 ceph-mon[25016]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:15:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:00 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:01 smithi149 ceph-mon[31282]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:01 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/939882411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:15:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:01 smithi090 ceph-mon[25016]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:01 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/939882411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:15:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:03 smithi149 ceph-mon[31282]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:03.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:03 smithi090 ceph-mon[25016]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:05 smithi090 ceph-mon[25016]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:05 smithi149 ceph-mon[31282]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:07.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:07 smithi090 ceph-mon[25016]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:07 smithi149 ceph-mon[31282]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:09 smithi090 ceph-mon[25016]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:09 smithi149 ceph-mon[31282]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:11.627 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:11 smithi090 ceph-mon[25016]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:11 smithi149 ceph-mon[31282]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:13.138 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:13.138 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:13.138 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:12 smithi090 ceph-mon[25016]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:15:13.138 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:12 smithi090 ceph-mon[25016]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:13.139 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:12 smithi090 ceph-mon[25016]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:15:13.271 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:13.271 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:13.271 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:12 smithi149 ceph-mon[31282]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:15:13.271 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:12 smithi149 ceph-mon[31282]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:13.271 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:12 smithi149 ceph-mon[31282]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:15:14.127 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:15:14.127 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:14.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:13 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:15:14.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:13 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:15.195 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:14 smithi090 ceph-mon[25016]: Deploying daemon osd.0 on smithi090 2024-04-24T08:15:15.196 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:14 smithi090 ceph-mon[25016]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:15.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:14 smithi149 ceph-mon[31282]: Deploying daemon osd.0 on smithi090 2024-04-24T08:15:15.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:14 smithi149 ceph-mon[31282]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:17.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:17 smithi090 ceph-mon[25016]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:17 smithi149 ceph-mon[31282]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:19.455 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:19 smithi090 ceph-mon[25016]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:19.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:19 smithi149 ceph-mon[31282]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:20.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:20.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: Reconfiguring ceph-exporter.smithi149 (monmap changed)... 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: Reconfiguring daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:20.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: Reconfiguring ceph-exporter.smithi149 (monmap changed)... 2024-04-24T08:15:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:15:20.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:20.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: Reconfiguring daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:15:20.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:20.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:22.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:22.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:22.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: Reconfiguring crash.smithi149 (monmap changed)... 2024-04-24T08:15:22.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:15:22.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:22.608 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: Reconfiguring daemon crash.smithi149 on smithi149 2024-04-24T08:15:22.608 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:22 smithi149 ceph-mon[31282]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: Reconfiguring crash.smithi149 (monmap changed)... 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:22.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: Reconfiguring daemon crash.smithi149 on smithi149 2024-04-24T08:15:22.783 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:22 smithi090 ceph-mon[25016]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:24.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:24.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: Reconfiguring mgr.smithi149.vavftl (monmap changed)... 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: Reconfiguring daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:15:24.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:24 smithi090 ceph-mon[25016]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:25.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:25.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: Reconfiguring mgr.smithi149.vavftl (monmap changed)... 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: Reconfiguring daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:15:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:24 smithi149 ceph-mon[31282]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:26.594 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:26.594 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: Reconfiguring mon.smithi149 (monmap changed)... 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: Reconfiguring daemon mon.smithi149 on smithi149 2024-04-24T08:15:26.595 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:26 smithi090 ceph-mon[25016]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: Reconfiguring mon.smithi149 (monmap changed)... 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: Reconfiguring daemon mon.smithi149 on smithi149 2024-04-24T08:15:26.622 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:26 smithi149 ceph-mon[31282]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T08:15:27.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:27.837 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:15:28.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:28.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T08:15:29.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:28 smithi149 ceph-mon[31282]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T08:15:29.030 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:28 smithi090 ceph-mon[25016]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:30.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.839+0000 7f12c4c00700 1 -- 172.21.15.90:0/3385232014 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f12c8107740 con 0x7f12ac06c610 2024-04-24T08:15:30.840 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 msgr2=0x7f12ac06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 0x7f12ac06eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12b0006010 tx=0x7f12b0000bc0 comp rx=0 tx=0).stop 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 msgr2=0x7f12c8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c8a0 tx=0x7f12b800cc60 comp rx=0 tx=0).stop 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 shutdown_connections 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f12ac06c610 0x7f12ac06eac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c80737b0 0x7f12c8197ca0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 --2- 172.21.15.90:0/3385232014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12c8074d80 0x7f12c81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 >> 172.21.15.90:0/3385232014 conn(0x7f12c80fb7f0 msgr2=0x7f12c8106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 shutdown_connections 2024-04-24T08:15:30.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:30.842+0000 7f12ce600700 1 -- 172.21.15.90:0/3385232014 wait complete. 2024-04-24T08:15:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:15:31.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.634 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T08:15:31.639 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_3... 2024-04-24T08:15:31.639 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:15:31.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:31.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:15:31.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:31.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:32.425 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:15:32.755 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-mon[25016]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:15:32.755 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-mon[25016]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:15:32.756 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:32.756 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-mon[25016]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:15:32.756 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[39906]: 2024-04-24T08:15:32.497+0000 7f922a400700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T08:15:32.756 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:15:32 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[39906]: 2024-04-24T08:15:32.502+0000 7f9221600700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:15:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:32 smithi149 ceph-mon[31282]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:15:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:32 smithi149 ceph-mon[31282]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:15:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:32 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:32 smithi149 ceph-mon[31282]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:15:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:33 smithi149 ceph-mon[31282]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:33 smithi149 ceph-mon[31282]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:15:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:33 smithi149 ceph-mon[31282]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T08:15:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:33 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:33 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:33.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:33 smithi090 ceph-mon[25016]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:33.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:33 smithi090 ceph-mon[25016]: from='osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:15:33.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:33 smithi090 ceph-mon[25016]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T08:15:33.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:33 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:33.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:33 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:34 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:15:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:34 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:15:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:34 smithi149 ceph-mon[31282]: osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] boot 2024-04-24T08:15:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:34 smithi149 ceph-mon[31282]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:15:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:34 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:34 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:15:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:34 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:15:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:34 smithi090 ceph-mon[25016]: osd.0 [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] boot 2024-04-24T08:15:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:34 smithi090 ceph-mon[25016]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:15:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:34 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:15:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:35 smithi149 ceph-mon[31282]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:35 smithi149 ceph-mon[31282]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T08:15:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:35 smithi090 ceph-mon[25016]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:15:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:35 smithi090 ceph-mon[25016]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T08:15:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0329764 s, 318 MB/s 2024-04-24T08:15:36.041 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T08:15:36.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-04-24T08:15:37.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:37 smithi090 ceph-mon[25016]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:15:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:37 smithi149 ceph-mon[31282]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:38.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:38 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:38.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:38 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:38.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:38 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:38.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:38 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 -- 172.21.15.90:0/3464581060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 msgr2=0x7f69d8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 --2- 172.21.15.90:0/3464581060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d8101890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009910 tx=0x7f69c4009c20 comp rx=0 tx=0).stop 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 -- 172.21.15.90:0/3464581060 shutdown_connections 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 --2- 172.21.15.90:0/3464581060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d8101890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 --2- 172.21.15.90:0/3464581060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 0x7f69d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.598+0000 7f69de200700 1 -- 172.21.15.90:0/3464581060 >> 172.21.15.90:0/3464581060 conn(0x7f69d80fb7f0 msgr2=0x7f69d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:15:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.599+0000 7f69de200700 1 -- 172.21.15.90:0/3464581060 shutdown_connections 2024-04-24T08:15:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.599+0000 7f69de200700 1 -- 172.21.15.90:0/3464581060 wait complete. 2024-04-24T08:15:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.600+0000 7f69de200700 1 Processor -- start 2024-04-24T08:15:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.600+0000 7f69de200700 1 -- start start 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69de200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 0x7f69d8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69de200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69de200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81987d0 con 0x7f69d8101440 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69de200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8198940 con 0x7f69d8100240 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69d6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69d6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50888/0 (socket says 172.21.15.90:50888) 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69d6a00700 1 -- 172.21.15.90:0/3341605945 learned_addr learned my addr 172.21.15.90:0/3341605945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.601+0000 7f69d7400700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 0x7f69d8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.602+0000 7f69d6a00700 1 -- 172.21.15.90:0/3341605945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 msgr2=0x7f69d8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:15:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.602+0000 7f69d6a00700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 0x7f69d8197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:15:39.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.602+0000 7f69d6a00700 1 -- 172.21.15.90:0/3341605945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40095f0 con 0x7f69d8101440 2024-04-24T08:15:39.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.602+0000 7f69d6a00700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69c400b5c0 tx=0x7f69c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:15:39.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.603+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c401d070 con 0x7f69d8101440 2024-04-24T08:15:39.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.603+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d819d370 con 0x7f69d8101440 2024-04-24T08:15:39.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.603+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f69c4022470 con 0x7f69d8101440 2024-04-24T08:15:39.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.603+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c400f690 con 0x7f69d8101440 2024-04-24T08:15:39.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.604+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d819d860 con 0x7f69d8101440 2024-04-24T08:15:39.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.605+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f69c4004af0 con 0x7f69d8101440 2024-04-24T08:15:39.607 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:39 smithi090 ceph-mon[25016]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:39.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.605+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8105520 con 0x7f69d8101440 2024-04-24T08:15:39.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.606+0000 7f69cfe00700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 0x7f69bc06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:15:39.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.607+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f69c4095b70 con 0x7f69d8101440 2024-04-24T08:15:39.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.608+0000 7f69d7400700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 0x7f69bc06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:15:39.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.609+0000 7f69d7400700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 0x7f69bc06eac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f69d8198b70 tx=0x7f69c800a380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:15:39.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.612+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f69c4092060 con 0x7f69d8101440 2024-04-24T08:15:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:39 smithi149 ceph-mon[31282]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:39.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:15:39.859+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f69d8061190 con 0x7f69bc06c610 2024-04-24T08:15:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:15:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:15:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:15:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:15:40.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:41.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:41 smithi090 ceph-mon[25016]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:15:41.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:41 smithi090 ceph-mon[25016]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:41 smithi149 ceph-mon[31282]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:15:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:41 smithi149 ceph-mon[31282]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:43.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:43 smithi090 ceph-mon[25016]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:43 smithi149 ceph-mon[31282]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi090 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi090 to 11841M 2024-04-24T08:15:45.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.478 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:45.478 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:15:45.478 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.478 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi090 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi090 to 11841M 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:15:46.489 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:46 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3887901573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "601421cf-b681-483d-93cf-5387d607186d"}]: dispatch 2024-04-24T08:15:46.489 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:46 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3887901573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "601421cf-b681-483d-93cf-5387d607186d"}]': finished 2024-04-24T08:15:46.489 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:46 smithi149 ceph-mon[31282]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:15:46.489 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:15:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:46 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3887901573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "601421cf-b681-483d-93cf-5387d607186d"}]: dispatch 2024-04-24T08:15:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:46 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3887901573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "601421cf-b681-483d-93cf-5387d607186d"}]': finished 2024-04-24T08:15:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:46 smithi090 ceph-mon[25016]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:15:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:15:47.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:47 smithi149 ceph-mon[31282]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:47 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1047977359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:15:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:47 smithi090 ceph-mon[25016]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:47 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1047977359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:15:48.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:48.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:49.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:49 smithi149 ceph-mon[31282]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:49.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:49 smithi090 ceph-mon[25016]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:50.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:15:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:51.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:51 smithi149 ceph-mon[31282]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:15:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:51 smithi090 ceph-mon[25016]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:15:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:15:53.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:53 smithi090 ceph-mon[25016]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:53.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:53 smithi149 ceph-mon[31282]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:55 smithi149 ceph-mon[31282]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:55 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:15:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:55 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:55 smithi090 ceph-mon[25016]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:55 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:15:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:55 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:15:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:56 smithi149 ceph-mon[31282]: Deploying daemon osd.1 on smithi090 2024-04-24T08:15:56.527 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:56 smithi090 ceph-mon[25016]: Deploying daemon osd.1 on smithi090 2024-04-24T08:15:57.502 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:57 smithi090 ceph-mon[25016]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:57.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:57 smithi149 ceph-mon[31282]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:59.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:15:59 smithi149 ceph-mon[31282]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:15:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:15:59 smithi090 ceph-mon[25016]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:01 smithi149 ceph-mon[31282]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:01.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:01 smithi090 ceph-mon[25016]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:03.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:03 smithi149 ceph-mon[31282]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:03.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:03.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:03 smithi090 ceph-mon[25016]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:05.374 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:05 smithi090 ceph-mon[25016]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:05.374 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:05.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:05 smithi149 ceph-mon[31282]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:07.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:06 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:07.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:06 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:07.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:06 smithi149 ceph-mon[31282]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:07.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:06 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:07.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:06 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:07.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:06 smithi090 ceph-mon[25016]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:07.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:07 smithi090 ceph-mon[25016]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:16:07.857 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 1 on host 'smithi090' 2024-04-24T08:16:07.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.856+0000 7f69cfe00700 1 -- 172.21.15.90:0/3341605945 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f69d8061190 con 0x7f69bc06c610 2024-04-24T08:16:07.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 msgr2=0x7f69bc06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 0x7f69bc06eac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f69d8198b70 tx=0x7f69c800a380 comp rx=0 tx=0).stop 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 msgr2=0x7f69d81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69c400b5c0 tx=0x7f69c400bd70 comp rx=0 tx=0).stop 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 shutdown_connections 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f69bc06c610 0x7f69bc06eac0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d8100240 0x7f69d8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 --2- 172.21.15.90:0/3341605945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69d8101440 0x7f69d81981d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:07.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 >> 172.21.15.90:0/3341605945 conn(0x7f69d80fb7f0 msgr2=0x7f69d8102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:16:07.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 shutdown_connections 2024-04-24T08:16:07.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:07.860+0000 7f69de200700 1 -- 172.21.15.90:0/3341605945 wait complete. 2024-04-24T08:16:08.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:07 smithi149 ceph-mon[31282]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:08.621 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:08.622 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:08 smithi090 ceph-mon[25016]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:08.862 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T08:16:08.869 INFO:tasks.cephadm:Deploying osd.2 on smithi090 with /dev/vg_nvme/lv_2... 2024-04-24T08:16:08.870 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:16:09.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:09.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:08 smithi149 ceph-mon[31282]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:09.407 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:16:09.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:09 smithi090 ceph-mon[25016]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:16:09.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:09 smithi090 ceph-mon[25016]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T08:16:09.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:09.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:10.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:09 smithi149 ceph-mon[31282]: from='osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:16:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:09 smithi149 ceph-mon[31282]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T08:16:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:10.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:16:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:16:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] boot 2024-04-24T08:16:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:16:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:10 smithi090 ceph-mon[25016]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:11.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:16:11.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:16:11.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: osd.1 [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] boot 2024-04-24T08:16:11.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:16:11.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:16:11.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:10 smithi149 ceph-mon[31282]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:16:11.953 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:11 smithi090 ceph-mon[25016]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T08:16:12.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:11 smithi149 ceph-mon[31282]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T08:16:13.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:12 smithi149 ceph-mon[31282]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:13.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:12 smithi090 ceph-mon[25016]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0518678 s, 202 MB/s 2024-04-24T08:16:13.300 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T08:16:14.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-04-24T08:16:14.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:13 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:14.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:13 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:14.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:13 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:15.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:14 smithi149 ceph-mon[31282]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:14 smithi090 ceph-mon[25016]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:15.584 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:16:17.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:17 smithi090 ceph-mon[25016]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:17 smithi149 ceph-mon[31282]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.881+0000 7f6324c00700 1 -- 172.21.15.90:0/414758056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320103330 msgr2=0x7f6320105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.881+0000 7f6324c00700 1 --2- 172.21.15.90:0/414758056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320103330 0x7f6320105790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f630c009910 tx=0x7f630c009c20 comp rx=0 tx=0).stop 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.881+0000 7f6324c00700 1 -- 172.21.15.90:0/414758056 shutdown_connections 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.881+0000 7f6324c00700 1 --2- 172.21.15.90:0/414758056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320103330 0x7f6320105790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.881+0000 7f6324c00700 1 --2- 172.21.15.90:0/414758056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320069250 0x7f6320102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.882+0000 7f6324c00700 1 -- 172.21.15.90:0/414758056 >> 172.21.15.90:0/414758056 conn(0x7f63200fa780 msgr2=0x7f63200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.882+0000 7f6324c00700 1 -- 172.21.15.90:0/414758056 shutdown_connections 2024-04-24T08:16:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.882+0000 7f6324c00700 1 -- 172.21.15.90:0/414758056 wait complete. 2024-04-24T08:16:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 Processor -- start 2024-04-24T08:16:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 -- start start 2024-04-24T08:16:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320103330 0x7f63200fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63200ff2d0 con 0x7f6320069250 2024-04-24T08:16:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f6324c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63200ff410 con 0x7f6320103330 2024-04-24T08:16:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.883+0000 7f631de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50132/0 (socket says 172.21.15.90:50132) 2024-04-24T08:16:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 -- 172.21.15.90:0/55093682 learned_addr learned my addr 172.21.15.90:0/55093682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631d400700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320103330 0x7f63200fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 -- 172.21.15.90:0/55093682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320103330 msgr2=0x7f63200fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320103330 0x7f63200fed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 -- 172.21.15.90:0/55093682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0095f0 con 0x7f6320069250 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.884+0000 7f631de00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f631000e8c0 tx=0x7f631000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.885+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631000cba0 con 0x7f6320069250 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.885+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f631000eec0 con 0x7f6320069250 2024-04-24T08:16:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.885+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63200ff6f0 con 0x7f6320069250 2024-04-24T08:16:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.885+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6310018650 con 0x7f6320069250 2024-04-24T08:16:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.885+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632019db50 con 0x7f6320069250 2024-04-24T08:16:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.886+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632018fc10 con 0x7f6320069250 2024-04-24T08:16:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.889+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f631000cd00 con 0x7f6320069250 2024-04-24T08:16:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.889+0000 7f6316a00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 0x7f630406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.890+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f6310014070 con 0x7f6320069250 2024-04-24T08:16:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.890+0000 7f631d400700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 0x7f630406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:17.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.891+0000 7f631d400700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 0x7f630406eac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b6f0 tx=0x7f630c009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:16:17.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:17.895+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6310051720 con 0x7f6320069250 2024-04-24T08:16:18.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:18.175+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6320061190 con 0x7f630406c610 2024-04-24T08:16:18.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:18 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:16:18.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:18 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:16:19.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:19.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:16:19.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:19.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:19.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:19.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:19.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:19.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:20.421 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:20 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi090 2024-04-24T08:16:20.421 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:20 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi090 to 5920M 2024-04-24T08:16:20.421 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:20.422 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:20 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi090 2024-04-24T08:16:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:20 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi090 to 5920M 2024-04-24T08:16:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:21.452 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:21 smithi090 ceph-mon[25016]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:21.522 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:21 smithi149 ceph-mon[31282]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:22.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/4003981782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "207e3d5c-8b5e-44dd-b55f-e67f20623236"}]: dispatch 2024-04-24T08:16:22.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/4003981782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "207e3d5c-8b5e-44dd-b55f-e67f20623236"}]': finished 2024-04-24T08:16:22.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:22 smithi090 ceph-mon[25016]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:16:22.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:22.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:22.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/4003981782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "207e3d5c-8b5e-44dd-b55f-e67f20623236"}]: dispatch 2024-04-24T08:16:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/4003981782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "207e3d5c-8b5e-44dd-b55f-e67f20623236"}]': finished 2024-04-24T08:16:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:22 smithi149 ceph-mon[31282]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:16:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:23.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:23 smithi149 ceph-mon[31282]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:23.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:23 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:23.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:23 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1521372462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:16:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:23 smithi090 ceph-mon[25016]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:23 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:23 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1521372462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:16:25.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:25 smithi149 ceph-mon[31282]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:25.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:25 smithi090 ceph-mon[25016]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:26 smithi149 ceph-mon[31282]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:27.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:26 smithi090 ceph-mon[25016]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:29.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:29 smithi090 ceph-mon[25016]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:29 smithi149 ceph-mon[31282]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:31 smithi149 ceph-mon[31282]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:31.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:31 smithi090 ceph-mon[25016]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:32 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:16:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:32 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:32.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:32 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:16:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:32 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:33 smithi149 ceph-mon[31282]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:33.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:33 smithi149 ceph-mon[31282]: Deploying daemon osd.2 on smithi090 2024-04-24T08:16:33.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:33 smithi090 ceph-mon[25016]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:33.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:33 smithi090 ceph-mon[25016]: Deploying daemon osd.2 on smithi090 2024-04-24T08:16:35.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:35 smithi090 ceph-mon[25016]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:35.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:35 smithi149 ceph-mon[31282]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:37.503 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:37 smithi090 ceph-mon[25016]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:37 smithi149 ceph-mon[31282]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:39 smithi149 ceph-mon[31282]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:39 smithi090 ceph-mon[25016]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:41.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:41 smithi090 ceph-mon[25016]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:41.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:41.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:41.385 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:41 smithi149 ceph-mon[31282]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:43 smithi149 ceph-mon[31282]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:43.540 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:43 smithi090 ceph-mon[25016]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:44.281 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:44 smithi090 ceph-mon[25016]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:16:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:44 smithi149 ceph-mon[31282]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:16:45.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.318+0000 7f6316a00700 1 -- 172.21.15.90:0/55093682 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6320061190 con 0x7f630406c610 2024-04-24T08:16:45.320 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 2 on host 'smithi090' 2024-04-24T08:16:45.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 msgr2=0x7f630406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:45.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 0x7f630406eac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b6f0 tx=0x7f630c009f70 comp rx=0 tx=0).stop 2024-04-24T08:16:45.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 msgr2=0x7f6320100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:45.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f631000e8c0 tx=0x7f631000ec80 comp rx=0 tx=0).stop 2024-04-24T08:16:45.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 shutdown_connections 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f630406c610 0x7f630406eac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6320069250 0x7f6320100740 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 --2- 172.21.15.90:0/55093682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6320103330 0x7f63200fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.321+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 >> 172.21.15.90:0/55093682 conn(0x7f63200fa780 msgr2=0x7f63200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.322+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 shutdown_connections 2024-04-24T08:16:45.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:16:45.322+0000 7f6324c00700 1 -- 172.21.15.90:0/55093682 wait complete. 2024-04-24T08:16:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:45 smithi149 ceph-mon[31282]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:45 smithi149 ceph-mon[31282]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:16:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:45 smithi149 ceph-mon[31282]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:16:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:45 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:45 smithi149 ceph-mon[31282]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:16:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:45 smithi090 ceph-mon[25016]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:45 smithi090 ceph-mon[25016]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:16:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:45 smithi090 ceph-mon[25016]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:16:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:45 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:45 smithi090 ceph-mon[25016]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:46.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:46 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: from='osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T08:16:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T08:16:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:46 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:46.710 DEBUG:teuthology.orchestra.run.smithi090:osd.2> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T08:16:46.716 INFO:tasks.cephadm:Deploying osd.3 on smithi149 with /dev/vg_nvme/lv_4... 2024-04-24T08:16:46.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:16:47.130 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] boot 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:16:47.501 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:47 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: osd.2 [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] boot 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:16:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:47 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:16:48.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:48 smithi090 ceph-mon[25016]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:16:48.886 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:48 smithi149 ceph-mon[31282]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: pgmap v97: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:49.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350161 s, 299 MB/s 2024-04-24T08:16:49.992 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: pgmap v97: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:50.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:50.811 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-04-24T08:16:51.006 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:16:51.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:50 smithi090 ceph-mon[25016]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T08:16:51.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:50 smithi090 ceph-mon[25016]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:51.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:51.007 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:50 smithi090 sudo[52857]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:16:51.007 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:50 smithi090 sudo[52857]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:51.007 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:50 smithi090 sudo[52857]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:51.007 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:50 smithi090 sudo[52857]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:51.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:16:51.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:50 smithi149 ceph-mon[31282]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T08:16:51.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:50 smithi149 ceph-mon[31282]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:51.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:16:51.203 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:16:51.543 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52877]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:16:51.544 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:51.544 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52877]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:51.544 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52877]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:51.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:51 smithi090 ceph-mon[25016]: osdmap e23: 3 total, 3 up, 3 in 2024-04-24T08:16:51.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:51 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:52.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:51 smithi149 ceph-mon[31282]: osdmap e23: 3 total, 3 up, 3 in 2024-04-24T08:16:52.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:51 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:52.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52883]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:16:52.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:52.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52883]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:52.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:51 smithi090 sudo[52883]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:52.699 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:52 smithi090 sudo[52907]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:16:52.699 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:52 smithi090 sudo[52907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:52.699 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:52 smithi090 sudo[52907]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:52.699 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:16:52 smithi090 sudo[52907]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:52.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:52 smithi149 ceph-mon[31282]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:52 smithi090 ceph-mon[25016]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 -- 172.21.15.149:0/1695204155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 msgr2=0x7f7ba0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/1695204155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0101730 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009910 tx=0x7f7b8c009c20 comp rx=0 tx=0).stop 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 -- 172.21.15.149:0/1695204155 shutdown_connections 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/1695204155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0101730 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/1695204155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 0x7f7ba0100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 -- 172.21.15.149:0/1695204155 >> 172.21.15.149:0/1695204155 conn(0x7f7ba00fb650 msgr2=0x7f7ba00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 -- 172.21.15.149:0/1695204155 shutdown_connections 2024-04-24T08:16:53.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.324+0000 7f7ba6e00700 1 -- 172.21.15.149:0/1695204155 wait complete. 2024-04-24T08:16:53.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.325+0000 7f7ba6e00700 1 Processor -- start 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba6e00700 1 -- start start 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba6e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 0x7f7ba0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba6e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba6e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0198580 con 0x7f7ba0100020 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba6e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01986c0 con 0x7f7ba01012c0 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:51572/0 (socket says 172.21.15.149:51572) 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.326+0000 7f7ba5000700 1 -- 172.21.15.149:0/3131714241 learned_addr learned my addr 172.21.15.149:0/3131714241 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:16:53.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7ba5a00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 0x7f7ba0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:53.329 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7ba5000700 1 -- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 msgr2=0x7f7ba0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:16:53.329 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7ba5000700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 0x7f7ba0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:16:53.329 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7ba5000700 1 -- 172.21.15.149:0/3131714241 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0095f0 con 0x7f7ba01012c0 2024-04-24T08:16:53.329 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7ba5000700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005b40 tx=0x7f7b8c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.327+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8c01d070 con 0x7f7ba01012c0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.328+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c00f460 con 0x7f7ba01012c0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.328+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba019d110 con 0x7f7ba01012c0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.328+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8c021620 con 0x7f7ba01012c0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.328+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba019d600 con 0x7f7ba01012c0 2024-04-24T08:16:53.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.329+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba01053a0 con 0x7f7ba01012c0 2024-04-24T08:16:53.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.330+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7b8c00fa40 con 0x7f7ba01012c0 2024-04-24T08:16:53.332 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.331+0000 7f7b9e000700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 0x7f7b8806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:16:53.332 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.331+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f7b8c08c2f0 con 0x7f7ba01012c0 2024-04-24T08:16:53.332 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.331+0000 7f7ba5a00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 0x7f7b8806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:16:53.333 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.332+0000 7f7ba5a00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 0x7f7b8806eb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7b94003f90 tx=0x7f7b94009180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:16:53.337 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.336+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7b8c05aca0 con 0x7f7ba01012c0 2024-04-24T08:16:53.388 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[52923]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:16:53.388 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[52923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:53.388 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[52923]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:53.388 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[52923]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:53.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:16:53.561+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ba0061190 con 0x7f7b8806c680 2024-04-24T08:16:54.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:53 smithi149 ceph-mon[31282]: mgrmap e19: smithi090.jsrfxo(active, since 3m), standbys: smithi149.vavftl 2024-04-24T08:16:54.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:53 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:16:54.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:53 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:16:54.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:53 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:54.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:53 smithi090 ceph-mon[25016]: mgrmap e19: smithi090.jsrfxo(active, since 3m), standbys: smithi149.vavftl 2024-04-24T08:16:54.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:53 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:16:54.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:53 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:16:54.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:53 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:54.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[53308]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:16:54.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[53308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:54.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[53308]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:54.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:16:53 smithi090 sudo[53308]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53312]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:16:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53312]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53312]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53312]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:16:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:54 smithi090 sudo[53315]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:55 smithi149 sudo[35115]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:16:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:55 smithi149 sudo[35115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:55 smithi149 sudo[35115]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:55 smithi149 sudo[35115]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:56.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 sudo[35144]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:16:56.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 sudo[35144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:16:56.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 sudo[35144]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-24T08:16:56.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 sudo[35144]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi090 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:16:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi090 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:16:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:16:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:16:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 99 KiB/s wr, 6 op/s 2024-04-24T08:16:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:16:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi090 to 3947M 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:57 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 99 KiB/s wr, 6 op/s 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi090 to 3947M 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:16:57.531 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:16:57.532 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.532 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:16:57.532 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:57 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:16:58.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/327501287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]: dispatch 2024-04-24T08:16:58.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]: dispatch 2024-04-24T08:16:58.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]': finished 2024-04-24T08:16:58.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:16:58.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:16:58.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:58 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1795962205' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:16:58.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/327501287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]: dispatch 2024-04-24T08:16:58.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]: dispatch 2024-04-24T08:16:58.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b060a8df-b343-438f-b840-fd635ac58da5"}]': finished 2024-04-24T08:16:58.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:16:58.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:16:58.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1795962205' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:16:59.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:16:59 smithi149 ceph-mon[31282]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2024-04-24T08:16:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:16:59 smithi090 ceph-mon[25016]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2024-04-24T08:17:01.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:01 smithi090 ceph-mon[25016]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 993 B/s rd, 56 KiB/s wr, 3 op/s 2024-04-24T08:17:01.403 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:01 smithi149 ceph-mon[31282]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 993 B/s rd, 56 KiB/s wr, 3 op/s 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:03.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-24T08:17:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:03.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:05 smithi149 ceph-mon[31282]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-24T08:17:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:05 smithi090 ceph-mon[25016]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-24T08:17:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:06 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:17:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:06 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:06 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:17:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:06 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:07 smithi149 ceph-mon[31282]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T08:17:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:07 smithi149 ceph-mon[31282]: Deploying daemon osd.3 on smithi149 2024-04-24T08:17:07.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:07 smithi090 ceph-mon[25016]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T08:17:07.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:07 smithi090 ceph-mon[25016]: Deploying daemon osd.3 on smithi149 2024-04-24T08:17:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:09 smithi149 ceph-mon[31282]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-04-24T08:17:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:09 smithi090 ceph-mon[25016]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-04-24T08:17:11.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:11 smithi149 ceph-mon[31282]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T08:17:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:11 smithi090 ceph-mon[25016]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T08:17:13.522 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:13 smithi149 ceph-mon[31282]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T08:17:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:13 smithi090 ceph-mon[25016]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T08:17:15.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:15 smithi149 ceph-mon[31282]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:15.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:15 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:15.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:15 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:15.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:15 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:15 smithi090 ceph-mon[25016]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:15 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:15 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:15 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:17.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:17 smithi149 ceph-mon[31282]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:17 smithi090 ceph-mon[25016]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:18 smithi149 ceph-mon[31282]: from='osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:17:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:18 smithi149 ceph-mon[31282]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:17:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:18 smithi090 ceph-mon[25016]: from='osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:17:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:18 smithi090 ceph-mon[25016]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:17:19.020 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.019+0000 7f7b9e000700 1 -- 172.21.15.149:0/3131714241 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7ba0061190 con 0x7f7b8806c680 2024-04-24T08:17:19.021 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 3 on host 'smithi149' 2024-04-24T08:17:19.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.023+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 msgr2=0x7f7b8806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:19.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.023+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 0x7f7b8806eb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7b94003f90 tx=0x7f7b94009180 comp rx=0 tx=0).stop 2024-04-24T08:17:19.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 msgr2=0x7f7ba0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:19.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005b40 tx=0x7f7b8c00bd70 comp rx=0 tx=0).stop 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 shutdown_connections 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7b8806c680 0x7f7b8806eb30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0100020 0x7f7ba0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 --2- 172.21.15.149:0/3131714241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ba01012c0 0x7f7ba0197f60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 >> 172.21.15.149:0/3131714241 conn(0x7f7ba00fb650 msgr2=0x7f7ba01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 shutdown_connections 2024-04-24T08:17:19.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:19.024+0000 7f7ba6e00700 1 -- 172.21.15.149:0/3131714241 wait complete. 2024-04-24T08:17:19.487 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:19.488 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:19 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:19 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:19.830 DEBUG:teuthology.orchestra.run.smithi149:osd.3> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T08:17:19.835 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-04-24T08:17:19.835 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:17:20.262 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:17:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:17:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: osdmap e26: 4 total, 3 up, 4 in 2024-04-24T08:17:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:20.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:20.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: osdmap e26: 4 total, 3 up, 4 in 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] boot 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:21 smithi149 ceph-mon[31282]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: osd.3 [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] boot 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:17:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:21 smithi090 ceph-mon[25016]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0321196 s, 326 MB/s 2024-04-24T08:17:23.195 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T08:17:23.232 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:23 smithi149 ceph-mon[31282]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:23.232 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:23 smithi149 ceph-mon[31282]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:17:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:23 smithi090 ceph-mon[25016]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:23 smithi090 ceph-mon[25016]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:17:24.274 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-04-24T08:17:24.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:24 smithi149 ceph-mon[31282]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T08:17:24.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:24 smithi090 ceph-mon[25016]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T08:17:24.909 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:17:25.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:25 smithi149 ceph-mon[31282]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:25.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:25 smithi149 ceph-mon[31282]: osdmap e31: 4 total, 4 up, 4 in 2024-04-24T08:17:25.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:25 smithi090 ceph-mon[25016]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:25.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:25 smithi090 ceph-mon[25016]: osdmap e31: 4 total, 4 up, 4 in 2024-04-24T08:17:27.394 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T08:17:27.394 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi149 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi149 to 3641M 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:27.395 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:27 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 -- 172.21.15.149:0/3692867681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c107a60 msgr2=0x7f234c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 --2- 172.21.15.149:0/3692867681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c107a60 0x7f234c107eb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2338009910 tx=0x7f2338009c20 comp rx=0 tx=0).stop 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 -- 172.21.15.149:0/3692867681 shutdown_connections 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 --2- 172.21.15.149:0/3692867681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c107a60 0x7f234c107eb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 --2- 172.21.15.149:0/3692867681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c106860 0x7f234c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 -- 172.21.15.149:0/3692867681 >> 172.21.15.149:0/3692867681 conn(0x7f234c0758b0 msgr2=0x7f234c077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 -- 172.21.15.149:0/3692867681 shutdown_connections 2024-04-24T08:17:27.686 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.684+0000 7f2354400700 1 -- 172.21.15.149:0/3692867681 wait complete. 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.685+0000 7f2354400700 1 Processor -- start 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2354400700 1 -- start start 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2354400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2354400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c107a60 0x7f234c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2354400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c19cc10 con 0x7f234c107a60 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2354400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c19cd50 con 0x7f234c106860 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2351a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:17:27.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2351a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:36076/0 (socket says 172.21.15.149:36076) 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.686+0000 7f2351a00700 1 -- 172.21.15.149:0/586082424 learned_addr learned my addr 172.21.15.149:0/586082424 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2351000700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c107a60 0x7f234c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2351a00700 1 -- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c107a60 msgr2=0x7f234c19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2351a00700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c107a60 0x7f234c19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2351a00700 1 -- 172.21.15.149:0/586082424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380095f0 con 0x7f234c106860 2024-04-24T08:17:27.689 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2351a00700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f234000d900 tx=0x7f234000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:17:27.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23400041f0 con 0x7f234c106860 2024-04-24T08:17:27.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2340004350 con 0x7f234c106860 2024-04-24T08:17:27.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.687+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c1a1800 con 0x7f234c106860 2024-04-24T08:17:27.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.688+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2340003da0 con 0x7f234c106860 2024-04-24T08:17:27.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.688+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c1a1d50 con 0x7f234c106860 2024-04-24T08:17:27.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.689+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c10f3b0 con 0x7f234c106860 2024-04-24T08:17:27.692 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.690+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2340009730 con 0x7f234c106860 2024-04-24T08:17:27.692 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.691+0000 7f234a000700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 0x7f233406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:17:27.692 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.691+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f234008b340 con 0x7f234c106860 2024-04-24T08:17:27.692 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.692+0000 7f2351000700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 0x7f233406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:17:27.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.693+0000 7f2351000700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 0x7f233406eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2338006010 tx=0x7f233800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:17:27.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.696+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2340059b30 con 0x7f234c106860 2024-04-24T08:17:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T08:17:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi149 2024-04-24T08:17:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi149 to 3641M 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:17:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:27 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:27.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:27.923+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f234c10c3b0 con 0x7f233406c680 2024-04-24T08:17:28.738 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:28 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:17:28.739 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:28 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:17:28.739 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:28 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:28.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:28 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:17:28.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:28 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:17:28.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:28 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:29.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:29 smithi149 ceph-mon[31282]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:17:29.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:29 smithi149 ceph-mon[31282]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:17:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:29 smithi090 ceph-mon[25016]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:17:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:29 smithi090 ceph-mon[25016]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:17:31.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:31 smithi149 ceph-mon[31282]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T08:17:31.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:31 smithi090 ceph-mon[25016]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T08:17:32.679 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:32 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1940945986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]: dispatch 2024-04-24T08:17:32.679 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:32 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]: dispatch 2024-04-24T08:17:32.680 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:32 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]': finished 2024-04-24T08:17:32.680 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:32 smithi149 ceph-mon[31282]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:17:32.680 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:32 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:32 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1940945986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]: dispatch 2024-04-24T08:17:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:32 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]: dispatch 2024-04-24T08:17:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:32 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6abf1d17-2210-4a69-b3d9-bef2daf51ae8"}]': finished 2024-04-24T08:17:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:32 smithi090 ceph-mon[25016]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:17:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:32 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:33.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:33 smithi090 ceph-mon[25016]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T08:17:33.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:33 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1405561696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:17:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:33 smithi149 ceph-mon[31282]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T08:17:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:33 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1405561696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:17:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:35 smithi149 ceph-mon[31282]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T08:17:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:35 smithi090 ceph-mon[25016]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T08:17:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:37.682 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:37 smithi149 ceph-mon[31282]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:37 smithi090 ceph-mon[25016]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:39 smithi149 ceph-mon[31282]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:39 smithi090 ceph-mon[25016]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:41.696 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:41 smithi149 ceph-mon[31282]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:41.697 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:17:41.697 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:41 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:41 smithi090 ceph-mon[25016]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:17:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:41 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:17:42.640 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:42 smithi149 ceph-mon[31282]: Deploying daemon osd.4 on smithi149 2024-04-24T08:17:42.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:42 smithi090 ceph-mon[25016]: Deploying daemon osd.4 on smithi149 2024-04-24T08:17:43.713 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:43 smithi149 ceph-mon[31282]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:43 smithi090 ceph-mon[25016]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:45.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:45 smithi149 ceph-mon[31282]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:45.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:45 smithi090 ceph-mon[25016]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:47.667 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:47 smithi149 ceph-mon[31282]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:47.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:47 smithi090 ceph-mon[25016]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:49 smithi149 ceph-mon[31282]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:49 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:49 smithi090 ceph-mon[25016]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:49 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:17:50.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:50.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:17:51.606 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:51 smithi149 ceph-mon[31282]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:51.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:51 smithi090 ceph-mon[25016]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:53 smithi149 ceph-mon[31282]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:53 smithi149 ceph-mon[31282]: from='osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:17:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:53 smithi149 ceph-mon[31282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:17:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:53 smithi090 ceph-mon[25016]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:53 smithi090 ceph-mon[25016]: from='osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:17:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:53 smithi090 ceph-mon[25016]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:17:53.581 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.580+0000 7f234a000700 1 -- 172.21.15.149:0/586082424 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f234c10c3b0 con 0x7f233406c680 2024-04-24T08:17:53.582 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-04-24T08:17:53.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.583+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 msgr2=0x7f233406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:53.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.583+0000 7f2354400700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 0x7f233406eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2338006010 tx=0x7f233800b540 comp rx=0 tx=0).stop 2024-04-24T08:17:53.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 msgr2=0x7f234c19c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:17:53.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f234000d900 tx=0x7f234000dcc0 comp rx=0 tx=0).stop 2024-04-24T08:17:53.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 shutdown_connections 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f233406c680 0x7f233406eb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c106860 0x7f234c19c0b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 --2- 172.21.15.149:0/586082424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234c107a60 0x7f234c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 >> 172.21.15.149:0/586082424 conn(0x7f234c0758b0 msgr2=0x7f234c10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 shutdown_connections 2024-04-24T08:17:53.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:17:53.584+0000 7f2354400700 1 -- 172.21.15.149:0/586082424 wait complete. 2024-04-24T08:17:54.359 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T08:17:54.365 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_2... 2024-04-24T08:17:54.365 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.777 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[42011]: 2024-04-24T08:17:54.532+0000 7f5d52c00700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T08:17:54.778 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:17:54 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[42011]: 2024-04-24T08:17:54.549+0000 7f5d4bc00700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:17:54.779 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:54 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:17:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:55 smithi090 ceph-mon[25016]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:55 smithi090 ceph-mon[25016]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:17:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:55 smithi090 ceph-mon[25016]: osdmap e34: 5 total, 4 up, 5 in 2024-04-24T08:17:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:55 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:55 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:55.970 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:55 smithi149 ceph-mon[31282]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-24T08:17:55.970 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:55 smithi149 ceph-mon[31282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:17:55.970 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:55 smithi149 ceph-mon[31282]: osdmap e34: 5 total, 4 up, 5 in 2024-04-24T08:17:55.971 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:55 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:55.971 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:55 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:56.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:56 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:17:56.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:56 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:17:56.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:56 smithi149 ceph-mon[31282]: osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] boot 2024-04-24T08:17:56.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:56 smithi149 ceph-mon[31282]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T08:17:56.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:56 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:56.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:56 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:17:56.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:56 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:17:56.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:56 smithi090 ceph-mon[25016]: osd.4 [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] boot 2024-04-24T08:17:56.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:56 smithi090 ceph-mon[25016]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T08:17:56.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:56 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:17:57.617 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:57 smithi149 ceph-mon[31282]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:17:57.617 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:57 smithi149 ceph-mon[31282]: osdmap e36: 5 total, 5 up, 5 in 2024-04-24T08:17:58.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:57 smithi090 ceph-mon[25016]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:17:58.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:57 smithi090 ceph-mon[25016]: osdmap e36: 5 total, 5 up, 5 in 2024-04-24T08:17:58.048 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:17:58.048 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:17:58.048 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T08:17:58.049 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T08:17:58.049 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324289 s, 323 MB/s 2024-04-24T08:17:58.049 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2024-04-24T08:17:58.049 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T08:17:59.208 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-04-24T08:17:59.829 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:17:59.894 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:17:59 smithi149 ceph-mon[31282]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:17:59 smithi090 ceph-mon[25016]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi149 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:01.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi149 to 1820M 2024-04-24T08:18:01.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:01.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:01.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:01 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi149 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi149 to 1820M 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:01.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:01 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:02.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 -- 172.21.15.149:0/2628590256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4100d60 msgr2=0x7f96e4103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:02.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2628590256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4100d60 0x7f96e4103140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009910 tx=0x7f96d4009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:02.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 -- 172.21.15.149:0/2628590256 shutdown_connections 2024-04-24T08:18:02.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2628590256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4103680 0x7f96e4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:02.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2628590256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4100d60 0x7f96e4103140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:02.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.557+0000 7f96eaa00700 1 -- 172.21.15.149:0/2628590256 >> 172.21.15.149:0/2628590256 conn(0x7f96e40fa780 msgr2=0x7f96e40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:02.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.558+0000 7f96eaa00700 1 -- 172.21.15.149:0/2628590256 shutdown_connections 2024-04-24T08:18:02.560 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:02 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:02.560 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:02 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:02.560 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:02 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:02.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.558+0000 7f96eaa00700 1 -- 172.21.15.149:0/2628590256 wait complete. 2024-04-24T08:18:02.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 Processor -- start 2024-04-24T08:18:02.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 -- start start 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4100d60 0x7f96e4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e4198890 con 0x7f96e4100d60 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96eaa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e41989d0 con 0x7f96e4103680 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.560+0000 7f96e3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:02.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:37614/0 (socket says 172.21.15.149:37614) 2024-04-24T08:18:02.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 -- 172.21.15.149:0/2962163703 learned_addr learned my addr 172.21.15.149:0/2962163703 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:18:02.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3e00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4100d60 0x7f96e4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:02.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 -- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4100d60 msgr2=0x7f96e4197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:02.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4100d60 0x7f96e4197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:02.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 -- 172.21.15.149:0/2962163703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40095f0 con 0x7f96e4103680 2024-04-24T08:18:02.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.561+0000 7f96e3400700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b4b0 tx=0x7f96d000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:02.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.562+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96d0011800 con 0x7f96e4103680 2024-04-24T08:18:02.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.562+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96d0011e10 con 0x7f96e4103680 2024-04-24T08:18:02.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.562+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96d000f2e0 con 0x7f96e4103680 2024-04-24T08:18:02.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.562+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e419d480 con 0x7f96e4103680 2024-04-24T08:18:02.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.562+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e419d9d0 con 0x7f96e4103680 2024-04-24T08:18:02.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.563+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e40fc340 con 0x7f96e4103680 2024-04-24T08:18:02.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.564+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f96d0010380 con 0x7f96e4103680 2024-04-24T08:18:02.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.565+0000 7f96e0c00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 0x7f96c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:02.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.565+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f96d008aee0 con 0x7f96e4103680 2024-04-24T08:18:02.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.566+0000 7f96e3e00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 0x7f96c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:02.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.567+0000 7f96e3e00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 0x7f96c806eb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:02.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.569+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f96d0059670 con 0x7f96e4103680 2024-04-24T08:18:02.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:02 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:02.739 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:02 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:02.739 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:02 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:02.797 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:02.795+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f96e4061190 con 0x7f96c806c680 2024-04-24T08:18:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:03 smithi149 ceph-mon[31282]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:18:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:18:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:03 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:03 smithi090 ceph-mon[25016]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:18:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:18:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:03 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:04.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:04 smithi149 ceph-mon[31282]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:18:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:04 smithi090 ceph-mon[25016]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:18:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:05 smithi149 ceph-mon[31282]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:05 smithi090 ceph-mon[25016]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/282383943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]: dispatch 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]: dispatch 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]': finished 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:07 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/2472391951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/282383943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]: dispatch 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]: dispatch 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "729bbd1d-e052-4a84-837c-8b19728361f9"}]': finished 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:07 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/2472391951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:18:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:09 smithi149 ceph-mon[31282]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:09 smithi090 ceph-mon[25016]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:11.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:11 smithi149 ceph-mon[31282]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:11 smithi090 ceph-mon[25016]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:13.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:13 smithi149 ceph-mon[31282]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:13 smithi090 ceph-mon[25016]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:15.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:15 smithi149 ceph-mon[31282]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:15.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:15 smithi090 ceph-mon[25016]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:17.667 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:17 smithi149 ceph-mon[31282]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:17.667 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:17 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:18:17.667 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:17 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:17 smithi090 ceph-mon[25016]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:17 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:18:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:17 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:18.611 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:18 smithi149 ceph-mon[31282]: Deploying daemon osd.5 on smithi149 2024-04-24T08:18:18.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:18 smithi090 ceph-mon[25016]: Deploying daemon osd.5 on smithi149 2024-04-24T08:18:19.766 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:19 smithi149 ceph-mon[31282]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:19 smithi090 ceph-mon[25016]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:20.968 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:21.657 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:21 smithi149 ceph-mon[31282]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:21 smithi090 ceph-mon[25016]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:23.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:23 smithi090 ceph-mon[25016]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:23 smithi149 ceph-mon[31282]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:25.736 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:25 smithi149 ceph-mon[31282]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:25.736 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:25 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:25.736 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:25 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:25.736 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:25 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:25 smithi090 ceph-mon[25016]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:25 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:25 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:25 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:27.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:27 smithi149 ceph-mon[31282]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:28.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:27 smithi090 ceph-mon[25016]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:28.771 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:28 smithi149 ceph-mon[31282]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:28.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:28 smithi090 ceph-mon[25016]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:29.968 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:29 smithi149 ceph-mon[31282]: from='osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:18:29.968 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:29 smithi149 ceph-mon[31282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:18:30.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:29 smithi090 ceph-mon[25016]: from='osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:18:30.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:29 smithi090 ceph-mon[25016]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:18:30.070 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.069+0000 7f96e0c00700 1 -- 172.21.15.149:0/2962163703 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f96e4061190 con 0x7f96c806c680 2024-04-24T08:18:30.070 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-04-24T08:18:30.074 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 msgr2=0x7f96c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 0x7f96c806eb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d400b540 comp rx=0 tx=0).stop 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 msgr2=0x7f96e4198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b4b0 tx=0x7f96d000b870 comp rx=0 tx=0).stop 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 shutdown_connections 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f96c806c680 0x7f96c806eb30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4100d60 0x7f96e4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 --2- 172.21.15.149:0/2962163703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e4103680 0x7f96e4198270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 >> 172.21.15.149:0/2962163703 conn(0x7f96e40fa780 msgr2=0x7f96e4100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 shutdown_connections 2024-04-24T08:18:30.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:30.073+0000 7f96eaa00700 1 -- 172.21.15.149:0/2962163703 wait complete. 2024-04-24T08:18:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:18:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:30.848 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T08:18:30.855 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-24T08:18:30.855 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T08:18:30.927 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:18:30.927 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T08:18:30.927 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:30.927 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:18:30.927 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:18:30.928 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:18:30.928 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:30.928 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:30.928 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:31.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:31 smithi090 ceph-mon[25016]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:18:31.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:31 smithi090 ceph-mon[25016]: osdmap e39: 6 total, 5 up, 6 in 2024-04-24T08:18:31.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:31.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:31.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:31 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:31.864 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:31 smithi149 ceph-mon[31282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T08:18:31.864 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:31 smithi149 ceph-mon[31282]: osdmap e39: 6 total, 5 up, 6 in 2024-04-24T08:18:31.864 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:31.864 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:31.864 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:31 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: purged_snaps scrub starts 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: purged_snaps scrub ok 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] boot 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:32 smithi090 ceph-mon[25016]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:33.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: purged_snaps scrub starts 2024-04-24T08:18:33.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: purged_snaps scrub ok 2024-04-24T08:18:33.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: osd.5 [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] boot 2024-04-24T08:18:33.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:18:33.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:18:33.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:32 smithi149 ceph-mon[31282]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 -- 172.21.15.90:0/2686326126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 msgr2=0x7f7784105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 --2- 172.21.15.90:0/2686326126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784105020 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7774009910 tx=0x7f7774009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 -- 172.21.15.90:0/2686326126 shutdown_connections 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 --2- 172.21.15.90:0/2686326126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7784105560 0x7f7784107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 --2- 172.21.15.90:0/2686326126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784105020 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 -- 172.21.15.90:0/2686326126 >> 172.21.15.90:0/2686326126 conn(0x7f77840fa5d0 msgr2=0x7f77840fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 -- 172.21.15.90:0/2686326126 shutdown_connections 2024-04-24T08:18:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.375+0000 7f7783e00700 1 -- 172.21.15.90:0/2686326126 wait complete. 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.376+0000 7f7783e00700 1 Processor -- start 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.376+0000 7f7783e00700 1 -- start start 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7783e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7783e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7784105560 0x7f7784198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7783e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7784198620 con 0x7f7784069ae0 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7783e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7784198760 con 0x7f7784105560 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39122/0 (socket says 172.21.15.90:39122) 2024-04-24T08:18:33.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 -- 172.21.15.90:0/2593696709 learned_addr learned my addr 172.21.15.90:0/2593696709 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 -- 172.21.15.90:0/2593696709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7784105560 msgr2=0x7f7784198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7784105560 0x7f7784198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.377+0000 7f7782a00700 1 -- 172.21.15.90:0/2593696709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740095f0 con 0x7f7784069ae0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f7782a00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7774000c00 tx=0x7f777400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777401d070 con 0x7f7784069ae0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778419d160 con 0x7f7784069ae0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f777400f460 con 0x7f7784069ae0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77740175f0 con 0x7f7784069ae0 2024-04-24T08:18:33.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.378+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778419d650 con 0x7f7784069ae0 2024-04-24T08:18:33.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.379+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784191c00 con 0x7f7784069ae0 2024-04-24T08:18:33.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.381+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f777400f5d0 con 0x7f7784069ae0 2024-04-24T08:18:33.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.382+0000 7f777b400700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 0x7f776806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:33.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.382+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f777408ce40 con 0x7f7784069ae0 2024-04-24T08:18:33.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.383+0000 7f7782000700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 0x7f776806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:33.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.383+0000 7f7782000700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 0x7f776806eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f777000a990 tx=0x7f7770005c80 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:33.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.386+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f777405b350 con 0x7f7784069ae0 2024-04-24T08:18:33.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.603+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f778402cee0 con 0x7f7784069ae0 2024-04-24T08:18:33.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.604+0000 7f777b400700 1 -- 172.21.15.90:0/2593696709 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f777405aee0 con 0x7f7784069ae0 2024-04-24T08:18:33.605 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:18:33.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 msgr2=0x7f776806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:33.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 0x7f776806eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f777000a990 tx=0x7f7770005c80 comp rx=0 tx=0).stop 2024-04-24T08:18:33.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 msgr2=0x7f7784197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:33.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7774000c00 tx=0x7f777400be20 comp rx=0 tx=0).stop 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 shutdown_connections 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f776806c680 0x7f776806eb30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7784069ae0 0x7f7784197ac0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 --2- 172.21.15.90:0/2593696709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7784105560 0x7f7784198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 >> 172.21.15.90:0/2593696709 conn(0x7f77840fa5d0 msgr2=0x7f77840fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.607+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 shutdown_connections 2024-04-24T08:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:33.608+0000 7f7783e00700 1 -- 172.21.15.90:0/2593696709 wait complete. 2024-04-24T08:18:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:33 smithi149 ceph-mon[31282]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:18:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:33 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2593696709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T08:18:33.929 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:33 smithi090 ceph-mon[25016]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:18:33.929 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:33 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2593696709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T08:18:34.351 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1713946711,"num_in_osds":6,"osd_in_since":1713946686,"num_remapped_pgs":0} 2024-04-24T08:18:34.352 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:18:34.735 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:34.882 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:34 smithi090 ceph-mon[25016]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:35.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:34 smithi149 ceph-mon[31282]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: Detected new or changed devices on smithi149 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi149 to 1213M 2024-04-24T08:18:35.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:35.898 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:35.898 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:35.898 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi149 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:36.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi149 to 1213M 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:36.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:36.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 -- 172.21.15.90:0/2611631885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c106b80 msgr2=0x7f8f0c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:36.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 --2- 172.21.15.90:0/2611631885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c106b80 0x7f8f0c106ff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009910 tx=0x7f8ef8009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 -- 172.21.15.90:0/2611631885 shutdown_connections 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 --2- 172.21.15.90:0/2611631885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c106b80 0x7f8f0c106ff0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 --2- 172.21.15.90:0/2611631885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c10c660 0x7f8f0c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.780+0000 7f8f12400700 1 -- 172.21.15.90:0/2611631885 >> 172.21.15.90:0/2611631885 conn(0x7f8f0c075e90 msgr2=0x7f8f0c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.781+0000 7f8f12400700 1 -- 172.21.15.90:0/2611631885 shutdown_connections 2024-04-24T08:18:36.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.781+0000 7f8f12400700 1 -- 172.21.15.90:0/2611631885 wait complete. 2024-04-24T08:18:36.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.782+0000 7f8f12400700 1 Processor -- start 2024-04-24T08:18:36.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f12400700 1 -- start start 2024-04-24T08:18:36.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f12400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c106b80 0x7f8f0c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:36.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f12400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:36.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f12400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19ce00 con 0x7f8f0c10c660 2024-04-24T08:18:36.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f12400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1a0b90 con 0x7f8f0c106b80 2024-04-24T08:18:36.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f0be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:36.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f0be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39154/0 (socket says 172.21.15.90:39154) 2024-04-24T08:18:36.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.783+0000 7f8f0be00700 1 -- 172.21.15.90:0/2929583757 learned_addr learned my addr 172.21.15.90:0/2929583757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:36.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f11000700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c106b80 0x7f8f0c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:36.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f0be00700 1 -- 172.21.15.90:0/2929583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c106b80 msgr2=0x7f8f0c19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f0be00700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c106b80 0x7f8f0c19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f0be00700 1 -- 172.21.15.90:0/2929583757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80095f0 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f0be00700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef801d070 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ef8022470 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c1a0e10 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.784+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef800f670 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.785+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c1a1300 con 0x7f8f0c10c660 2024-04-24T08:18:36.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.786+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c10eb40 con 0x7f8f0c10c660 2024-04-24T08:18:36.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.787+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8ef8004c30 con 0x7f8f0c10c660 2024-04-24T08:18:36.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.788+0000 7f8f09600700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 0x7f8ef406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:36.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.788+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8ef808cb70 con 0x7f8f0c10c660 2024-04-24T08:18:36.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.788+0000 7f8f11000700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 0x7f8ef406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:36.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.789+0000 7f8f11000700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 0x7f8ef406eb30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005fd0 tx=0x7f8f0000a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:36.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:36.793+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8ef8057640 con 0x7f8f0c10c660 2024-04-24T08:18:37.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.007+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0c10e520 con 0x7f8f0c10c660 2024-04-24T08:18:37.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:37.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:37.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:37.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:37.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:36 smithi090 ceph-mon[25016]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:37.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.008+0000 7f8f09600700 1 -- 172.21.15.90:0/2929583757 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11312 (secure 0 0 0) 0x7f8ef805ac60 con 0x7f8f0c10c660 2024-04-24T08:18:37.010 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:18:37.010 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:18:32.600771+0000","last_up_change":"2024-04-24T08:18:31.586535+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6803","nonce":3550641031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6805","nonce":3550641031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6809","nonce":3550641031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6807","nonce":3550641031}]},"public_addr":"172.21.15.90:6803/3550641031","cluster_addr":"172.21.15.90:6805/3550641031","heartbeat_back_addr":"172.21.15.90:6809/3550641031","heartbeat_front_addr":"172.21.15.90:6807/3550641031","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6811","nonce":2560478642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6813","nonce":2560478642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6817","nonce":2560478642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6815","nonce":2560478642}]},"public_addr":"172.21.15.90:6811/2560478642","cluster_addr":"172.21.15.90:6813/2560478642","heartbeat_back_addr":"172.21.15.90:6817/2560478642","heartbeat_front_addr":"172.21.15.90:6815/2560478642","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6819","nonce":358354662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6821","nonce":358354662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6825","nonce":358354662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6823","nonce":358354662}]},"public_addr":"172.21.15.90:6819/358354662","cluster_addr":"172.21.15.90:6821/358354662","heartbeat_back_addr":"172.21.15.90:6825/358354662","heartbeat_front_addr":"172.21.15.90:6823/358354662","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6801","nonce":1453378074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6803","nonce":1453378074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6807","nonce":1453378074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6805","nonce":1453378074}]},"public_addr":"172.21.15.149:6801/1453378074","cluster_addr":"172.21.15.149:6803/1453378074","heartbeat_back_addr":"172.21.15.149:6807/1453378074","heartbeat_front_addr":"172.21.15.149:6805/1453378074","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6809","nonce":197987858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6811","nonce":197987858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6815","nonce":197987858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6813","nonce":197987858}]},"public_addr":"172.21.15.149:6809/197987858","cluster_addr":"172.21.15.149:6811/197987858","heartbeat_back_addr":"172.21.15.149:6815/197987858","heartbeat_front_addr":"172.21.15.149:6813/197987858","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6817","nonce":142618710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6819","nonce":142618710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6823","nonce":142618710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6821","nonce":142618710}]},"public_addr":"172.21.15.149:6817/142618710","cluster_addr":"172.21.15.149:6819/142618710","heartbeat_back_addr":"172.21.15.149:6823/142618710","heartbeat_front_addr":"172.21.15.149:6821/142618710","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 msgr2=0x7f8ef406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 0x7f8ef406eb30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005fd0 tx=0x7f8f0000a560 comp rx=0 tx=0).stop 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 msgr2=0x7f8f0c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef800be00 comp rx=0 tx=0).stop 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 shutdown_connections 2024-04-24T08:18:37.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8ef406c680 0x7f8ef406eb30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f0c106b80 0x7f8f0c19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 --2- 172.21.15.90:0/2929583757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f0c10c660 0x7f8f0c19c720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 >> 172.21.15.90:0/2929583757 conn(0x7f8f0c075e90 msgr2=0x7f8f0c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 shutdown_connections 2024-04-24T08:18:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:37.012+0000 7f8f12400700 1 -- 172.21.15.90:0/2929583757 wait complete. 2024-04-24T08:18:37.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:18:37.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:18:37.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:18:37.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:18:37.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:36 smithi149 ceph-mon[31282]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:37.848 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:16:48.618419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:18:37.848 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T08:18:37.930 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:37 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2929583757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:38.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:37 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2929583757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:38.254 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:39.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:38 smithi149 ceph-mon[31282]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:39.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:38 smithi090 ceph-mon[25016]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 -- 172.21.15.90:0/1081018859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 msgr2=0x7efc1810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 --2- 172.21.15.90:0/1081018859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1810ccc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009910 tx=0x7efc0c009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 -- 172.21.15.90:0/1081018859 shutdown_connections 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 --2- 172.21.15.90:0/1081018859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 0x7efc18106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 --2- 172.21.15.90:0/1081018859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1810ccc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.364+0000 7efc1f800700 1 -- 172.21.15.90:0/1081018859 >> 172.21.15.90:0/1081018859 conn(0x7efc18076060 msgr2=0x7efc18078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.365+0000 7efc1f800700 1 -- 172.21.15.90:0/1081018859 shutdown_connections 2024-04-24T08:18:40.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.365+0000 7efc1f800700 1 -- 172.21.15.90:0/1081018859 wait complete. 2024-04-24T08:18:40.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.366+0000 7efc1f800700 1 Processor -- start 2024-04-24T08:18:40.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.366+0000 7efc1f800700 1 -- start start 2024-04-24T08:18:40.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc1f800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 0x7efc1819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:40.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc1f800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:40.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc1f800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1819d090 con 0x7efc1810c8f0 2024-04-24T08:18:40.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc1f800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1819d1d0 con 0x7efc18106b80 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33422/0 (socket says 172.21.15.90:33422) 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 -- 172.21.15.90:0/3645458675 learned_addr learned my addr 172.21.15.90:0/3645458675 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc1ce00700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 0x7efc1819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 -- 172.21.15.90:0/3645458675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 msgr2=0x7efc1819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 0x7efc1819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.367+0000 7efc17e00700 1 -- 172.21.15.90:0/3645458675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c0095f0 con 0x7efc1810c8f0 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc17e00700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc0400c930 tx=0x7efc0400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:40.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc04007ad0 con 0x7efc1810c8f0 2024-04-24T08:18:40.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc04007c30 con 0x7efc1810c8f0 2024-04-24T08:18:40.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc04018630 con 0x7efc1810c8f0 2024-04-24T08:18:40.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc181a1020 con 0x7efc1810c8f0 2024-04-24T08:18:40.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.368+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc181a1520 con 0x7efc1810c8f0 2024-04-24T08:18:40.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.369+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18109b20 con 0x7efc1810c8f0 2024-04-24T08:18:40.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.371+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7efc0400f450 con 0x7efc1810c8f0 2024-04-24T08:18:40.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.371+0000 7efc15600700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 0x7efc0006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:40.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.372+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efc0408b710 con 0x7efc1810c8f0 2024-04-24T08:18:40.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.372+0000 7efc1ce00700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 0x7efc0006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.372+0000 7efc1ce00700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 0x7efc0006eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006010 tx=0x7efc0c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:40.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.376+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efc04056110 con 0x7efc1810c8f0 2024-04-24T08:18:40.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.589+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc1802cc30 con 0x7efc1810c8f0 2024-04-24T08:18:40.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.590+0000 7efc15600700 1 -- 172.21.15.90:0/3645458675 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc04059730 con 0x7efc1810c8f0 2024-04-24T08:18:40.591 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T08:18:40.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 msgr2=0x7efc0006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 0x7efc0006eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006010 tx=0x7efc0c01a040 comp rx=0 tx=0).stop 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 msgr2=0x7efc1819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc0400c930 tx=0x7efc0400ccf0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 shutdown_connections 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efc0006c680 0x7efc0006eb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18106b80 0x7efc1819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.593+0000 7efc1f800700 1 --2- 172.21.15.90:0/3645458675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc1810c8f0 0x7efc1819ca40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.594+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 >> 172.21.15.90:0/3645458675 conn(0x7efc18076060 msgr2=0x7efc18077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.594+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 shutdown_connections 2024-04-24T08:18:40.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:40.594+0000 7efc1f800700 1 -- 172.21.15.90:0/3645458675 wait complete. 2024-04-24T08:18:41.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:41 smithi090 ceph-mon[25016]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:41.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:41 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3645458675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:18:41.379 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T08:18:41.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T08:18:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:41 smithi149 ceph-mon[31282]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:41 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3645458675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:18:41.793 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:43.510 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:43 smithi090 ceph-mon[25016]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:43 smithi149 ceph-mon[31282]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.828+0000 7f28c2400700 1 -- 172.21.15.90:0/3869217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 msgr2=0x7f28bc107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 --2- 172.21.15.90:0/3869217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc107e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009910 tx=0x7f28a4009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 -- 172.21.15.90:0/3869217227 shutdown_connections 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 --2- 172.21.15.90:0/3869217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc107e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 --2- 172.21.15.90:0/3869217227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28bc106840 0x7f28bc106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:43.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 -- 172.21.15.90:0/3869217227 >> 172.21.15.90:0/3869217227 conn(0x7f28bc075970 msgr2=0x7f28bc077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:43.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 -- 172.21.15.90:0/3869217227 shutdown_connections 2024-04-24T08:18:43.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.829+0000 7f28c2400700 1 -- 172.21.15.90:0/3869217227 wait complete. 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 Processor -- start 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 -- start start 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28bc106840 0x7f28bc19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc19cc40 con 0x7f28bc107a40 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28c2400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc19cd80 con 0x7f28bc106840 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.830+0000 7f28baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33450/0 (socket says 172.21.15.90:33450) 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 -- 172.21.15.90:0/3259069985 learned_addr learned my addr 172.21.15.90:0/3259069985 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 -- 172.21.15.90:0/3259069985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28bc106840 msgr2=0x7f28bc19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:43.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28bc106840 0x7f28bc19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 -- 172.21.15.90:0/3259069985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28bc107a40 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28baa00700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005b40 tx=0x7f28a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a401d070 con 0x7f28bc107a40 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f28a4022470 con 0x7f28bc107a40 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.831+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a400f690 con 0x7f28bc107a40 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.832+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc1a17d0 con 0x7f28bc107a40 2024-04-24T08:18:43.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.832+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc1a1cc0 con 0x7f28bc107a40 2024-04-24T08:18:43.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.833+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc10f390 con 0x7f28bc107a40 2024-04-24T08:18:43.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.834+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f28a40225e0 con 0x7f28bc107a40 2024-04-24T08:18:43.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.835+0000 7f28b3e00700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 0x7f28a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:43.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.835+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f28a408cc90 con 0x7f28bc107a40 2024-04-24T08:18:43.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.835+0000 7f28bb400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 0x7f28a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:43.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.836+0000 7f28bb400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 0x7f28a006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28bc1078a0 tx=0x7f28ac005ca0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:43.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:43.840+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f28a405b1a0 con 0x7f28bc107a40 2024-04-24T08:18:44.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.142+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f28bc1a24b0 con 0x7f28bc107a40 2024-04-24T08:18:44.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.154+0000 7f28b3e00700 1 -- 172.21.15.90:0/3259069985 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f28a405ad30 con 0x7f28bc107a40 2024-04-24T08:18:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-24T08:18:44.155 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBkwChm/SOiCBAAH0uE6yh/OwC0XwZovpRdiw== 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 msgr2=0x7f28a006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 0x7f28a006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28bc1078a0 tx=0x7f28ac005ca0 comp rx=0 tx=0).stop 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 msgr2=0x7f28bc19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005b40 tx=0x7f28a400be20 comp rx=0 tx=0).stop 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 shutdown_connections 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f28a006c680 0x7f28a006eb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28bc106840 0x7f28bc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 --2- 172.21.15.90:0/3259069985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28bc107a40 0x7f28bc19c620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:44.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 >> 172.21.15.90:0/3259069985 conn(0x7f28bc075970 msgr2=0x7f28bc10ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:44.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 shutdown_connections 2024-04-24T08:18:44.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:44.157+0000 7f28c2400700 1 -- 172.21.15.90:0/3259069985 wait complete. 2024-04-24T08:18:44.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:44 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3259069985' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:44.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:44 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3259069985' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:18:44.515 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:44 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3259069985' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:44.515 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:44 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3259069985' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:18:44.884 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:18:44.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T08:18:44.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T08:18:45.013 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T08:18:45.416 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi149/config 2024-04-24T08:18:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:45 smithi149 ceph-mon[31282]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:45 smithi090 ceph-mon[25016]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:47.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:47 smithi149 ceph-mon[31282]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:47 smithi090 ceph-mon[25016]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:47.875 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 -- 172.21.15.149:0/524868053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf81036d0 msgr2=0x7febf8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:47.875 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 --2- 172.21.15.149:0/524868053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf81036d0 0x7febf8103b40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7febe4009910 tx=0x7febe4009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:47.875 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 -- 172.21.15.149:0/524868053 shutdown_connections 2024-04-24T08:18:47.875 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 --2- 172.21.15.149:0/524868053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf81036d0 0x7febf8103b40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:47.876 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 --2- 172.21.15.149:0/524868053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf8102430 0x7febf8102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:47.876 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 -- 172.21.15.149:0/524868053 >> 172.21.15.149:0/524868053 conn(0x7febf80fda80 msgr2=0x7febf80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:47.876 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 -- 172.21.15.149:0/524868053 shutdown_connections 2024-04-24T08:18:47.876 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.873+0000 7febff200700 1 -- 172.21.15.149:0/524868053 wait complete. 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 Processor -- start 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 -- start start 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 0x7febf8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8198890 con 0x7febf81036d0 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.875+0000 7febff200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81989d0 con 0x7febf8102430 2024-04-24T08:18:47.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:47.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:52612/0 (socket says 172.21.15.149:52612) 2024-04-24T08:18:47.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 -- 172.21.15.149:0/1114368283 learned_addr learned my addr 172.21.15.149:0/1114368283 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T08:18:47.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7400700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 0x7febf8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 -- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 msgr2=0x7febf8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 0x7febf8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.876+0000 7febf7e00700 1 -- 172.21.15.149:0/1114368283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40095f0 con 0x7febf8102430 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.877+0000 7febf7400700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 0x7febf8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.877+0000 7febf7e00700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febe800b4b0 tx=0x7febe800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.877+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe8011800 con 0x7febf8102430 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.877+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf819d480 con 0x7febf8102430 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.878+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7febe8011e10 con 0x7febf8102430 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.878+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe800f2e0 con 0x7febf8102430 2024-04-24T08:18:47.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.878+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf819d9d0 con 0x7febf8102430 2024-04-24T08:18:47.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.879+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf810af90 con 0x7febf8102430 2024-04-24T08:18:47.881 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.880+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7febe8010380 con 0x7febf8102430 2024-04-24T08:18:47.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.881+0000 7febf4c00700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 0x7febdc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:47.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.881+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7febe808b1b0 con 0x7febf8102430 2024-04-24T08:18:47.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.882+0000 7febf7400700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 0x7febdc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:47.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.883+0000 7febf7400700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 0x7febdc06eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febe40059d0 tx=0x7febe4005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:47.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:47.886+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7febe8059660 con 0x7febf8102430 2024-04-24T08:18:48.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.182+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7febf819e070 con 0x7febf8102430 2024-04-24T08:18:48.195 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.194+0000 7febf4c00700 1 -- 172.21.15.149:0/1114368283 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7febe80591f0 con 0x7febf8102430 2024-04-24T08:18:48.195 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-04-24T08:18:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQBowChm58oPCxAAOIgGmHrizOmMfV0EcAIRTQ== 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.197+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 msgr2=0x7febdc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.197+0000 7febff200700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 0x7febdc06eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febe40059d0 tx=0x7febe4005920 comp rx=0 tx=0).stop 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.197+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 msgr2=0x7febf8197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.197+0000 7febff200700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febe800b4b0 tx=0x7febe800b870 comp rx=0 tx=0).stop 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 shutdown_connections 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7febdc06c680 0x7febdc06eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf8102430 0x7febf8197d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 --2- 172.21.15.149:0/1114368283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febf81036d0 0x7febf8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:48.199 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 >> 172.21.15.149:0/1114368283 conn(0x7febf80fda80 msgr2=0x7febf8106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:48.200 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 shutdown_connections 2024-04-24T08:18:48.200 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:18:48.198+0000 7febff200700 1 -- 172.21.15.149:0/1114368283 wait complete. 2024-04-24T08:18:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:48 smithi149 ceph-mon[31282]: from='client.? 172.21.15.149:0/1114368283' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:48 smithi149 ceph-mon[31282]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:48 smithi149 ceph-mon[31282]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:18:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:48 smithi090 ceph-mon[25016]: from='client.? 172.21.15.149:0/1114368283' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:48 smithi090 ceph-mon[25016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:18:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:48 smithi090 ceph-mon[25016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:18:49.111 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:18:49.111 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-24T08:18:49.112 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T08:18:49.237 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T08:18:49.238 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T08:18:49.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T08:18:49.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:49 smithi149 ceph-mon[31282]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:49.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:49 smithi090 ceph-mon[25016]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:49.644 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:50.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:18:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:51 smithi090 ceph-mon[25016]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:51 smithi149 ceph-mon[31282]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 -- 172.21.15.90:0/246167441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98068540 msgr2=0x7f0e980689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/246167441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98068540 0x7f0e980689b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009910 tx=0x7f0e80009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 -- 172.21.15.90:0/246167441 shutdown_connections 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/246167441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98068540 0x7f0e980689b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/246167441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98106320 0x7f0e981066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 -- 172.21.15.90:0/246167441 >> 172.21.15.90:0/246167441 conn(0x7f0e98075130 msgr2=0x7f0e98075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 -- 172.21.15.90:0/246167441 shutdown_connections 2024-04-24T08:18:51.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.821+0000 7f0e9dc00700 1 -- 172.21.15.90:0/246167441 wait complete. 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.822+0000 7f0e9dc00700 1 Processor -- start 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.822+0000 7f0e9dc00700 1 -- start start 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e9dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98068540 0x7f0e980ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e9dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e9dc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981007b0 con 0x7f0e98106320 2024-04-24T08:18:51.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e9dc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98100920 con 0x7f0e98068540 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53834/0 (socket says 172.21.15.90:53834) 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 -- 172.21.15.90:0/1476369300 learned_addr learned my addr 172.21.15.90:0/1476369300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e96a00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98068540 0x7f0e980ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 -- 172.21.15.90:0/1476369300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98068540 msgr2=0x7f0e980ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98068540 0x7f0e980ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.823+0000 7f0e8fe00700 1 -- 172.21.15.90:0/1476369300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800095f0 con 0x7f0e98106320 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b5c0 tx=0x7f0e8000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:51.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8001d070 con 0x7f0e98106320 2024-04-24T08:18:51.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e80022470 con 0x7f0e98106320 2024-04-24T08:18:51.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8000f670 con 0x7f0e98106320 2024-04-24T08:18:51.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e981a2030 con 0x7f0e98106320 2024-04-24T08:18:51.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.824+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981a23e0 con 0x7f0e98106320 2024-04-24T08:18:51.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.825+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e98108770 con 0x7f0e98106320 2024-04-24T08:18:51.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.826+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0e80004c30 con 0x7f0e98106320 2024-04-24T08:18:51.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.827+0000 7f0e8f400700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 0x7f0e7c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:51.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.827+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0e8008cb10 con 0x7f0e98106320 2024-04-24T08:18:51.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.828+0000 7f0e96a00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 0x7f0e7c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:51.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.828+0000 7f0e96a00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 0x7f0e7c06eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005fd0 tx=0x7f0e8800a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:51.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:51.832+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e800575c0 con 0x7f0e98106320 2024-04-24T08:18:52.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.108+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0e9804f2a0 con 0x7f0e98106320 2024-04-24T08:18:52.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.112+0000 7f0e8f400700 1 -- 172.21.15.90:0/1476369300 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172853 (secure 0 0 0) 0x7f0e8005abe0 con 0x7f0e98106320 2024-04-24T08:18:52.115 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 msgr2=0x7f0e7c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 0x7f0e7c06eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005fd0 tx=0x7f0e8800a560 comp rx=0 tx=0).stop 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 msgr2=0x7f0e981001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b5c0 tx=0x7f0e8000be00 comp rx=0 tx=0).stop 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 shutdown_connections 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0e7c06c680 0x7f0e7c06eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e98068540 0x7f0e980ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 --2- 172.21.15.90:0/1476369300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e98106320 0x7f0e981001e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 >> 172.21.15.90:0/1476369300 conn(0x7f0e98075130 msgr2=0x7f0e980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:52.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.116+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 shutdown_connections 2024-04-24T08:18:52.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:52.117+0000 7f0e9dc00700 1 -- 172.21.15.90:0/1476369300 wait complete. 2024-04-24T08:18:52.435 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:52 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1476369300' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T08:18:52.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:52 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1476369300' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T08:18:52.801 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi090.jsrfxo","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1367639192},{"type":"v1","addr":"172.21.15.90:6801","nonce":1367639192}]},"active_addr":"172.21.15.90:6801/1367639192","active_change":"2024-04-24T08:13:49.915214+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi149.vavftl","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/","prometheus":"http://172.21.15.90:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3128177159}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":563018427}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3635708408}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":182534323}]}]} 2024-04-24T08:18:52.805 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T08:18:52.805 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T08:18:52.805 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:18:53.183 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:53.603 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:53 smithi090 ceph-mon[25016]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:53 smithi149 ceph-mon[31282]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:55.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2956414652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075280 msgr2=0x7ff7c4075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:55.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2956414652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075280 0x7ff7c4075650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009910 tx=0x7ff7b8009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:55.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2956414652 shutdown_connections 2024-04-24T08:18:55.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2956414652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075b90 0x7ff7c4110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2956414652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075280 0x7ff7c4075650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2956414652 >> 172.21.15.90:0/2956414652 conn(0x7ff7c40fdf90 msgr2=0x7ff7c41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2956414652 shutdown_connections 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.121+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2956414652 wait complete. 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.122+0000 7ff7cbe00700 1 Processor -- start 2024-04-24T08:18:55.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.122+0000 7ff7cbe00700 1 -- start start 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7cbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 0x7ff7c419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7cbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419d1c0 con 0x7ff7c4075b90 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7cbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41a0f50 con 0x7ff7c4075280 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53860/0 (socket says 172.21.15.90:53860) 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7c8a00700 1 -- 172.21.15.90:0/2957814660 learned_addr learned my addr 172.21.15.90:0/2957814660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:55.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7c9400700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 0x7ff7c419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.123+0000 7ff7c8a00700 1 -- 172.21.15.90:0/2957814660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 msgr2=0x7ff7c419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c8a00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 0x7ff7c419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c8a00700 1 -- 172.21.15.90:0/2957814660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7c4075b90 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c9400700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 0x7ff7c419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c8a00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b4b0 tx=0x7ff7b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b0011800 con 0x7ff7c4075b90 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7b0011e10 con 0x7ff7c4075b90 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b000f2e0 con 0x7ff7c4075b90 2024-04-24T08:18:55.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c41a1230 con 0x7ff7c4075b90 2024-04-24T08:18:55.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.124+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c41a1780 con 0x7ff7c4075b90 2024-04-24T08:18:55.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.126+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c410e340 con 0x7ff7c4075b90 2024-04-24T08:18:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.126+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff7b0010380 con 0x7ff7c4075b90 2024-04-24T08:18:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.127+0000 7ff7c2000700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.127+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff7b008b130 con 0x7ff7c4075b90 2024-04-24T08:18:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.128+0000 7ff7c9400700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:55.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.128+0000 7ff7c9400700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 0x7ff7ac06eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800b850 tx=0x7ff7b800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:55.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.131+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff7b0055be0 con 0x7ff7c4075b90 2024-04-24T08:18:55.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.353+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7c404f2a0 con 0x7ff7c4075b90 2024-04-24T08:18:55.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.355+0000 7ff7c2000700 1 -- 172.21.15.90:0/2957814660 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11312 (secure 0 0 0) 0x7ff7b0059200 con 0x7ff7c4075b90 2024-04-24T08:18:55.356 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:18:55.356 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:18:32.600771+0000","last_up_change":"2024-04-24T08:18:31.586535+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6803","nonce":3550641031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6805","nonce":3550641031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6809","nonce":3550641031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6807","nonce":3550641031}]},"public_addr":"172.21.15.90:6803/3550641031","cluster_addr":"172.21.15.90:6805/3550641031","heartbeat_back_addr":"172.21.15.90:6809/3550641031","heartbeat_front_addr":"172.21.15.90:6807/3550641031","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6811","nonce":2560478642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6813","nonce":2560478642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6817","nonce":2560478642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6815","nonce":2560478642}]},"public_addr":"172.21.15.90:6811/2560478642","cluster_addr":"172.21.15.90:6813/2560478642","heartbeat_back_addr":"172.21.15.90:6817/2560478642","heartbeat_front_addr":"172.21.15.90:6815/2560478642","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6819","nonce":358354662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6821","nonce":358354662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6825","nonce":358354662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6823","nonce":358354662}]},"public_addr":"172.21.15.90:6819/358354662","cluster_addr":"172.21.15.90:6821/358354662","heartbeat_back_addr":"172.21.15.90:6825/358354662","heartbeat_front_addr":"172.21.15.90:6823/358354662","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6801","nonce":1453378074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6803","nonce":1453378074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6807","nonce":1453378074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6805","nonce":1453378074}]},"public_addr":"172.21.15.149:6801/1453378074","cluster_addr":"172.21.15.149:6803/1453378074","heartbeat_back_addr":"172.21.15.149:6807/1453378074","heartbeat_front_addr":"172.21.15.149:6805/1453378074","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6809","nonce":197987858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6811","nonce":197987858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6815","nonce":197987858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6813","nonce":197987858}]},"public_addr":"172.21.15.149:6809/197987858","cluster_addr":"172.21.15.149:6811/197987858","heartbeat_back_addr":"172.21.15.149:6815/197987858","heartbeat_front_addr":"172.21.15.149:6813/197987858","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6817","nonce":142618710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6819","nonce":142618710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6823","nonce":142618710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6821","nonce":142618710}]},"public_addr":"172.21.15.149:6817/142618710","cluster_addr":"172.21.15.149:6819/142618710","heartbeat_back_addr":"172.21.15.149:6823/142618710","heartbeat_front_addr":"172.21.15.149:6821/142618710","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 msgr2=0x7ff7ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 0x7ff7ac06eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800b850 tx=0x7ff7b800b540 comp rx=0 tx=0).stop 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 msgr2=0x7ff7c419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b4b0 tx=0x7ff7b000b870 comp rx=0 tx=0).stop 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 shutdown_connections 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c4075280 0x7ff7c419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 --2- 172.21.15.90:0/2957814660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c4075b90 0x7ff7c419cae0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:55.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.358+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 >> 172.21.15.90:0/2957814660 conn(0x7ff7c40fdf90 msgr2=0x7ff7c40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:55.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.359+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 shutdown_connections 2024-04-24T08:18:55.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:55.359+0000 7ff7cbe00700 1 -- 172.21.15.90:0/2957814660 wait complete. 2024-04-24T08:18:55.728 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:55 smithi090 ceph-mon[25016]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:55 smithi149 ceph-mon[31282]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:56.148 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T08:18:56.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:18:56.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:56 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2957814660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:18:56.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:56 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2957814660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:57 smithi090 ceph-mon[25016]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:57.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:57 smithi149 ceph-mon[31282]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:58.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 -- 172.21.15.90:0/3756483785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075b90 msgr2=0x7fd83c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:58.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 --2- 172.21.15.90:0/3756483785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075b90 0x7fd83c110ba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd828009910 tx=0x7fd828009c20 comp rx=0 tx=0).stop 2024-04-24T08:18:58.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 -- 172.21.15.90:0/3756483785 shutdown_connections 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 --2- 172.21.15.90:0/3756483785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075b90 0x7fd83c110ba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 --2- 172.21.15.90:0/3756483785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075280 0x7fd83c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 -- 172.21.15.90:0/3756483785 >> 172.21.15.90:0/3756483785 conn(0x7fd83c0fdf40 msgr2=0x7fd83c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 -- 172.21.15.90:0/3756483785 shutdown_connections 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.758+0000 7fd841e00700 1 -- 172.21.15.90:0/3756483785 wait complete. 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.759+0000 7fd841e00700 1 Processor -- start 2024-04-24T08:18:58.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd841e00700 1 -- start start 2024-04-24T08:18:58.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd841e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd841e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075b90 0x7fd83c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd841e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c19d000 con 0x7fd83c075280 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd841e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c19d140 con 0x7fd83c075b90 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.760+0000 7fd83aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53884/0 (socket says 172.21.15.90:53884) 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 -- 172.21.15.90:0/2949964756 learned_addr learned my addr 172.21.15.90:0/2949964756 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83a000700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075b90 0x7fd83c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 -- 172.21.15.90:0/2949964756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075b90 msgr2=0x7fd83c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075b90 0x7fd83c19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 -- 172.21.15.90:0/2949964756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280095f0 con 0x7fd83c075280 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.761+0000 7fd83aa00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c930 tx=0x7fd82c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.762+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd82c007ad0 con 0x7fd83c075280 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.762+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd82c007c30 con 0x7fd83c075280 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.762+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd82c018630 con 0x7fd83c075280 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.762+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c1a0f90 con 0x7fd83c075280 2024-04-24T08:18:58.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.762+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c1a1490 con 0x7fd83c075280 2024-04-24T08:18:58.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.763+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c10e310 con 0x7fd83c075280 2024-04-24T08:18:58.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.764+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fd82c00f450 con 0x7fd83c075280 2024-04-24T08:18:58.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.765+0000 7fd840a00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 0x7fd82006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:18:58.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.766+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd82c08b7e0 con 0x7fd83c075280 2024-04-24T08:18:58.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.766+0000 7fd83a000700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 0x7fd82006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:18:58.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.766+0000 7fd83a000700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 0x7fd82006eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:18:58.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.770+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd82c056160 con 0x7fd83c075280 2024-04-24T08:18:58.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.987+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd83c04f2a0 con 0x7fd83c075280 2024-04-24T08:18:58.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.989+0000 7fd840a00700 1 -- 172.21.15.90:0/2949964756 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11312 (secure 0 0 0) 0x7fd83c04f2a0 con 0x7fd83c075280 2024-04-24T08:18:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:18:58.991 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:18:32.600771+0000","last_up_change":"2024-04-24T08:18:31.586535+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6803","nonce":3550641031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6805","nonce":3550641031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6809","nonce":3550641031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6807","nonce":3550641031}]},"public_addr":"172.21.15.90:6803/3550641031","cluster_addr":"172.21.15.90:6805/3550641031","heartbeat_back_addr":"172.21.15.90:6809/3550641031","heartbeat_front_addr":"172.21.15.90:6807/3550641031","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6811","nonce":2560478642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6813","nonce":2560478642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6817","nonce":2560478642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6815","nonce":2560478642}]},"public_addr":"172.21.15.90:6811/2560478642","cluster_addr":"172.21.15.90:6813/2560478642","heartbeat_back_addr":"172.21.15.90:6817/2560478642","heartbeat_front_addr":"172.21.15.90:6815/2560478642","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6819","nonce":358354662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6821","nonce":358354662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6825","nonce":358354662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6823","nonce":358354662}]},"public_addr":"172.21.15.90:6819/358354662","cluster_addr":"172.21.15.90:6821/358354662","heartbeat_back_addr":"172.21.15.90:6825/358354662","heartbeat_front_addr":"172.21.15.90:6823/358354662","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6801","nonce":1453378074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6803","nonce":1453378074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6807","nonce":1453378074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6805","nonce":1453378074}]},"public_addr":"172.21.15.149:6801/1453378074","cluster_addr":"172.21.15.149:6803/1453378074","heartbeat_back_addr":"172.21.15.149:6807/1453378074","heartbeat_front_addr":"172.21.15.149:6805/1453378074","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6809","nonce":197987858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6811","nonce":197987858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6815","nonce":197987858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6813","nonce":197987858}]},"public_addr":"172.21.15.149:6809/197987858","cluster_addr":"172.21.15.149:6811/197987858","heartbeat_back_addr":"172.21.15.149:6815/197987858","heartbeat_front_addr":"172.21.15.149:6813/197987858","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6817","nonce":142618710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6819","nonce":142618710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6823","nonce":142618710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6821","nonce":142618710}]},"public_addr":"172.21.15.149:6817/142618710","cluster_addr":"172.21.15.149:6819/142618710","heartbeat_back_addr":"172.21.15.149:6823/142618710","heartbeat_front_addr":"172.21.15.149:6821/142618710","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:18:58.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 msgr2=0x7fd82006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 0x7fd82006eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82800b540 comp rx=0 tx=0).stop 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 msgr2=0x7fd83c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c930 tx=0x7fd82c00ccf0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 shutdown_connections 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd82006c680 0x7fd82006eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd83c075280 0x7fd83c19c470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 --2- 172.21.15.90:0/2949964756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd83c075b90 0x7fd83c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 >> 172.21.15.90:0/2949964756 conn(0x7fd83c0fdf40 msgr2=0x7fd83c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.992+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 shutdown_connections 2024-04-24T08:18:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:18:58.993+0000 7fd841e00700 1 -- 172.21.15.90:0/2949964756 wait complete. 2024-04-24T08:18:59.741 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:59 smithi090 ceph-mon[25016]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:59.741 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:18:59 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2949964756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:59 smithi149 ceph-mon[31282]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:18:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:18:59 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2949964756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:18:59.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T08:18:59.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T08:18:59.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T08:18:59.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T08:18:59.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T08:18:59.810 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T08:19:00.325 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:00.333 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:00.343 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:00.361 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:00.378 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:00.385 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:01.580 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:01 smithi149 ceph-mon[31282]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:01.625 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:01 smithi090 ceph-mon[25016]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:03.593 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:03 smithi090 ceph-mon[25016]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:03 smithi149 ceph-mon[31282]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:05.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:05 smithi090 ceph-mon[25016]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:05.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:05 smithi149 ceph-mon[31282]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:07.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:07 smithi090 ceph-mon[25016]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:07 smithi149 ceph-mon[31282]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.936+0000 7fde1b200700 1 -- 172.21.15.90:0/28270275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14068540 msgr2=0x7fde140689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.936+0000 7fde1b200700 1 --2- 172.21.15.90:0/28270275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14068540 0x7fde140689b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009910 tx=0x7fddfc009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 -- 172.21.15.90:0/28270275 shutdown_connections 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 --2- 172.21.15.90:0/28270275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14068540 0x7fde140689b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 --2- 172.21.15.90:0/28270275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14101000 0x7fde141013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 -- 172.21.15.90:0/28270275 >> 172.21.15.90:0/28270275 conn(0x7fde14075130 msgr2=0x7fde14075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 -- 172.21.15.90:0/28270275 shutdown_connections 2024-04-24T08:19:07.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.937+0000 7fde1b200700 1 -- 172.21.15.90:0/28270275 wait complete. 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 Processor -- start 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 -- start start 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14068540 0x7fde14198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14198c90 con 0x7fde14101000 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.938+0000 7fde1b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1419ca20 con 0x7fde14068540 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:07.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57034/0 (socket says 172.21.15.90:57034) 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 -- 172.21.15.90:0/1833716945 learned_addr learned my addr 172.21.15.90:0/1833716945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde13e00700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14068540 0x7fde14198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 -- 172.21.15.90:0/1833716945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14068540 msgr2=0x7fde14198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14068540 0x7fde14198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.939+0000 7fde0b400700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0095f0 con 0x7fde14101000 2024-04-24T08:19:07.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde0b400700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddfc006010 tx=0x7fddfc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddfc01d070 con 0x7fde14101000 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1419cca0 con 0x7fde14101000 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fddfc00f460 con 0x7fde14101000 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddfc017630 con 0x7fde14101000 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.940+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1419d190 con 0x7fde14101000 2024-04-24T08:19:07.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.941+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde1410f830 con 0x7fde14101000 2024-04-24T08:19:07.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.942+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fddfc00f5d0 con 0x7fde14101000 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.943+0000 7fde11600700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 0x7fddf806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.943+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fddfc08cb70 con 0x7fde14101000 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.943+0000 7fde13e00700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 0x7fddf806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.943+0000 7fde11600700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 0x7fddf8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.944+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddf8074e10 con 0x7fddf8072350 2024-04-24T08:19:07.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.944+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddfc0912a0 con 0x7fde14101000 2024-04-24T08:19:07.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.944+0000 7fde19200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 0x7fddf8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:07.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.944+0000 7fde13e00700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 0x7fddf806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde04007700 tx=0x7fde04007690 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:07.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.945+0000 7fde19200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 0x7fddf8074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:07.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.945+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== osd.0 v2:172.21.15.90:6802/3550641031 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fddf8074e10 con 0x7fddf8072350 2024-04-24T08:19:07.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.963+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 --> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde1404f2a0 con 0x7fddf8072350 2024-04-24T08:19:07.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.963+0000 7fde11600700 1 -- 172.21.15.90:0/1833716945 <== osd.0 v2:172.21.15.90:6802/3550641031 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fde1404f2a0 con 0x7fddf8072350 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.963+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 msgr2=0x7fddf8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.963+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 0x7fddf8074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 msgr2=0x7fddf806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 0x7fddf806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde04007700 tx=0x7fde04007690 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 msgr2=0x7fde141985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddfc006010 tx=0x7fddfc00be20 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 shutdown_connections 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6802/3550641031,v1:172.21.15.90:6803/3550641031] conn(0x7fddf8072350 0x7fddf8074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fddf806c680 0x7fddf806eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14068540 0x7fde14198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 --2- 172.21.15.90:0/1833716945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde14101000 0x7fde141985b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:07.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 >> 172.21.15.90:0/1833716945 conn(0x7fde14075130 msgr2=0x7fde140fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:07.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 shutdown_connections 2024-04-24T08:19:07.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:07.964+0000 7fde1b200700 1 -- 172.21.15.90:0/1833716945 wait complete. 2024-04-24T08:19:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.051+0000 7fb7d9600700 1 -- 172.21.15.90:0/1217010809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 msgr2=0x7fb7d4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.051+0000 7fb7d9600700 1 --2- 172.21.15.90:0/1217010809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d4106ff0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009910 tx=0x7fb7bc009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 -- 172.21.15.90:0/1217010809 shutdown_connections 2024-04-24T08:19:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 --2- 172.21.15.90:0/1217010809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d4106ff0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 --2- 172.21.15.90:0/1217010809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 0x7fb7d410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 -- 172.21.15.90:0/1217010809 >> 172.21.15.90:0/1217010809 conn(0x7fb7d4076060 msgr2=0x7fb7d4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 -- 172.21.15.90:0/1217010809 shutdown_connections 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.052+0000 7fb7d9600700 1 -- 172.21.15.90:0/1217010809 wait complete. 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 Processor -- start 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 -- start start 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 0x7fb7d419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419d060 con 0x7fb7d4106b80 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d9600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d41a0df0 con 0x7fb7d410c8f0 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57044/0 (socket says 172.21.15.90:57044) 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.053+0000 7fb7d2a00700 1 -- 172.21.15.90:0/4113829444 learned_addr learned my addr 172.21.15.90:0/4113829444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7ca000700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 0x7fb7d419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7d2a00700 1 -- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 msgr2=0x7fb7d419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7d2a00700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 0x7fb7d419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7d2a00700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0095f0 con 0x7fb7d4106b80 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7d2a00700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400d8d0 tx=0x7fb7c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c4009940 con 0x7fb7d4106b80 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb7c4010460 con 0x7fb7d4106b80 2024-04-24T08:19:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d41a10d0 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.054+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c400f5f0 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.055+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d41a1620 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.055+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7d4113c30 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.056+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb7c40105d0 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.057+0000 7fb7cbe00700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 0x7fb7b806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb7c408b510 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7cbe00700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 0x7fb7b8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7b8074e10 con 0x7fb7b8072350 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7ca000700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 0x7fb7b806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c408db40 con 0x7fb7d4106b80 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7ca000700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 0x7fb7b806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.058+0000 7fb7d3400700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 0x7fb7b8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.059+0000 7fb7d3400700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 0x7fb7b8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.062+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== osd.3 v2:172.21.15.149:6800/1453378074 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb7b8074e10 con 0x7fb7b8072350 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.079+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 --> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7d404f2a0 con 0x7fb7b8072350 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7cbe00700 1 -- 172.21.15.90:0/4113829444 <== osd.3 v2:172.21.15.149:6800/1453378074 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7d404f2a0 con 0x7fb7b8072350 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 msgr2=0x7fb7b8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 0x7fb7b8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 msgr2=0x7fb7b806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 0x7fb7b806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc005fd0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 msgr2=0x7fb7d419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.080+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400d8d0 tx=0x7fb7c400dc90 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 shutdown_connections 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:6800/1453378074,v1:172.21.15.149:6801/1453378074] conn(0x7fb7b8072350 0x7fb7b8074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fb7b806c680 0x7fb7b806eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7d4106b80 0x7fb7d419c440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 --2- 172.21.15.90:0/4113829444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d410c8f0 0x7fb7d419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 >> 172.21.15.90:0/4113829444 conn(0x7fb7d4076060 msgr2=0x7fb7d4077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 shutdown_connections 2024-04-24T08:19:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.081+0000 7fb7d9600700 1 -- 172.21.15.90:0/4113829444 wait complete. 2024-04-24T08:19:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:09 smithi090 ceph-mon[25016]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:09 smithi149 ceph-mon[31282]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 -- 172.21.15.90:0/2678214248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 msgr2=0x7fc0e0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 --2- 172.21.15.90:0/2678214248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0104e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009910 tx=0x7fc0cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 -- 172.21.15.90:0/2678214248 shutdown_connections 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 --2- 172.21.15.90:0/2678214248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0104e50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 --2- 172.21.15.90:0/2678214248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 0x7fc0e01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 -- 172.21.15.90:0/2678214248 >> 172.21.15.90:0/2678214248 conn(0x7fc0e00fbda0 msgr2=0x7fc0e00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 -- 172.21.15.90:0/2678214248 shutdown_connections 2024-04-24T08:19:10.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.097+0000 7fc0e7600700 1 -- 172.21.15.90:0/2678214248 wait complete. 2024-04-24T08:19:10.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.098+0000 7fc0e7600700 1 Processor -- start 2024-04-24T08:19:10.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.098+0000 7fc0e7600700 1 -- start start 2024-04-24T08:19:10.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0e7600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 0x7fc0e0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0e7600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0e7600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0198d50 con 0x7fc0e0100e80 2024-04-24T08:19:10.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0e7600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019cae0 con 0x7fc0e01004e0 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35688/0 (socket says 172.21.15.90:35688) 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0dfe00700 1 -- 172.21.15.90:0/3077964250 learned_addr learned my addr 172.21.15.90:0/3077964250 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.099+0000 7fc0e4c00700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 0x7fc0e0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dfe00700 1 -- 172.21.15.90:0/3077964250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 msgr2=0x7fc0e0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dfe00700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 0x7fc0e0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dfe00700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc0095f0 con 0x7fc0e0100e80 2024-04-24T08:19:10.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dfe00700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00bda0 tx=0x7fc0cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0cc01d070 con 0x7fc0e0100e80 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc022470 con 0x7fc0e0100e80 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e019cd60 con 0x7fc0e0100e80 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0cc00f690 con 0x7fc0e0100e80 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.100+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e019d250 con 0x7fc0e0100e80 2024-04-24T08:19:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.101+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0e010d650 con 0x7fc0e0100e80 2024-04-24T08:19:10.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.102+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc0cc022ae0 con 0x7fc0e0100e80 2024-04-24T08:19:10.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.103+0000 7fc0dd600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 0x7fc0c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.103+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc0cc08cbb0 con 0x7fc0e0100e80 2024-04-24T08:19:10.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0dd600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 0x7fc0c8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0c8074e10 con 0x7fc0c8072350 2024-04-24T08:19:10.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0e4c00700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 0x7fc0c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0cc0912a0 con 0x7fc0e0100e80 2024-04-24T08:19:10.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0e5600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 0x7fc0c8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0e4c00700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 0x7fc0c806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40060b0 tx=0x7fc0d4007b80 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.104+0000 7fc0e5600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 0x7fc0c8074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.105+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== osd.1 v2:172.21.15.90:6810/2560478642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc0c8074e10 con 0x7fc0c8072350 2024-04-24T08:19:10.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.122+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 --> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0e004f2a0 con 0x7fc0c8072350 2024-04-24T08:19:10.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0dd600700 1 -- 172.21.15.90:0/3077964250 <== osd.1 v2:172.21.15.90:6810/2560478642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0e004f2a0 con 0x7fc0c8072350 2024-04-24T08:19:10.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 msgr2=0x7fc0c8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 0x7fc0c8074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 msgr2=0x7fc0c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 0x7fc0c806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40060b0 tx=0x7fc0d4007b80 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 msgr2=0x7fc0e0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.123+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00bda0 tx=0x7fc0cc00be80 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 shutdown_connections 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc0c806c680 0x7fc0c806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e01004e0 0x7fc0e0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:6810/2560478642,v1:172.21.15.90:6811/2560478642] conn(0x7fc0c8072350 0x7fc0c8074760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 --2- 172.21.15.90:0/3077964250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0e0100e80 0x7fc0e0198670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 >> 172.21.15.90:0/3077964250 conn(0x7fc0e00fbda0 msgr2=0x7fc0e00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 shutdown_connections 2024-04-24T08:19:10.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.124+0000 7fc0e7600700 1 -- 172.21.15.90:0/3077964250 wait complete. 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.999+0000 7fad8cc00700 1 -- 172.21.15.90:0/3648003654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 msgr2=0x7fad8810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:09.999+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3648003654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad8810c5e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad70009940 tx=0x7fad70009c50 comp rx=0 tx=0).stop 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 -- 172.21.15.90:0/3648003654 shutdown_connections 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3648003654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad8810c5e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3648003654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 0x7fad88073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 -- 172.21.15.90:0/3648003654 >> 172.21.15.90:0/3648003654 conn(0x7fad880fbda0 msgr2=0x7fad880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 -- 172.21.15.90:0/3648003654 shutdown_connections 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.000+0000 7fad8cc00700 1 -- 172.21.15.90:0/3648003654 wait complete. 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 Processor -- start 2024-04-24T08:19:10.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 -- start start 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 0x7fad881981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88198dc0 con 0x7fad880739a0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.001+0000 7fad8cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8819cb50 con 0x7fad88073090 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35660/0 (socket says 172.21.15.90:35660) 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 -- 172.21.15.90:0/3899178721 learned_addr learned my addr 172.21.15.90:0/3899178721 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad85e00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 0x7fad881981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 -- 172.21.15.90:0/3899178721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 msgr2=0x7fad881981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 0x7fad881981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700095f0 con 0x7fad880739a0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.002+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad700055f0 tx=0x7fad700056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.003+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7001d070 con 0x7fad880739a0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.003+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad7000f460 con 0x7fad880739a0 2024-04-24T08:19:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.003+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad70021620 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.003+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8819cdd0 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.003+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8819d270 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.004+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad64000ff0 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.005+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fad70003a60 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.006+0000 7fad7ea00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 0x7fad6c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.006+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fad7008d200 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad85e00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 0x7fad6c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad7ea00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 0x7fad6c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad6c074e10 con 0x7fad6c072350 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad7008d5c0 con 0x7fad880739a0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad86800700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 0x7fad6c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.007+0000 7fad85e00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 0x7fad6c06eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.008+0000 7fad86800700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 0x7fad6c074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:10.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.008+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== osd.2 v2:172.21.15.90:6818/358354662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fad6c074e10 con 0x7fad6c072350 2024-04-24T08:19:10.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.025+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 --> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad64002ce0 con 0x7fad6c072350 2024-04-24T08:19:10.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad7ea00700 1 -- 172.21.15.90:0/3899178721 <== osd.2 v2:172.21.15.90:6818/358354662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad64002ce0 con 0x7fad6c072350 2024-04-24T08:19:10.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 msgr2=0x7fad6c074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 0x7fad6c074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 msgr2=0x7fad6c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 0x7fad6c06eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78008040 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 msgr2=0x7fad881986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.026+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad700055f0 tx=0x7fad700056d0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 shutdown_connections 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6818/358354662,v1:172.21.15.90:6819/358354662] conn(0x7fad6c072350 0x7fad6c074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fad6c06c680 0x7fad6c06eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad88073090 0x7fad881981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 --2- 172.21.15.90:0/3899178721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad880739a0 0x7fad881986e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 >> 172.21.15.90:0/3899178721 conn(0x7fad880fbda0 msgr2=0x7fad88106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 shutdown_connections 2024-04-24T08:19:10.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:10.027+0000 7fad8cc00700 1 -- 172.21.15.90:0/3899178721 wait complete. 2024-04-24T08:19:11.177 INFO:teuthology.orchestra.run.smithi090.stdout:115964117016 2024-04-24T08:19:11.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T08:19:11.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:11 smithi090 ceph-mon[25016]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:11.501 INFO:teuthology.orchestra.run.smithi090.stdout:38654705709 2024-04-24T08:19:11.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T08:19:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:11 smithi149 ceph-mon[31282]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:12.148 INFO:teuthology.orchestra.run.smithi090.stdout:81604378654 2024-04-24T08:19:12.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T08:19:12.440 INFO:teuthology.orchestra.run.smithi090.stdout:60129542181 2024-04-24T08:19:12.441 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.367+0000 7fc8efa00700 1 -- 172.21.15.90:0/297708959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 msgr2=0x7fc8e810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.367+0000 7fc8efa00700 1 --2- 172.21.15.90:0/297708959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e810ccc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009910 tx=0x7fc8dc009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 -- 172.21.15.90:0/297708959 shutdown_connections 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 --2- 172.21.15.90:0/297708959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 0x7fc8e8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 --2- 172.21.15.90:0/297708959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e810ccc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 -- 172.21.15.90:0/297708959 >> 172.21.15.90:0/297708959 conn(0x7fc8e8076060 msgr2=0x7fc8e8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:13.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 -- 172.21.15.90:0/297708959 shutdown_connections 2024-04-24T08:19:13.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.368+0000 7fc8efa00700 1 -- 172.21.15.90:0/297708959 wait complete. 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 Processor -- start 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 -- start start 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 0x7fc8e819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e819d120 con 0x7fc8e810c8f0 2024-04-24T08:19:13.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.369+0000 7fc8efa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e81a0eb0 con 0x7fc8e8106b80 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35700/0 (socket says 172.21.15.90:35700) 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 -- 172.21.15.90:0/1232407351 learned_addr learned my addr 172.21.15.90:0/1232407351 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8ed000700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 0x7fc8e819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 -- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 msgr2=0x7fc8e819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 0x7fc8e819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.370+0000 7fc8e7e00700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0095f0 con 0x7fc8e810c8f0 2024-04-24T08:19:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8e7e00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d8d0 tx=0x7fc8d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d4009940 con 0x7fc8e810c8f0 2024-04-24T08:19:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4010460 con 0x7fc8e810c8f0 2024-04-24T08:19:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d400f5f0 con 0x7fc8e810c8f0 2024-04-24T08:19:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e81a1190 con 0x7fc8e810c8f0 2024-04-24T08:19:13.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.371+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e81a16e0 con 0x7fc8e810c8f0 2024-04-24T08:19:13.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.372+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8e8113c30 con 0x7fc8e810c8f0 2024-04-24T08:19:13.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.373+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc8d40105d0 con 0x7fc8e810c8f0 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.374+0000 7fc8e5600700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 0x7fc8d006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.374+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc8d408b510 con 0x7fc8e810c8f0 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.374+0000 7fc8ed000700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 0x7fc8d006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.375+0000 7fc8e5600700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 0x7fc8d00747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.375+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8d0074e60 con 0x7fc8d00723a0 2024-04-24T08:19:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.375+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d408da70 con 0x7fc8e810c8f0 2024-04-24T08:19:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.375+0000 7fc8ed000700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 0x7fc8d006eb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006010 tx=0x7fc8dc01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.375+0000 7fc8eda00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 0x7fc8d00747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.376+0000 7fc8eda00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 0x7fc8d00747b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.379+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== osd.4 v2:172.21.15.149:6808/197987858 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc8d0074e60 con 0x7fc8d00723a0 2024-04-24T08:19:13.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.396+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 --> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc8e804f2a0 con 0x7fc8d00723a0 2024-04-24T08:19:13.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8e5600700 1 -- 172.21.15.90:0/1232407351 <== osd.4 v2:172.21.15.149:6808/197987858 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc8e804f2a0 con 0x7fc8d00723a0 2024-04-24T08:19:13.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 msgr2=0x7fc8d00747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 0x7fc8d00747b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 msgr2=0x7fc8d006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 0x7fc8d006eb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006010 tx=0x7fc8dc01a040 comp rx=0 tx=0).stop 2024-04-24T08:19:13.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 msgr2=0x7fc8e819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.397+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d8d0 tx=0x7fc8d400dc90 comp rx=0 tx=0).stop 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 shutdown_connections 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:6808/197987858,v1:172.21.15.149:6809/197987858] conn(0x7fc8d00723a0 0x7fc8d00747b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fc8d006c6d0 0x7fc8d006eb80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8106b80 0x7fc8e819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 --2- 172.21.15.90:0/1232407351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e810c8f0 0x7fc8e819ca40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 >> 172.21.15.90:0/1232407351 conn(0x7fc8e8076060 msgr2=0x7fc8e8077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:13.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 shutdown_connections 2024-04-24T08:19:13.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.398+0000 7fc8efa00700 1 -- 172.21.15.90:0/1232407351 wait complete. 2024-04-24T08:19:13.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:13 smithi090 ceph-mon[25016]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 -- 172.21.15.90:0/1674572524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 msgr2=0x7f2034104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 --2- 172.21.15.90:0/1674572524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034104e50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2020009910 tx=0x7f2020009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 -- 172.21.15.90:0/1674572524 shutdown_connections 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 --2- 172.21.15.90:0/1674572524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034104e50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 --2- 172.21.15.90:0/1674572524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 0x7f20341008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 -- 172.21.15.90:0/1674572524 >> 172.21.15.90:0/1674572524 conn(0x7f20340fbda0 msgr2=0x7f20340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 -- 172.21.15.90:0/1674572524 shutdown_connections 2024-04-24T08:19:13.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.608+0000 7f203aa00700 1 -- 172.21.15.90:0/1674572524 wait complete. 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.609+0000 7f203aa00700 1 Processor -- start 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.609+0000 7f203aa00700 1 -- start start 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.609+0000 7f203aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 0x7f2034073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f203aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f203aa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034073ad0 con 0x7f2034100e80 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f203aa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034073c10 con 0x7f20341004e0 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35730/0 (socket says 172.21.15.90:35730) 2024-04-24T08:19:13.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 -- 172.21.15.90:0/2200653989 learned_addr learned my addr 172.21.15.90:0/2200653989 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:13.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033e00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 0x7f2034073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 -- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 msgr2=0x7f2034073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 0x7f2034073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.610+0000 7f2033400700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200095f0 con 0x7f2034100e80 2024-04-24T08:19:13.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f2033400700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f202000bda0 tx=0x7f202000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202001d070 con 0x7f2034100e80 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f202000f460 con 0x7f2034100e80 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2020017630 con 0x7f2034100e80 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2034073e90 con 0x7f2034100e80 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.611+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20341a2430 con 0x7f2034100e80 2024-04-24T08:19:13.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.612+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f203410d650 con 0x7f2034100e80 2024-04-24T08:19:13.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.614+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f202000fab0 con 0x7f2034100e80 2024-04-24T08:19:13.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.614+0000 7f2030c00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 0x7f201806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f202008ca40 con 0x7f2034100e80 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2033e00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 0x7f201806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2030c00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 0x7f2018074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2018074e10 con 0x7f2018072350 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20200912a0 con 0x7f2034100e80 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.615+0000 7f2033e00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 0x7f201806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2024005950 tx=0x7f20240058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.616+0000 7f2038a00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 0x7f2018074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:13.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.616+0000 7f2038a00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 0x7f2018074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:13.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.618+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== osd.5 v2:172.21.15.149:6816/142618710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2018074e10 con 0x7f2018072350 2024-04-24T08:19:13.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.637+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 --> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f203404f2a0 con 0x7f2018072350 2024-04-24T08:19:13.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f2030c00700 1 -- 172.21.15.90:0/2200653989 <== osd.5 v2:172.21.15.149:6816/142618710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f203404f2a0 con 0x7f2018072350 2024-04-24T08:19:13.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 msgr2=0x7f2018074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 0x7f2018074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 msgr2=0x7f201806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 0x7f201806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2024005950 tx=0x7f20240058e0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 msgr2=0x7f2034073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f202000bda0 tx=0x7f202000be80 comp rx=0 tx=0).stop 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 shutdown_connections 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:6816/142618710,v1:172.21.15.149:6817/142618710] conn(0x7f2018072350 0x7f2018074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f201806c680 0x7f201806eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20341004e0 0x7f2034073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 --2- 172.21.15.90:0/2200653989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2034100e80 0x7f2034073590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.638+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 >> 172.21.15.90:0/2200653989 conn(0x7f20340fbda0 msgr2=0x7f20340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.639+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 shutdown_connections 2024-04-24T08:19:13.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:13.639+0000 7f203aa00700 1 -- 172.21.15.90:0/2200653989 wait complete. 2024-04-24T08:19:13.643 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:13 smithi149 ceph-mon[31282]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:14.534 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:14.550 INFO:teuthology.orchestra.run.smithi090.stdout:150323855378 2024-04-24T08:19:14.550 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T08:19:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:15.710 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:15 smithi090 ceph-mon[25016]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:15.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:15 smithi149 ceph-mon[31282]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:17 smithi090 ceph-mon[25016]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:17 smithi149 ceph-mon[31282]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:18.382 INFO:teuthology.orchestra.run.smithi090.stdout:171798691851 2024-04-24T08:19:18.383 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T08:19:19.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:19 smithi090 ceph-mon[25016]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:19.733 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:19 smithi149 ceph-mon[31282]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:20.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 -- 172.21.15.90:0/1146207902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6041004e0 msgr2=0x7fe6041008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 --2- 172.21.15.90:0/1146207902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6041004e0 0x7fe6041008b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009910 tx=0x7fe5f4009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 -- 172.21.15.90:0/1146207902 shutdown_connections 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 --2- 172.21.15.90:0/1146207902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe604100e80 0x7fe604104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 --2- 172.21.15.90:0/1146207902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6041004e0 0x7fe6041008b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 -- 172.21.15.90:0/1146207902 >> 172.21.15.90:0/1146207902 conn(0x7fe6040fbda0 msgr2=0x7fe6040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 -- 172.21.15.90:0/1146207902 shutdown_connections 2024-04-24T08:19:20.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.089+0000 7fe60a800700 1 -- 172.21.15.90:0/1146207902 wait complete. 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.090+0000 7fe60a800700 1 Processor -- start 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.090+0000 7fe60a800700 1 -- start start 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe60a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6041004e0 0x7fe604073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe60a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe60a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604073ad0 con 0x7fe604100e80 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe60a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604073c10 con 0x7fe6041004e0 2024-04-24T08:19:20.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45862/0 (socket says 172.21.15.90:45862) 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 -- 172.21.15.90:0/259557797 learned_addr learned my addr 172.21.15.90:0/259557797 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe603400700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6041004e0 0x7fe604073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 -- 172.21.15.90:0/259557797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6041004e0 msgr2=0x7fe604073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6041004e0 0x7fe604073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.091+0000 7fe602a00700 1 -- 172.21.15.90:0/259557797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40095f0 con 0x7fe604100e80 2024-04-24T08:19:20.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe602a00700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000c8f0 tx=0x7fe5f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f0007a50 con 0x7fe604100e80 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6041a20c0 con 0x7fe604100e80 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe5f0007bb0 con 0x7fe604100e80 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f0018630 con 0x7fe604100e80 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.092+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041a24d0 con 0x7fe604100e80 2024-04-24T08:19:20.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.093+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6041087f0 con 0x7fe604100e80 2024-04-24T08:19:20.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.096+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe5f0007d20 con 0x7fe604100e80 2024-04-24T08:19:20.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.096+0000 7fe5fbe00700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 0x7fe5e806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.097+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe5f008b500 con 0x7fe604100e80 2024-04-24T08:19:20.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.097+0000 7fe603400700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 0x7fe5e806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.098+0000 7fe603400700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 0x7fe5e806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006010 tx=0x7fe5f401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:20.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.101+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe5f0055f20 con 0x7fe604100e80 2024-04-24T08:19:20.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.335+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe60404f2a0 con 0x7fe604100e80 2024-04-24T08:19:20.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.336+0000 7fe5fbe00700 1 -- 172.21.15.90:0/259557797 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5f0059540 con 0x7fe604100e80 2024-04-24T08:19:20.337 INFO:teuthology.orchestra.run.smithi090.stdout:81604378656 2024-04-24T08:19:20.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.338+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 msgr2=0x7fe5e806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.338+0000 7fe60a800700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 0x7fe5e806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006010 tx=0x7fe5f401a040 comp rx=0 tx=0).stop 2024-04-24T08:19:20.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.338+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 msgr2=0x7fe604073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.338+0000 7fe60a800700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000c8f0 tx=0x7fe5f000ccb0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 shutdown_connections 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe5e806c680 0x7fe5e806eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6041004e0 0x7fe604073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 --2- 172.21.15.90:0/259557797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe604100e80 0x7fe604073590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 >> 172.21.15.90:0/259557797 conn(0x7fe6040fbda0 msgr2=0x7fe6040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 shutdown_connections 2024-04-24T08:19:20.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.339+0000 7fe60a800700 1 -- 172.21.15.90:0/259557797 wait complete. 2024-04-24T08:19:20.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:20.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:20 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/54488434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T08:19:20.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:20 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/259557797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 -- 172.21.15.90:0/1838738297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 msgr2=0x7f105810c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 --2- 172.21.15.90:0/1838738297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f105810c560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1044009910 tx=0x7f1044009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 -- 172.21.15.90:0/1838738297 shutdown_connections 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 --2- 172.21.15.90:0/1838738297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f105810c560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 --2- 172.21.15.90:0/1838738297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 0x7f10580734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 -- 172.21.15.90:0/1838738297 >> 172.21.15.90:0/1838738297 conn(0x7f10580fbd80 msgr2=0x7f10580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 -- 172.21.15.90:0/1838738297 shutdown_connections 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.943+0000 7f105f600700 1 -- 172.21.15.90:0/1838738297 wait complete. 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.944+0000 7f105f600700 1 Processor -- start 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f105f600700 1 -- start start 2024-04-24T08:19:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f105f600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 0x7f1058197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f105f600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f105f600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105819c930 con 0x7f1058073a00 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f105f600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105819caa0 con 0x7f10580730f0 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f1057e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f1057e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45832/0 (socket says 172.21.15.90:45832) 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.945+0000 7f1057e00700 1 -- 172.21.15.90:0/54488434 learned_addr learned my addr 172.21.15.90:0/54488434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f105cc00700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 0x7f1058197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1057e00700 1 -- 172.21.15.90:0/54488434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 msgr2=0x7f1058197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1057e00700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 0x7f1058197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1057e00700 1 -- 172.21.15.90:0/54488434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440095f0 con 0x7f1058073a00 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1057e00700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f104400b5c0 tx=0x7f104400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f104401d070 con 0x7f1058073a00 2024-04-24T08:19:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105819cd80 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1044022470 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.946+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f104400f670 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.947+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105819d2d0 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.948+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1058109c60 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.949+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1044004c30 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.950+0000 7f1055600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 0x7f104006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.950+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f104408cbb0 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.950+0000 7f105cc00700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 0x7f104006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.951+0000 7f105cc00700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 0x7f104006eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c00a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:19.955+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f10440575e0 con 0x7f1058073a00 2024-04-24T08:19:20.733 INFO:teuthology.orchestra.run.smithi090.stdout:38654705711 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.195+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f10581992e0 con 0x7f1058073a00 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.196+0000 7f1055600700 1 -- 172.21.15.90:0/54488434 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1044027090 con 0x7f1058073a00 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.198+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 msgr2=0x7f104006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.198+0000 7f105f600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 0x7f104006eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c00a560 comp rx=0 tx=0).stop 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.198+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 msgr2=0x7f10581985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.198+0000 7f105f600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f104400b5c0 tx=0x7f104400be00 comp rx=0 tx=0).stop 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 shutdown_connections 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f104006c680 0x7f104006eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10580730f0 0x7f1058197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 --2- 172.21.15.90:0/54488434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1058073a00 0x7f10581985f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 >> 172.21.15.90:0/54488434 conn(0x7f10580fbd80 msgr2=0x7f1058106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 shutdown_connections 2024-04-24T08:19:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:20.199+0000 7f105f600700 1 -- 172.21.15.90:0/54488434 wait complete. 2024-04-24T08:19:20.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:20.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:20 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/54488434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T08:19:20.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:20 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/259557797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.491+0000 7f723f400700 1 -- 172.21.15.90:0/1424917909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 msgr2=0x7f72380734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.491+0000 7f723f400700 1 --2- 172.21.15.90:0/1424917909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f72380734c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f722c009910 tx=0x7f722c009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 -- 172.21.15.90:0/1424917909 shutdown_connections 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 --2- 172.21.15.90:0/1424917909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 0x7f7238110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 --2- 172.21.15.90:0/1424917909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f72380734c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 -- 172.21.15.90:0/1424917909 >> 172.21.15.90:0/1424917909 conn(0x7f72380fbda0 msgr2=0x7f72380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 -- 172.21.15.90:0/1424917909 shutdown_connections 2024-04-24T08:19:21.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.492+0000 7f723f400700 1 -- 172.21.15.90:0/1424917909 wait complete. 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 Processor -- start 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 -- start start 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 0x7f723819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723819d190 con 0x7f72380730f0 2024-04-24T08:19:21.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.493+0000 7f723f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72381a0f20 con 0x7f7238073a00 2024-04-24T08:19:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45882/0 (socket says 172.21.15.90:45882) 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 -- 172.21.15.90:0/3000760953 learned_addr learned my addr 172.21.15.90:0/3000760953 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f7237e00700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 0x7f723819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 -- 172.21.15.90:0/3000760953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 msgr2=0x7f723819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 0x7f723819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 -- 172.21.15.90:0/3000760953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0095f0 con 0x7f72380730f0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.494+0000 7f723ca00700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b5c0 tx=0x7f722c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.495+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722c01d070 con 0x7f72380730f0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.495+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f722c022470 con 0x7f72380730f0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.495+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722c00f670 con 0x7f72380730f0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.495+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72381a1200 con 0x7f72380730f0 2024-04-24T08:19:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.495+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72381a1700 con 0x7f72380730f0 2024-04-24T08:19:21.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.496+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723810e4d0 con 0x7f72380730f0 2024-04-24T08:19:21.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.497+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f722c004c30 con 0x7f72380730f0 2024-04-24T08:19:21.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.498+0000 7f7235600700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 0x7f722006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:21.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.498+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f722c08cb10 con 0x7f72380730f0 2024-04-24T08:19:21.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.498+0000 7f7237e00700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 0x7f722006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:21.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.499+0000 7f7237e00700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 0x7f722006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:21.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.503+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f722c0575c0 con 0x7f72380730f0 2024-04-24T08:19:21.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:21 smithi090 ceph-mon[25016]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:21.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.723+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f723819d8d0 con 0x7f72380730f0 2024-04-24T08:19:21.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.724+0000 7f7235600700 1 -- 172.21.15.90:0/3000760953 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f722c027020 con 0x7f72380730f0 2024-04-24T08:19:21.725 INFO:teuthology.orchestra.run.smithi090.stdout:60129542183 2024-04-24T08:19:21.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 msgr2=0x7f722006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 0x7f722006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224008040 comp rx=0 tx=0).stop 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 msgr2=0x7f723819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b5c0 tx=0x7f722c00be00 comp rx=0 tx=0).stop 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 shutdown_connections 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f722006c680 0x7f722006eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72380730f0 0x7f723819c570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 --2- 172.21.15.90:0/3000760953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7238073a00 0x7f723819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:21.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 >> 172.21.15.90:0/3000760953 conn(0x7f72380fbda0 msgr2=0x7f7238102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:21.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 shutdown_connections 2024-04-24T08:19:21.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:21.727+0000 7f723f400700 1 -- 172.21.15.90:0/3000760953 wait complete. 2024-04-24T08:19:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:21 smithi149 ceph-mon[31282]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:22.161 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378654 got 81604378656 for osd.2 2024-04-24T08:19:22.161 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:22.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3000760953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T08:19:22.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3000760953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T08:19:23.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705709 got 38654705711 for osd.0 2024-04-24T08:19:23.246 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:23.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542181 got 60129542183 for osd.1 2024-04-24T08:19:23.541 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:23.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.830+0000 7f6fd7a00700 1 -- 172.21.15.90:0/3349074303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd0100e80 msgr2=0x7f6fd0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:23.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.830+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/3349074303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd0100e80 0x7f6fd0104e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009910 tx=0x7f6fbc009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:23.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 -- 172.21.15.90:0/3349074303 shutdown_connections 2024-04-24T08:19:23.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/3349074303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd0100e80 0x7f6fd0104e50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/3349074303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd01004e0 0x7f6fd01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 -- 172.21.15.90:0/3349074303 >> 172.21.15.90:0/3349074303 conn(0x7f6fd00fbda0 msgr2=0x7f6fd00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 -- 172.21.15.90:0/3349074303 shutdown_connections 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.831+0000 7f6fd7a00700 1 -- 172.21.15.90:0/3349074303 wait complete. 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.832+0000 7f6fd7a00700 1 Processor -- start 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.832+0000 7f6fd7a00700 1 -- start start 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd7a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd7a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd0100e80 0x7f6fd0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd7a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0198df0 con 0x7f6fd01004e0 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd7a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd019cb80 con 0x7f6fd0100e80 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45908/0 (socket says 172.21.15.90:45908) 2024-04-24T08:19:23.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 -- 172.21.15.90:0/2710576822 learned_addr learned my addr 172.21.15.90:0/2710576822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 -- 172.21.15.90:0/2710576822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd0100e80 msgr2=0x7f6fd0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd0100e80 0x7f6fd0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.833+0000 7f6fd5000700 1 -- 172.21.15.90:0/2710576822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0095f0 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fd5000700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b4b0 tx=0x7f6fc400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc4011800 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd019ce60 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6fc4011e10 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc400f2e0 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.834+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd019d3b0 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.835+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd01045d0 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.836+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6fc4010380 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.837+0000 7f6fcd600700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 0x7f6fb806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.837+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6fc408b250 con 0x7f6fd01004e0 2024-04-24T08:19:23.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.838+0000 7f6fcfe00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 0x7f6fb806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:23.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.838+0000 7f6fcfe00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 0x7f6fb806eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b5c0 tx=0x7f6fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:23.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:22.845+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6fc4055bd0 con 0x7f6fd01004e0 2024-04-24T08:19:23.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.067+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6fd004f2a0 con 0x7f6fd01004e0 2024-04-24T08:19:23.633 INFO:teuthology.orchestra.run.smithi090.stdout:115964117019 2024-04-24T08:19:23.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.068+0000 7f6fcd600700 1 -- 172.21.15.90:0/2710576822 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fc40591f0 con 0x7f6fd01004e0 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 msgr2=0x7f6fb806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 0x7f6fb806eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b5c0 tx=0x7f6fbc005fd0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 msgr2=0x7f6fd01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b4b0 tx=0x7f6fc400b870 comp rx=0 tx=0).stop 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 shutdown_connections 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6fb806c680 0x7f6fb806eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd01004e0 0x7f6fd01981d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 --2- 172.21.15.90:0/2710576822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd0100e80 0x7f6fd0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 >> 172.21.15.90:0/2710576822 conn(0x7f6fd00fbda0 msgr2=0x7f6fd00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 shutdown_connections 2024-04-24T08:19:23.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:23.071+0000 7f6fd7a00700 1 -- 172.21.15.90:0/2710576822 wait complete. 2024-04-24T08:19:23.729 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:23 smithi090 ceph-mon[25016]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:23.729 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:23 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2710576822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T08:19:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:23 smithi149 ceph-mon[31282]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:23 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2710576822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T08:19:24.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117016 got 115964117019 for osd.3 2024-04-24T08:19:24.421 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:24.474 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:25.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:25 smithi090 ceph-mon[25016]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:25.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:25 smithi149 ceph-mon[31282]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:27.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:27 smithi090 ceph-mon[25016]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.687+0000 7f4677400700 1 -- 172.21.15.90:0/2838028948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4670100e80 msgr2=0x7f4670104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.687+0000 7f4677400700 1 --2- 172.21.15.90:0/2838028948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4670100e80 0x7f4670104e50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f465c009910 tx=0x7f465c009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 -- 172.21.15.90:0/2838028948 shutdown_connections 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 --2- 172.21.15.90:0/2838028948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4670100e80 0x7f4670104e50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 --2- 172.21.15.90:0/2838028948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46701004e0 0x7f46701008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 -- 172.21.15.90:0/2838028948 >> 172.21.15.90:0/2838028948 conn(0x7f46700fbda0 msgr2=0x7f46700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 -- 172.21.15.90:0/2838028948 shutdown_connections 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.688+0000 7f4677400700 1 -- 172.21.15.90:0/2838028948 wait complete. 2024-04-24T08:19:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.689+0000 7f4677400700 1 Processor -- start 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.689+0000 7f4677400700 1 -- start start 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4677400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4677400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670100e80 0x7f4670196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4677400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4670196b50 con 0x7f46701004e0 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4677400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467019a8e0 con 0x7f4670100e80 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4674a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4674a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45922/0 (socket says 172.21.15.90:45922) 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f4674a00700 1 -- 172.21.15.90:0/2402474288 learned_addr learned my addr 172.21.15.90:0/2402474288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.690+0000 7f466fe00700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670100e80 0x7f4670196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4674a00700 1 -- 172.21.15.90:0/2402474288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670100e80 msgr2=0x7f4670196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4674a00700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670100e80 0x7f4670196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4674a00700 1 -- 172.21.15.90:0/2402474288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0095f0 con 0x7f46701004e0 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4674a00700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f466400c8a0 tx=0x7f466400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466400cea0 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4664004830 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467019abc0 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46640055b0 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.691+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467019b110 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.693+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4670108850 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.693+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4664007ad0 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.694+0000 7f466d600700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 0x7f465806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.694+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f466408b290 con 0x7f46701004e0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.695+0000 7f466fe00700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 0x7f465806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.695+0000 7f466fe00700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 0x7f465806eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4670197550 tx=0x7f465c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.699+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4664055c90 con 0x7f46701004e0 2024-04-24T08:19:27.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:27 smithi149 ceph-mon[31282]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:27.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.797+0000 7f919a600700 1 -- 172.21.15.90:0/1471664626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194068540 msgr2=0x7f91940689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.797+0000 7f919a600700 1 --2- 172.21.15.90:0/1471664626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194068540 0x7f91940689b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9180009910 tx=0x7f9180009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:27.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 -- 172.21.15.90:0/1471664626 shutdown_connections 2024-04-24T08:19:27.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 --2- 172.21.15.90:0/1471664626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194068540 0x7f91940689b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 --2- 172.21.15.90:0/1471664626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194105030 0x7f9194105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 -- 172.21.15.90:0/1471664626 >> 172.21.15.90:0/1471664626 conn(0x7f9194074fa0 msgr2=0x7f91940753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 -- 172.21.15.90:0/1471664626 shutdown_connections 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.798+0000 7f919a600700 1 -- 172.21.15.90:0/1471664626 wait complete. 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 Processor -- start 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 -- start start 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194068540 0x7f9194197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9194198a20 con 0x7f9194105030 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.799+0000 7f919a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919419c7b0 con 0x7f9194068540 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45950/0 (socket says 172.21.15.90:45950) 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 -- 172.21.15.90:0/2964713781 learned_addr learned my addr 172.21.15.90:0/2964713781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9199200700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194068540 0x7f9194197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 -- 172.21.15.90:0/2964713781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194068540 msgr2=0x7f9194197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194068540 0x7f9194197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 -- 172.21.15.90:0/2964713781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800095f0 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.800+0000 7f9193e00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9180006010 tx=0x7f918000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.801+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f918001d070 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.801+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9180004f50 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.801+0000 7f919a600700 1 -- 172.21.15.90:0/2964713781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919419ca30 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.801+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9180021410 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.801+0000 7f919a600700 1 -- 172.21.15.90:0/2964713781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919419cf20 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.802+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91700052f0 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.803+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f918000f460 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.804+0000 7f9191600700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 0x7f917c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.804+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f918008c850 con 0x7f9194105030 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.805+0000 7f9199200700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 0x7f917c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.806+0000 7f9199200700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 0x7f917c06eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9188005950 tx=0x7f91880058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.809+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f91800571d0 con 0x7f9194105030 2024-04-24T08:19:27.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.921+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f467004f2a0 con 0x7f46701004e0 2024-04-24T08:19:27.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.922+0000 7f466d600700 1 -- 172.21.15.90:0/2402474288 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46640592b0 con 0x7f46701004e0 2024-04-24T08:19:27.923 INFO:teuthology.orchestra.run.smithi090.stdout:171798691853 2024-04-24T08:19:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 msgr2=0x7f465806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 0x7f465806eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4670197550 tx=0x7f465c01a040 comp rx=0 tx=0).stop 2024-04-24T08:19:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 msgr2=0x7f4670195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f466400c8a0 tx=0x7f466400cc60 comp rx=0 tx=0).stop 2024-04-24T08:19:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 shutdown_connections 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f465806c680 0x7f465806eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46701004e0 0x7f4670195f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 --2- 172.21.15.90:0/2402474288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670100e80 0x7f4670196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 >> 172.21.15.90:0/2402474288 conn(0x7f46700fbda0 msgr2=0x7f46700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 shutdown_connections 2024-04-24T08:19:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:27.925+0000 7f4677400700 1 -- 172.21.15.90:0/2402474288 wait complete. 2024-04-24T08:19:28.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.041+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f91700059c0 con 0x7f9194105030 2024-04-24T08:19:28.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.042+0000 7f9191600700 1 -- 172.21.15.90:0/2964713781 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9180026090 con 0x7f9194105030 2024-04-24T08:19:28.043 INFO:teuthology.orchestra.run.smithi090.stdout:150323855381 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 msgr2=0x7f917c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 0x7f917c06eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9188005950 tx=0x7f91880058e0 comp rx=0 tx=0).stop 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 msgr2=0x7f9194198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9180006010 tx=0x7f918000be20 comp rx=0 tx=0).stop 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 shutdown_connections 2024-04-24T08:19:28.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f917c06c680 0x7f917c06eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:28.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9194068540 0x7f9194197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:28.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 --2- 172.21.15.90:0/2964713781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9194105030 0x7f9194198340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:28.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.045+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 >> 172.21.15.90:0/2964713781 conn(0x7f9194074fa0 msgr2=0x7f9194106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:28.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.046+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 shutdown_connections 2024-04-24T08:19:28.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:28.046+0000 7f9186a00700 1 -- 172.21.15.90:0/2964713781 wait complete. 2024-04-24T08:19:28.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:28 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2402474288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T08:19:28.649 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:28 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2964713781' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T08:19:28.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691851 got 171798691853 for osd.5 2024-04-24T08:19:28.700 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:28.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:28 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2402474288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T08:19:28.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:28 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2964713781' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T08:19:29.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855378 got 150323855381 for osd.4 2024-04-24T08:19:29.378 DEBUG:teuthology.parallel:result is None 2024-04-24T08:19:29.378 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T08:19:29.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T08:19:29.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:29 smithi149 ceph-mon[31282]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:29.785 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:29 smithi090 ceph-mon[25016]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:31.502 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:31 smithi090 ceph-mon[25016]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:31 smithi149 ceph-mon[31282]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 -- 172.21.15.90:0/230353050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba4810c8f0 msgr2=0x7fba4810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 --2- 172.21.15.90:0/230353050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba4810c8f0 0x7fba4810ccc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba38009910 tx=0x7fba38009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 -- 172.21.15.90:0/230353050 shutdown_connections 2024-04-24T08:19:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 --2- 172.21.15.90:0/230353050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba48106b80 0x7fba48106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 --2- 172.21.15.90:0/230353050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba4810c8f0 0x7fba4810ccc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 -- 172.21.15.90:0/230353050 >> 172.21.15.90:0/230353050 conn(0x7fba48076060 msgr2=0x7fba48078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:32.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 -- 172.21.15.90:0/230353050 shutdown_connections 2024-04-24T08:19:32.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.027+0000 7fba4ea00700 1 -- 172.21.15.90:0/230353050 wait complete. 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.028+0000 7fba4ea00700 1 Processor -- start 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.028+0000 7fba4ea00700 1 -- start start 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba4ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4810c8f0 0x7fba4819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba4ea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4819d090 con 0x7fba48106b80 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba4ea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4819d1d0 con 0x7fba4810c8f0 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57474/0 (socket says 172.21.15.90:57474) 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 -- 172.21.15.90:0/3835044993 learned_addr learned my addr 172.21.15.90:0/3835044993 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47400700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4810c8f0 0x7fba4819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:32.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 -- 172.21.15.90:0/3835044993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4810c8f0 msgr2=0x7fba4819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4810c8f0 0x7fba4819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.029+0000 7fba47e00700 1 -- 172.21.15.90:0/3835044993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380095f0 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba47e00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba38006010 tx=0x7fba3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba3801d070 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fba3800f460 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba38017630 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba481a0fc0 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.030+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481a14b0 con 0x7fba48106b80 2024-04-24T08:19:32.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.031+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba48109b20 con 0x7fba48106b80 2024-04-24T08:19:32.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.032+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fba3800f5d0 con 0x7fba48106b80 2024-04-24T08:19:32.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.033+0000 7fba44c00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 0x7fba2c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:32.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.033+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fba3808cb30 con 0x7fba48106b80 2024-04-24T08:19:32.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.033+0000 7fba47400700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 0x7fba2c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:32.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.034+0000 7fba47400700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 0x7fba2c06eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba4819db20 tx=0x7fba34005d50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:32.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.038+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fba38057530 con 0x7fba48106b80 2024-04-24T08:19:32.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.246+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba481a17e0 con 0x7fba2c06c680 2024-04-24T08:19:32.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.249+0000 7fba44c00700 1 -- 172.21.15.90:0/3835044993 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27399 (secure 0 0 0) 0x7fba481a17e0 con 0x7fba2c06c680 2024-04-24T08:19:32.251 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:19:32.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 msgr2=0x7fba2c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:32.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 0x7fba2c06eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba4819db20 tx=0x7fba34005d50 comp rx=0 tx=0).stop 2024-04-24T08:19:32.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 msgr2=0x7fba4819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:32.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba38006010 tx=0x7fba3800be20 comp rx=0 tx=0).stop 2024-04-24T08:19:32.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 shutdown_connections 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fba2c06c680 0x7fba2c06eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba48106b80 0x7fba4819c500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 --2- 172.21.15.90:0/3835044993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4810c8f0 0x7fba4819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 >> 172.21.15.90:0/3835044993 conn(0x7fba48076060 msgr2=0x7fba48077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.252+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 shutdown_connections 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:32.253+0000 7fba4ea00700 1 -- 172.21.15.90:0/3835044993 wait complete. 2024-04-24T08:19:32.254 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T08:19:33.031 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-04-24T08:19:32.010743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003311"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T08:18:31.594844+0000","last_change":"2024-04-24T08:17:55.533940+0000","last_active":"2024-04-24T08:18:31.594844+0000","last_peered":"2024-04-24T08:18:31.594844+0000","last_clean":"2024-04-24T08:18:31.594844+0000","last_became_active":"2024-04-24T08:17:55.532967+0000","last_became_peered":"2024-04-24T08:17:55.532967+0000","last_unstale":"2024-04-24T08:18:31.594844+0000","last_undegraded":"2024-04-24T08:18:31.594844+0000","last_fullsized":"2024-04-24T08:18:31.594844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:16:48.668575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:16:48.668575+0000","last_clean_scrub_stamp":"2024-04-24T08:16:48.668575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.706181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2190000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.0109999999999999,"15min":1.0029999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.85199999999999998}]},{"osd":1,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.97299999999999998,"15min":0.96499999999999997},"min":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.337}]},{"osd":2,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0149999999999999,"15min":1.0049999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0649999999999999,"15min":1.0620000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1830000000000001}]},{"osd":4,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.262}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.89200000000000002,"15min":0.87},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Apr 24 08:18:53 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.93899999999999995,"15min":0.92100000000000004},"min":{"1min":0.73599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Wed Apr 24 08:19:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1279999999999999,"15min":1.1220000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.097},{"interface":"front","average":{"1min":1.262,"5min":1.1479999999999999,"15min":1.129},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Apr 24 08:18:56 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.052},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 08:19:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76000000000000001,"15min":0.73299999999999998},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.232,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85299999999999998,"15min":0.82899999999999996},"min":{"1min":0.496,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.323,"5min":1.837,"15min":1.837},"last":0.91300000000000003}]},{"osd":2,"last update":"Wed Apr 24 08:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2010000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.518},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1890000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.181}]},{"osd":3,"last update":"Wed Apr 24 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.536,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.22},{"interface":"front","average":{"1min":1.075,"5min":1.127,"15min":1.1359999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.427}]},{"osd":4,"last update":"Wed Apr 24 08:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.296},{"interface":"front","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.349}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.75,"15min":0.67800000000000005},"min":{"1min":0.54400000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.1830000000000001}]},{"osd":2,"last update":"Wed Apr 24 08:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.88,"15min":0.86199999999999999},"min":{"1min":0.57699999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.012,"5min":0.92000000000000004,"15min":0.90500000000000003},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed Apr 24 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0760000000000001,"15min":1.0609999999999999},"min":{"1min":0.79100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.087,"15min":1.083},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.522}]},{"osd":4,"last update":"Wed Apr 24 08:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.014},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:19:33.033 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T08:19:33.438 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:33.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:33 smithi090 ceph-mon[25016]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:33.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:33 smithi090 ceph-mon[25016]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:19:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:33 smithi149 ceph-mon[31282]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:33.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:33 smithi149 ceph-mon[31282]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:19:35.581 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:35 smithi090 ceph-mon[25016]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:35.581 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 -- 172.21.15.90:0/2237687211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 msgr2=0x7f79e41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 --2- 172.21.15.90:0/2237687211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e41008b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009910 tx=0x7f79d4009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 -- 172.21.15.90:0/2237687211 shutdown_connections 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 --2- 172.21.15.90:0/2237687211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 --2- 172.21.15.90:0/2237687211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e41008b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 -- 172.21.15.90:0/2237687211 >> 172.21.15.90:0/2237687211 conn(0x7f79e40fbda0 msgr2=0x7f79e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 -- 172.21.15.90:0/2237687211 shutdown_connections 2024-04-24T08:19:35.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.612+0000 7f79e9600700 1 -- 172.21.15.90:0/2237687211 wait complete. 2024-04-24T08:19:35.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.613+0000 7f79e9600700 1 Processor -- start 2024-04-24T08:19:35.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e9600700 1 -- start start 2024-04-24T08:19:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4073ad0 con 0x7f79e41004e0 2024-04-24T08:19:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e9600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4073c10 con 0x7f79e4100e80 2024-04-24T08:19:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57480/0 (socket says 172.21.15.90:57480) 2024-04-24T08:19:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.614+0000 7f79e2a00700 1 -- 172.21.15.90:0/989744706 learned_addr learned my addr 172.21.15.90:0/989744706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2a00700 1 -- 172.21.15.90:0/989744706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 msgr2=0x7f79e4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2000700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2a00700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2a00700 1 -- 172.21.15.90:0/989744706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40095f0 con 0x7f79e41004e0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2000700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79e2a00700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006010 tx=0x7f79d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d401d070 con 0x7f79e41004e0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f79d400f460 con 0x7f79e41004e0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.615+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d4021640 con 0x7f79e41004e0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.616+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e4073e90 con 0x7f79e41004e0 2024-04-24T08:19:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.616+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e41a2440 con 0x7f79e41004e0 2024-04-24T08:19:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.617+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e41087f0 con 0x7f79e41004e0 2024-04-24T08:19:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.618+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f79d400fab0 con 0x7f79e41004e0 2024-04-24T08:19:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.619+0000 7f79db400700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 0x7f79c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.619+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f79d408c9a0 con 0x7f79e41004e0 2024-04-24T08:19:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.619+0000 7f79e2000700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 0x7f79c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.620+0000 7f79e2000700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 0x7f79c806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79e4074840 tx=0x7f79cc008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:35.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.623+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f79d4057320 con 0x7f79e41004e0 2024-04-24T08:19:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:35 smithi149 ceph-mon[31282]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:35.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.832+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79e4061190 con 0x7f79c806c680 2024-04-24T08:19:35.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.835+0000 7f79db400700 1 -- 172.21.15.90:0/989744706 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27399 (secure 0 0 0) 0x7f79e4061190 con 0x7f79c806c680 2024-04-24T08:19:35.836 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:19:35.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.837+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 msgr2=0x7f79c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:35.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.837+0000 7f79e9600700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 0x7f79c806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79e4074840 tx=0x7f79cc008040 comp rx=0 tx=0).stop 2024-04-24T08:19:35.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.837+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 msgr2=0x7f79e4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:35.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.837+0000 7f79e9600700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006010 tx=0x7f79d400be20 comp rx=0 tx=0).stop 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 shutdown_connections 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f79c806c680 0x7f79c806eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e41004e0 0x7f79e4073050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 --2- 172.21.15.90:0/989744706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e4100e80 0x7f79e4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 >> 172.21.15.90:0/989744706 conn(0x7f79e40fbda0 msgr2=0x7f79e40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 shutdown_connections 2024-04-24T08:19:35.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:35.838+0000 7f79e9600700 1 -- 172.21.15.90:0/989744706 wait complete. 2024-04-24T08:19:35.840 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T08:19:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:36 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:19:36.642 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-24T08:19:34.011259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T08:18:31.594844+0000","last_change":"2024-04-24T08:17:55.533940+0000","last_active":"2024-04-24T08:18:31.594844+0000","last_peered":"2024-04-24T08:18:31.594844+0000","last_clean":"2024-04-24T08:18:31.594844+0000","last_became_active":"2024-04-24T08:17:55.532967+0000","last_became_peered":"2024-04-24T08:17:55.532967+0000","last_unstale":"2024-04-24T08:18:31.594844+0000","last_undegraded":"2024-04-24T08:18:31.594844+0000","last_fullsized":"2024-04-24T08:18:31.594844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:16:48.668575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:16:48.668575+0000","last_clean_scrub_stamp":"2024-04-24T08:16:48.668575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.706181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2190000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.0109999999999999,"15min":1.0029999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.85199999999999998}]},{"osd":1,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.97299999999999998,"15min":0.96499999999999997},"min":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.337}]},{"osd":2,"last update":"Wed Apr 24 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0149999999999999,"15min":1.0049999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0649999999999999,"15min":1.0620000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1830000000000001}]},{"osd":4,"last update":"Wed Apr 24 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.262}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.89200000000000002,"15min":0.87},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Apr 24 08:18:53 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.93899999999999995,"15min":0.92100000000000004},"min":{"1min":0.73599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Wed Apr 24 08:19:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1279999999999999,"15min":1.1220000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.097},{"interface":"front","average":{"1min":1.262,"5min":1.1479999999999999,"15min":1.129},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Apr 24 08:18:56 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.052},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 08:19:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76000000000000001,"15min":0.73299999999999998},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.232,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85299999999999998,"15min":0.82899999999999996},"min":{"1min":0.496,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.323,"5min":1.837,"15min":1.837},"last":0.91300000000000003}]},{"osd":2,"last update":"Wed Apr 24 08:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2010000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.518},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.0109999999999999,"15min":1.02},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1890000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.181}]},{"osd":3,"last update":"Wed Apr 24 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.536,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.22},{"interface":"front","average":{"1min":1.075,"5min":1.127,"15min":1.1359999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.427}]},{"osd":4,"last update":"Wed Apr 24 08:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.296},{"interface":"front","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.349}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 08:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.75,"15min":0.67800000000000005},"min":{"1min":0.54400000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.1830000000000001}]},{"osd":2,"last update":"Wed Apr 24 08:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.88,"15min":0.86199999999999999},"min":{"1min":0.57699999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.012,"5min":0.92000000000000004,"15min":0.90500000000000003},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed Apr 24 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.0760000000000001,"15min":1.0609999999999999},"min":{"1min":0.79100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.087,"15min":1.083},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.522}]},{"osd":4,"last update":"Wed Apr 24 08:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.014},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:19:36.643 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T08:19:36.643 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T08:19:36.643 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T08:19:36.643 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T08:19:36.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:36 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:19:37.014 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:37.519 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:37 smithi090 ceph-mon[25016]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:19:37.519 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:37 smithi090 ceph-mon[25016]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:37.519 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:19:37.519 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:19:37.519 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:19:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:37 smithi149 ceph-mon[31282]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:19:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:37 smithi149 ceph-mon[31282]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:19:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:19:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:19:39.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 -- 172.21.15.90:0/2362254980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 msgr2=0x7f21fc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 --2- 172.21.15.90:0/2362254980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc106ff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009910 tx=0x7f21e8009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 -- 172.21.15.90:0/2362254980 shutdown_connections 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 --2- 172.21.15.90:0/2362254980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc106ff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 --2- 172.21.15.90:0/2362254980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 0x7f21fc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 -- 172.21.15.90:0/2362254980 >> 172.21.15.90:0/2362254980 conn(0x7f21fc076060 msgr2=0x7f21fc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 -- 172.21.15.90:0/2362254980 shutdown_connections 2024-04-24T08:19:39.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.177+0000 7f2202e00700 1 -- 172.21.15.90:0/2362254980 wait complete. 2024-04-24T08:19:39.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.178+0000 7f2202e00700 1 Processor -- start 2024-04-24T08:19:39.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.178+0000 7f2202e00700 1 -- start start 2024-04-24T08:19:39.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f2202e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f2202e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 0x7f21fc19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f2202e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc19d090 con 0x7f21fc106b80 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f2202e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc1a0e20 con 0x7f21fc10c8f0 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57500/0 (socket says 172.21.15.90:57500) 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 -- 172.21.15.90:0/1580740907 learned_addr learned my addr 172.21.15.90:0/1580740907 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fb400700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 0x7f21fc19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 -- 172.21.15.90:0/1580740907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 msgr2=0x7f21fc19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 0x7f21fc19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.179+0000 7f21fbe00700 1 -- 172.21.15.90:0/1580740907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80095f0 con 0x7f21fc106b80 2024-04-24T08:19:39.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f21fbe00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c8a0 tx=0x7f21ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21ec00cea0 con 0x7f21fc106b80 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21ec004830 con 0x7f21fc106b80 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc1a1100 con 0x7f21fc106b80 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21ec0055b0 con 0x7f21fc106b80 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.180+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc1a1650 con 0x7f21fc106b80 2024-04-24T08:19:39.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.181+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc10ecc0 con 0x7f21fc106b80 2024-04-24T08:19:39.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.182+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f21ec007ad0 con 0x7f21fc106b80 2024-04-24T08:19:39.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.183+0000 7f21f8c00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 0x7f21e006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:39.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.183+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f21ec08b1f0 con 0x7f21fc106b80 2024-04-24T08:19:39.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.184+0000 7f21fb400700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 0x7f21e006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:39.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.184+0000 7f21fb400700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 0x7f21e006eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b5c0 tx=0x7f21e8005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:39.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.188+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f21ec055c90 con 0x7f21fc106b80 2024-04-24T08:19:39.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.448+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f21fc04f2a0 con 0x7f21fc106b80 2024-04-24T08:19:39.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.449+0000 7f21f8c00700 1 -- 172.21.15.90:0/1580740907 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f21ec0592b0 con 0x7f21fc106b80 2024-04-24T08:19:39.450 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:19:39.450 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T08:19:39.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 msgr2=0x7f21e006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:39.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 0x7f21e006eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b5c0 tx=0x7f21e8005960 comp rx=0 tx=0).stop 2024-04-24T08:19:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 msgr2=0x7f21fc19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c8a0 tx=0x7f21ec00cc60 comp rx=0 tx=0).stop 2024-04-24T08:19:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 shutdown_connections 2024-04-24T08:19:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f21e006c680 0x7f21e006eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21fc106b80 0x7f21fc19c470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 --2- 172.21.15.90:0/1580740907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21fc10c8f0 0x7f21fc19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:39.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 >> 172.21.15.90:0/1580740907 conn(0x7f21fc076060 msgr2=0x7f21fc077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:39.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 shutdown_connections 2024-04-24T08:19:39.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:39.452+0000 7f2202e00700 1 -- 172.21.15.90:0/1580740907 wait complete. 2024-04-24T08:19:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:39 smithi149 ceph-mon[31282]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:39 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1580740907' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T08:19:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:39 smithi090 ceph-mon[25016]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:39.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:39 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1580740907' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T08:19:40.290 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T08:19:40.290 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T08:19:40.291 INFO:teuthology.run_tasks:Running task print... 2024-04-24T08:19:40.299 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-04-24T08:19:40.300 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:19:40.309 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:19:40.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-24T08:19:40.725 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:41.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:41 smithi090 ceph-mon[25016]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:41.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:41 smithi149 ceph-mon[31282]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.922+0000 7f4187c00700 1 -- 172.21.15.90:0/3122964620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180102440 msgr2=0x7f4180102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.922+0000 7f4187c00700 1 --2- 172.21.15.90:0/3122964620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180102440 0x7f4180102850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4170009910 tx=0x7f4170009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 -- 172.21.15.90:0/3122964620 shutdown_connections 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 --2- 172.21.15.90:0/3122964620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180103640 0x7f4180103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 --2- 172.21.15.90:0/3122964620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180102440 0x7f4180102850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 -- 172.21.15.90:0/3122964620 >> 172.21.15.90:0/3122964620 conn(0x7f41800fd9f0 msgr2=0x7f41800ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 -- 172.21.15.90:0/3122964620 shutdown_connections 2024-04-24T08:19:42.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.923+0000 7f4187c00700 1 -- 172.21.15.90:0/3122964620 wait complete. 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 Processor -- start 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 -- start start 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180102440 0x7f4180197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4180198850 con 0x7f4180103640 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.924+0000 7f4187c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019d260 con 0x7f4180102440 2024-04-24T08:19:42.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45372/0 (socket says 172.21.15.90:45372) 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 -- 172.21.15.90:0/832319281 learned_addr learned my addr 172.21.15.90:0/832319281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4185200700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180102440 0x7f4180197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 -- 172.21.15.90:0/832319281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180102440 msgr2=0x7f4180197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180102440 0x7f4180197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 -- 172.21.15.90:0/832319281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700095f0 con 0x7f4180103640 2024-04-24T08:19:42.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.925+0000 7f4177e00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f416800c8a0 tx=0x7f416800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.926+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416800cea0 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.926+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4168004830 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.926+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418019d460 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.926+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41680055b0 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.926+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418019d980 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.927+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418010af90 con 0x7f4180103640 2024-04-24T08:19:42.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.928+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4168007ad0 con 0x7f4180103640 2024-04-24T08:19:42.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.929+0000 7f417e000700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 0x7f416406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:42.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.930+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f416808b300 con 0x7f4180103640 2024-04-24T08:19:42.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.930+0000 7f4185200700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 0x7f416406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:42.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.930+0000 7f4185200700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 0x7f416406eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4170005090 tx=0x7f417000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:42.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:42.934+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4168055c80 con 0x7f4180103640 2024-04-24T08:19:43.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.152+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f418010b150 con 0x7f4180103640 2024-04-24T08:19:43.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.159+0000 7f417e000700 1 -- 172.21.15.90:0/832319281 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f41680592a0 con 0x7f4180103640 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.162+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 msgr2=0x7f416406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.162+0000 7f4187c00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 0x7f416406eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4170005090 tx=0x7f417000b540 comp rx=0 tx=0).stop 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.162+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 msgr2=0x7f4180198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.162+0000 7f4187c00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f416800c8a0 tx=0x7f416800cc60 comp rx=0 tx=0).stop 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 shutdown_connections 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f416406c680 0x7f416406eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4180102440 0x7f4180197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 --2- 172.21.15.90:0/832319281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4180103640 0x7f4180198230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 >> 172.21.15.90:0/832319281 conn(0x7f41800fd9f0 msgr2=0x7f4180106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 shutdown_connections 2024-04-24T08:19:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:43.163+0000 7f4187c00700 1 -- 172.21.15.90:0/832319281 wait complete. 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/832319281' entity='client.admin' 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:19:43.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:43 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/832319281' entity='client.admin' 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:19:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:43 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:19:44.089 INFO:teuthology.run_tasks:Running task print... 2024-04-24T08:19:44.098 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-24T08:19:44.098 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:19:44.109 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:19:44.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T08:19:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:45.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:45 smithi090 ceph-mon[25016]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:45.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:45 smithi149 ceph-mon[31282]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 -- 172.21.15.90:0/133432264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 msgr2=0x7fcc68102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 --2- 172.21.15.90:0/133432264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68102840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009910 tx=0x7fcc5c009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 -- 172.21.15.90:0/133432264 shutdown_connections 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 --2- 172.21.15.90:0/133432264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc681036d0 0x7fcc68103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 --2- 172.21.15.90:0/133432264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68102840 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 -- 172.21.15.90:0/133432264 >> 172.21.15.90:0/133432264 conn(0x7fcc680fda80 msgr2=0x7fcc680ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 -- 172.21.15.90:0/133432264 shutdown_connections 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.572+0000 7fcc70200700 1 -- 172.21.15.90:0/133432264 wait complete. 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 Processor -- start 2024-04-24T08:19:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 -- start start 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc681036d0 0x7fcc68198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68198850 con 0x7fcc68102430 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.573+0000 7fcc70200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819d260 con 0x7fcc681036d0 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45390/0 (socket says 172.21.15.90:45390) 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 -- 172.21.15.90:0/590299265 learned_addr learned my addr 172.21.15.90:0/590299265 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 -- 172.21.15.90:0/590299265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc681036d0 msgr2=0x7fcc68198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:46.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc681036d0 0x7fcc68198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 -- 172.21.15.90:0/590299265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0095f0 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.574+0000 7fcc6d800700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.575+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5c01d070 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.575+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c00f460 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.575+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6819d400 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.575+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5c017630 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.575+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6819d8c0 con 0x7fcc68102430 2024-04-24T08:19:46.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.576+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6810af10 con 0x7fcc68102430 2024-04-24T08:19:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.577+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fcc5c003a60 con 0x7fcc68102430 2024-04-24T08:19:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.578+0000 7fcc66000700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 0x7fcc5006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.578+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcc5c08c9b0 con 0x7fcc68102430 2024-04-24T08:19:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.578+0000 7fcc6ce00700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 0x7fcc5006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.579+0000 7fcc6ce00700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 0x7fcc5006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005950 tx=0x7fcc540058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:46.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.583+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcc5c057460 con 0x7fcc68102430 2024-04-24T08:19:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.815+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc68108020 con 0x7fcc5006c680 2024-04-24T08:19:46.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.817+0000 7fcc66000700 1 -- 172.21.15.90:0/590299265 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcc68108020 con 0x7fcc5006c680 2024-04-24T08:19:46.818 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-04-24T08:19:46.819 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-04-24T08:19:46.819 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-04-24T08:19:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 msgr2=0x7fcc5006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 0x7fcc5006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005950 tx=0x7fcc540058e0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 msgr2=0x7fcc68197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c00be20 comp rx=0 tx=0).stop 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 shutdown_connections 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fcc5006c680 0x7fcc5006eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc68102430 0x7fcc68197cf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 --2- 172.21.15.90:0/590299265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc681036d0 0x7fcc68198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.820+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 >> 172.21.15.90:0/590299265 conn(0x7fcc680fda80 msgr2=0x7fcc68106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.821+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 shutdown_connections 2024-04-24T08:19:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:46.821+0000 7fcc70200700 1 -- 172.21.15.90:0/590299265 wait complete. 2024-04-24T08:19:47.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:47 smithi149 ceph-mon[31282]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:47.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:47 smithi090 ceph-mon[25016]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:47.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T08:19:48.354 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:48.729 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:48 smithi090 ceph-mon[25016]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:48.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:48 smithi149 ceph-mon[31282]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:49 smithi149 ceph-mon[31282]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:49 smithi090 ceph-mon[25016]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:50.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 -- 172.21.15.90:0/2519024744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 msgr2=0x7fadfc073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:50.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 --2- 172.21.15.90:0/2519024744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc073c20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fade8009940 tx=0x7fade8009c50 comp rx=0 tx=0).stop 2024-04-24T08:19:50.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 -- 172.21.15.90:0/2519024744 shutdown_connections 2024-04-24T08:19:50.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 --2- 172.21.15.90:0/2519024744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc073c20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 --2- 172.21.15.90:0/2519024744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 0x7fadfc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 -- 172.21.15.90:0/2519024744 >> 172.21.15.90:0/2519024744 conn(0x7fadfc0fb7f0 msgr2=0x7fadfc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:50.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 -- 172.21.15.90:0/2519024744 shutdown_connections 2024-04-24T08:19:50.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.602+0000 7fae03e00700 1 -- 172.21.15.90:0/2519024744 wait complete. 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.603+0000 7fae03e00700 1 Processor -- start 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.603+0000 7fae03e00700 1 -- start start 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae03e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae03e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 0x7fadfc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae03e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc198900 con 0x7fadfc0737b0 2024-04-24T08:19:50.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae03e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc19d310 con 0x7fadfc074d80 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52346/0 (socket says 172.21.15.90:52346) 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 -- 172.21.15.90:0/1709763033 learned_addr learned my addr 172.21.15.90:0/1709763033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae00a00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 0x7fadfc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 -- 172.21.15.90:0/1709763033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 msgr2=0x7fadfc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 0x7fadfc1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.604+0000 7fae01400700 1 -- 172.21.15.90:0/1709763033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade80095f0 con 0x7fadfc0737b0 2024-04-24T08:19:50.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fae01400700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b4b0 tx=0x7fadf000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf0011800 con 0x7fadfc0737b0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fadf0011e10 con 0x7fadfc0737b0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc19d510 con 0x7fadfc0737b0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf000f2e0 con 0x7fadfc0737b0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.605+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc19da30 con 0x7fadfc0737b0 2024-04-24T08:19:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.606+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc10b000 con 0x7fadfc0737b0 2024-04-24T08:19:50.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.607+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fadf0010380 con 0x7fadfc0737b0 2024-04-24T08:19:50.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.608+0000 7fadfa000700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 0x7fade406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:50.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.608+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fadf008b110 con 0x7fadfc0737b0 2024-04-24T08:19:50.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.608+0000 7fae00a00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 0x7fade406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:50.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.609+0000 7fae00a00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 0x7fade406eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fade8009910 tx=0x7fade8005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:50.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.612+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fadf0055b10 con 0x7fadfc0737b0 2024-04-24T08:19:50.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:50.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:19:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.838+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fadfc19e0f0 con 0x7fade406c680 2024-04-24T08:19:50.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.864+0000 7fadfa000700 1 -- 172.21.15.90:0/1709763033 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7fadfc19e0f0 con 0x7fade406c680 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (4m) 2m ago 6m 13.7M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (7m) 2m ago 7m 7067k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (5m) 80s ago 5m 6995k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (7m) 2m ago 7m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (5m) 80s ago 5m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (4m) 2m ago 6m 75.3M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:9283,8765,8443 running (8m) 2m ago 8m 479M - 18.2.0 dc2bc1663786 7ccc358e4d84 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (5m) 80s ago 5m 436M - 18.2.0 dc2bc1663786 30e493581e1d 2024-04-24T08:19:50.866 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (8m) 2m ago 8m 43.0M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (5m) 80s ago 5m 32.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (7m) 2m ago 7m 7796k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (5m) 80s ago 5m 8254k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (4m) 2m ago 4m 40.2M 3947M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (3m) 2m ago 3m 39.0M 3947M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (3m) 2m ago 3m 39.3M 3947M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (2m) 80s ago 2m 40.9M 1213M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (2m) 80s ago 2m 38.3M 1213M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (86s) 80s ago 85s 11.0M 1213M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:19:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (4m) 2m ago 6m 39.7M - 2.43.0 a07b618ecd1d 0199ca15cf9f 2024-04-24T08:19:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.867+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 msgr2=0x7fade406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.867+0000 7fae03e00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 0x7fade406eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fade8009910 tx=0x7fade8005fb0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.867+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 msgr2=0x7fadfc197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.867+0000 7fae03e00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b4b0 tx=0x7fadf000b870 comp rx=0 tx=0).stop 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 shutdown_connections 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fade406c680 0x7fade406eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadfc0737b0 0x7fadfc197da0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 --2- 172.21.15.90:0/1709763033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadfc074d80 0x7fadfc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 >> 172.21.15.90:0/1709763033 conn(0x7fadfc0fb7f0 msgr2=0x7fadfc106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 shutdown_connections 2024-04-24T08:19:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:50.868+0000 7fae03e00700 1 -- 172.21.15.90:0/1709763033 wait complete. 2024-04-24T08:19:51.661 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T08:19:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:51 smithi149 ceph-mon[31282]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:51 smithi090 ceph-mon[25016]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:52.065 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:52.785 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:52 smithi090 ceph-mon[25016]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:53.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:52 smithi149 ceph-mon[31282]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:53.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:53 smithi090 ceph-mon[25016]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:54.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:53 smithi149 ceph-mon[31282]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.266+0000 7f2566200700 1 -- 172.21.15.90:0/597853475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 msgr2=0x7f2560102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.266+0000 7f2566200700 1 --2- 172.21.15.90:0/597853475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560102690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2554009910 tx=0x7f2554009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 -- 172.21.15.90:0/597853475 shutdown_connections 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 --2- 172.21.15.90:0/597853475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 0x7f25601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 --2- 172.21.15.90:0/597853475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560102690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 -- 172.21.15.90:0/597853475 >> 172.21.15.90:0/597853475 conn(0x7f25600fd8d0 msgr2=0x7f25600ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 -- 172.21.15.90:0/597853475 shutdown_connections 2024-04-24T08:19:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.267+0000 7f2566200700 1 -- 172.21.15.90:0/597853475 wait complete. 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.268+0000 7f2566200700 1 Processor -- start 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.268+0000 7f2566200700 1 -- start start 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2566200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2566200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 0x7f2560197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2566200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601985f0 con 0x7f2560102280 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2566200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2560198730 con 0x7f2560103480 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52364/0 (socket says 172.21.15.90:52364) 2024-04-24T08:19:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 -- 172.21.15.90:0/148527082 learned_addr learned my addr 172.21.15.90:0/148527082 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 -- 172.21.15.90:0/148527082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 msgr2=0x7f2560197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f255fe00700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 0x7f2560197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 0x7f2560197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.269+0000 7f2564e00700 1 -- 172.21.15.90:0/148527082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540095f0 con 0x7f2560102280 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f2564e00700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f255400bda0 tx=0x7f255400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f255401d070 con 0x7f2560102280 2024-04-24T08:19:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2554004c50 con 0x7f2560102280 2024-04-24T08:19:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256019d180 con 0x7f2560102280 2024-04-24T08:19:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f255400f690 con 0x7f2560102280 2024-04-24T08:19:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.270+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256019d670 con 0x7f2560102280 2024-04-24T08:19:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.271+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256010add0 con 0x7f2560102280 2024-04-24T08:19:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.272+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f255400f900 con 0x7f2560102280 2024-04-24T08:19:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.273+0000 7f255d600700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 0x7f254806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.273+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f255408cdf0 con 0x7f2560102280 2024-04-24T08:19:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.273+0000 7f255fe00700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 0x7f254806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.274+0000 7f255fe00700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 0x7f254806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2550005950 tx=0x7f2550005cf0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:54.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.278+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f255405b200 con 0x7f2560102280 2024-04-24T08:19:54.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.501+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2560107dd0 con 0x7f254806c680 2024-04-24T08:19:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.507+0000 7f255d600700 1 -- 172.21.15.90:0/148527082 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f2560107dd0 con 0x7f254806c680 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter 2/2 2m ago 7m * 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:crash 2/2 2m ago 7m * 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 2/2 2m ago 8m count:2 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:mon 2/2 2m ago 6m smithi090:172.21.15.90=smithi090;smithi149:172.21.15.149=smithi149;count:2 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:osd 6 2m ago - 2024-04-24T08:19:54.508 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-04-24T08:19:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 msgr2=0x7f254806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 0x7f254806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2550005950 tx=0x7f2550005cf0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 msgr2=0x7f2560197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f255400bda0 tx=0x7f255400be80 comp rx=0 tx=0).stop 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 shutdown_connections 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f254806c680 0x7f254806eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2560102280 0x7f2560197a90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 --2- 172.21.15.90:0/148527082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560103480 0x7f2560197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 >> 172.21.15.90:0/148527082 conn(0x7f25600fd8d0 msgr2=0x7f25601066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 shutdown_connections 2024-04-24T08:19:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:54.510+0000 7f2566200700 1 -- 172.21.15.90:0/148527082 wait complete. 2024-04-24T08:19:55.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T08:19:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:55 smithi149 ceph-mon[31282]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:55 smithi149 ceph-mon[31282]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:55 smithi090 ceph-mon[25016]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:55 smithi090 ceph-mon[25016]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:19:55.827 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:57.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:57 smithi090 ceph-mon[25016]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:57.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 -- 172.21.15.90:0/4060042306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 msgr2=0x7f8f50101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:57.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 --2- 172.21.15.90:0/4060042306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f50101890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009910 tx=0x7f8f3c009c20 comp rx=0 tx=0).stop 2024-04-24T08:19:57.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 -- 172.21.15.90:0/4060042306 shutdown_connections 2024-04-24T08:19:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 --2- 172.21.15.90:0/4060042306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f50101890 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 --2- 172.21.15.90:0/4060042306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 -- 172.21.15.90:0/4060042306 >> 172.21.15.90:0/4060042306 conn(0x7f8f500fb7f0 msgr2=0x7f8f500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 -- 172.21.15.90:0/4060042306 shutdown_connections 2024-04-24T08:19:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.988+0000 7f8f56000700 1 -- 172.21.15.90:0/4060042306 wait complete. 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.989+0000 7f8f56000700 1 Processor -- start 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.989+0000 7f8f56000700 1 -- start start 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f56000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f56000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f56000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501987d0 con 0x7f8f50101440 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f56000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50198940 con 0x7f8f50100240 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52390/0 (socket says 172.21.15.90:52390) 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 -- 172.21.15.90:0/1515593268 learned_addr learned my addr 172.21.15.90:0/1515593268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4f400700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 -- 172.21.15.90:0/1515593268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 msgr2=0x7f8f50197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4ea00700 1 -- 172.21.15.90:0/1515593268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0095f0 con 0x7f8f50101440 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.990+0000 7f8f4f400700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f4ea00700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005b40 tx=0x7f8f3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3c01d070 con 0x7f8f50101440 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f3c022470 con 0x7f8f50101440 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5019d370 con 0x7f8f50101440 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3c00f690 con 0x7f8f50101440 2024-04-24T08:19:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.991+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5019d860 con 0x7f8f50101440 2024-04-24T08:19:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.992+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f30005320 con 0x7f8f50101440 2024-04-24T08:19:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.993+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8f3c004b70 con 0x7f8f50101440 2024-04-24T08:19:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.994+0000 7f8f47e00700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 0x7f8f3406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:19:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.994+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8f3c08cc10 con 0x7f8f50101440 2024-04-24T08:19:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.995+0000 7f8f4f400700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 0x7f8f3406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:19:57.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.995+0000 7f8f4f400700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 0x7f8f3406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005de0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:19:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:57.999+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8f3c057690 con 0x7f8f50101440 2024-04-24T08:19:58.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:57 smithi149 ceph-mon[31282]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:58.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.222+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f30000bf0 con 0x7f8f3406c680 2024-04-24T08:19:58.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.226+0000 7f8f47e00700 1 -- 172.21.15.90:0/1515593268 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f8f30000bf0 con 0x7f8f3406c680 2024-04-24T08:19:58.227 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-04-24T08:19:58.227 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-04-24T08:19:58.227 INFO:teuthology.orchestra.run.smithi090.stdout:smithi149 172.21.15.149 2024-04-24T08:19:58.227 INFO:teuthology.orchestra.run.smithi090.stdout:2 hosts in cluster 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 msgr2=0x7f8f3406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 0x7f8f3406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005de0 comp rx=0 tx=0).stop 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 msgr2=0x7f8f501981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005b40 tx=0x7f8f3c00be20 comp rx=0 tx=0).stop 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 shutdown_connections 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8f3406c680 0x7f8f3406eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:58.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f50100240 0x7f8f50197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:58.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 --2- 172.21.15.90:0/1515593268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f50101440 0x7f8f501981d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:19:58.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 >> 172.21.15.90:0/1515593268 conn(0x7f8f500fb7f0 msgr2=0x7f8f50102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:19:58.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 shutdown_connections 2024-04-24T08:19:58.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:19:58.229+0000 7f8f56000700 1 -- 172.21.15.90:0/1515593268 wait complete. 2024-04-24T08:19:59.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T08:19:59.390 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:19:59.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:59 smithi090 ceph-mon[25016]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:19:59.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:19:59 smithi090 ceph-mon[25016]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:59 smithi149 ceph-mon[31282]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:19:59 smithi149 ceph-mon[31282]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:00.732 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:00 smithi090 ceph-mon[25016]: overall HEALTH_OK 2024-04-24T08:20:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:00 smithi149 ceph-mon[31282]: overall HEALTH_OK 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 -- 172.21.15.90:0/1373356000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 msgr2=0x7f2668106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 --2- 172.21.15.90:0/1373356000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f2668106c50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2658009910 tx=0x7f2658009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 -- 172.21.15.90:0/1373356000 shutdown_connections 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 --2- 172.21.15.90:0/1373356000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f2668107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 --2- 172.21.15.90:0/1373356000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f2668106c50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 -- 172.21.15.90:0/1373356000 >> 172.21.15.90:0/1373356000 conn(0x7f2668075970 msgr2=0x7f2668077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.531+0000 7f266d600700 1 -- 172.21.15.90:0/1373356000 shutdown_connections 2024-04-24T08:20:01.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.532+0000 7f266d600700 1 -- 172.21.15.90:0/1373356000 wait complete. 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 Processor -- start 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 -- start start 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f266819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266819cd10 con 0x7f2668106840 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f266d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266819ce50 con 0x7f2668107a40 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f2666a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f2666a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52828/0 (socket says 172.21.15.90:52828) 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.533+0000 7f2666a00700 1 -- 172.21.15.90:0/767234512 learned_addr learned my addr 172.21.15.90:0/767234512 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:01.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f265fe00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f266819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f2666a00700 1 -- 172.21.15.90:0/767234512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 msgr2=0x7f266819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f2666a00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f266819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f2666a00700 1 -- 172.21.15.90:0/767234512 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26580095f0 con 0x7f2668106840 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f265fe00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f266819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f2666a00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f265800bd70 tx=0x7f265800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f265801d070 con 0x7f2668106840 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f265800f460 con 0x7f2668106840 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26681a18a0 con 0x7f2668106840 2024-04-24T08:20:01.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.534+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2658021640 con 0x7f2668106840 2024-04-24T08:20:01.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.535+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26681a1d90 con 0x7f2668106840 2024-04-24T08:20:01.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.536+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266810f410 con 0x7f2668106840 2024-04-24T08:20:01.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.538+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f265800fab0 con 0x7f2668106840 2024-04-24T08:20:01.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.539+0000 7f265f400700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 0x7f264c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:01.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.539+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f265808ce00 con 0x7f2668106840 2024-04-24T08:20:01.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.539+0000 7f265fe00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 0x7f264c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:01.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.540+0000 7f265fe00700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 0x7f264c06eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2650007900 tx=0x7f2650008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:01.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.542+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f265805b310 con 0x7f2668106840 2024-04-24T08:20:01.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.768+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f26681a2490 con 0x7f264c06c680 2024-04-24T08:20:01.769 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:01 smithi090 ceph-mon[25016]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:01.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.770+0000 7f265f400700 1 -- 172.21.15.90:0/767234512 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f26681a2490 con 0x7f264c06c680 2024-04-24T08:20:01.771 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T08:20:01.771 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 3m ago LVM detected, locked 2024-04-24T08:20:01.771 INFO:teuthology.orchestra.run.smithi090.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 86s ago LVM detected, locked 2024-04-24T08:20:01.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.773+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 msgr2=0x7f264c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:01.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.773+0000 7f266d600700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 0x7f264c06eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2650007900 tx=0x7f2650008040 comp rx=0 tx=0).stop 2024-04-24T08:20:01.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.773+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 msgr2=0x7f266819c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:01.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.773+0000 7f266d600700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f265800bd70 tx=0x7f265800be50 comp rx=0 tx=0).stop 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 shutdown_connections 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f264c06c680 0x7f264c06eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668106840 0x7f266819c1b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 --2- 172.21.15.90:0/767234512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2668107a40 0x7f266819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 >> 172.21.15.90:0/767234512 conn(0x7f2668075970 msgr2=0x7f266810ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 shutdown_connections 2024-04-24T08:20:01.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:01.774+0000 7f266d600700 1 -- 172.21.15.90:0/767234512 wait complete. 2024-04-24T08:20:01.928 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:01 smithi149 ceph-mon[31282]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:02.602 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:20:02.612 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:02.613 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-24T08:20:03.036 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:03 smithi090 ceph-mon[25016]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:03 smithi090 ceph-mon[25016]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:04.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:03 smithi149 ceph-mon[31282]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:04.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:03 smithi149 ceph-mon[31282]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 -- 172.21.15.90:0/2920814610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 msgr2=0x7f9220073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 --2- 172.21.15.90:0/2920814610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f9220073c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9208009910 tx=0x7f9208009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 -- 172.21.15.90:0/2920814610 shutdown_connections 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 --2- 172.21.15.90:0/2920814610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f9220073c20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 --2- 172.21.15.90:0/2920814610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 0x7f92200731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 -- 172.21.15.90:0/2920814610 >> 172.21.15.90:0/2920814610 conn(0x7f92200fb7f0 msgr2=0x7f92200fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 -- 172.21.15.90:0/2920814610 shutdown_connections 2024-04-24T08:20:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.230+0000 7f9225800700 1 -- 172.21.15.90:0/2920814610 wait complete. 2024-04-24T08:20:05.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.231+0000 7f9225800700 1 Processor -- start 2024-04-24T08:20:05.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f9225800700 1 -- start start 2024-04-24T08:20:05.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f9225800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:05.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f9225800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 0x7f922019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f9225800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922019cb40 con 0x7f92200737b0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f9225800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922019cc80 con 0x7f9220074d80 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f921ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f921ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52848/0 (socket says 172.21.15.90:52848) 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.232+0000 7f921ea00700 1 -- 172.21.15.90:0/893713435 learned_addr learned my addr 172.21.15.90:0/893713435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f9216000700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 0x7f922019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f921ea00700 1 -- 172.21.15.90:0/893713435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 msgr2=0x7f922019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f921ea00700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 0x7f922019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f921ea00700 1 -- 172.21.15.90:0/893713435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080095f0 con 0x7f92200737b0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f921ea00700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f921000cfd0 tx=0x7f921000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.233+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f921000ad70 con 0x7f92200737b0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.234+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92100128d0 con 0x7f92200737b0 2024-04-24T08:20:05.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.234+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92201a1740 con 0x7f92200737b0 2024-04-24T08:20:05.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.234+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9210011880 con 0x7f92200737b0 2024-04-24T08:20:05.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.234+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92201a1c60 con 0x7f92200737b0 2024-04-24T08:20:05.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.235+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922010f380 con 0x7f92200737b0 2024-04-24T08:20:05.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.236+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9210012420 con 0x7f92200737b0 2024-04-24T08:20:05.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.237+0000 7f9217e00700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 0x7f920406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:05.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.238+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f921008b720 con 0x7f92200737b0 2024-04-24T08:20:05.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.238+0000 7f9216000700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 0x7f920406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.238+0000 7f9216000700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 0x7f920406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9208006010 tx=0x7f920800b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:05.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.242+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9210059b30 con 0x7f92200737b0 2024-04-24T08:20:05.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:05.485+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f92201031e0 con 0x7f920406c680 2024-04-24T08:20:06.276 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:06 smithi090 ceph-mon[25016]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:06.276 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:06 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:06.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:06 smithi149 ceph-mon[31282]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:06.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:06 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:07.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.145+0000 7f9217e00700 1 -- 172.21.15.90:0/893713435 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f92201031e0 con 0x7f920406c680 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 msgr2=0x7f920406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 0x7f920406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9208006010 tx=0x7f920800b580 comp rx=0 tx=0).stop 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 msgr2=0x7f922019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f921000cfd0 tx=0x7f921000a340 comp rx=0 tx=0).stop 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 shutdown_connections 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f920406c680 0x7f920406eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92200737b0 0x7f922019c070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:07.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 --2- 172.21.15.90:0/893713435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9220074d80 0x7f922019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:07.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 >> 172.21.15.90:0/893713435 conn(0x7f92200fb7f0 msgr2=0x7f9220101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:07.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 shutdown_connections 2024-04-24T08:20:07.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:07.149+0000 7f9225800700 1 -- 172.21.15.90:0/893713435 wait complete. 2024-04-24T08:20:07.494 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:07.494 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T08:20:07.494 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:07.494 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T08:20:07.495 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:20:07.495 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T08:20:07.495 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:07 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[25012]: 2024-04-24T08:20:07.118+0000 7fa7a4200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:20:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:07 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T08:20:08.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T08:20:08.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: fsmap cephfs:0 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: Saving service mds.cephfs spec with placement count:4 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:08.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:08.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:08.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:08.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:08.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:08 smithi090 ceph-mon[25016]: Deploying daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:20:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T08:20:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: fsmap cephfs:0 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: Saving service mds.cephfs spec with placement count:4 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:08.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:08.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:08.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:08.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:08.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:08 smithi149 ceph-mon[31282]: Deploying daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:20:09.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:09 smithi090 ceph-mon[25016]: pgmap v221: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:09.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:09 smithi090 ceph-mon[25016]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T08:20:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:09 smithi149 ceph-mon[31282]: pgmap v221: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:09 smithi149 ceph-mon[31282]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T08:20:10.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:10 smithi149 ceph-mon[31282]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T08:20:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:10 smithi090 ceph-mon[25016]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T08:20:11.360 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:11 smithi090 ceph-mon[25016]: pgmap v224: 65 pgs: 10 creating+peering, 10 active+clean, 45 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:11.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:11 smithi149 ceph-mon[31282]: pgmap v224: 65 pgs: 10 creating+peering, 10 active+clean, 45 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:12.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.409+0000 7f7b08c00700 1 -- 172.21.15.90:0/4262953838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 msgr2=0x7f7b04107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.409+0000 7f7b08c00700 1 --2- 172.21.15.90:0/4262953838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b04107e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009910 tx=0x7f7af0009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 -- 172.21.15.90:0/4262953838 shutdown_connections 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 --2- 172.21.15.90:0/4262953838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b04107e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 --2- 172.21.15.90:0/4262953838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 0x7f7b04106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 -- 172.21.15.90:0/4262953838 >> 172.21.15.90:0/4262953838 conn(0x7f7b04075800 msgr2=0x7f7b04077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 -- 172.21.15.90:0/4262953838 shutdown_connections 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.410+0000 7f7b08c00700 1 -- 172.21.15.90:0/4262953838 wait complete. 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 Processor -- start 2024-04-24T08:20:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 -- start start 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 0x7f7b0419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0419cc10 con 0x7f7b04107a40 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b08c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0419cd50 con 0x7f7b04106840 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.411+0000 7f7b01400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60194/0 (socket says 172.21.15.90:60194) 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 -- 172.21.15.90:0/375505399 learned_addr learned my addr 172.21.15.90:0/375505399 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01e00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 0x7f7b0419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 -- 172.21.15.90:0/375505399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 msgr2=0x7f7b0419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:12.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 0x7f7b0419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 -- 172.21.15.90:0/375505399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af00095f0 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.412+0000 7f7b01400700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7af000bd70 tx=0x7f7af000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.413+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af001d070 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.413+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7af0005dc0 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.413+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b041a17a0 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.413+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af0021410 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.413+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b041a1c90 con 0x7f7b04107a40 2024-04-24T08:20:12.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.414+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0410f390 con 0x7f7b04107a40 2024-04-24T08:20:12.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.416+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7af000f7d0 con 0x7f7b04107a40 2024-04-24T08:20:12.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.417+0000 7f7afaa00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 0x7f7ae806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:12.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.417+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7af008cc20 con 0x7f7b04107a40 2024-04-24T08:20:12.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.417+0000 7f7b01e00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 0x7f7ae806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:12.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.418+0000 7f7b01e00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 0x7f7ae806eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b041078a0 tx=0x7f7af400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:12.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.422+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7af005acf0 con 0x7f7b04107a40 2024-04-24T08:20:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.695+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7b0410f580 con 0x7f7b04107a40 2024-04-24T08:20:12.699 INFO:teuthology.orchestra.run.smithi090.stdout:e4 2024-04-24T08:20:12.699 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:20:12.699 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 4 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:12.678825+0000 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:20:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14462} 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{0:14462} state up:creating seq 1 addr [v2:172.21.15.90:6826/3029914438,v1:172.21.15.90:6827/3029914438] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:20:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:12.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.696+0000 7f7afaa00700 1 -- 172.21.15.90:0/375505399 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7f7af000f460 con 0x7f7b04107a40 2024-04-24T08:20:12.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 msgr2=0x7f7ae806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:12.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 0x7f7ae806eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b041078a0 tx=0x7f7af400b410 comp rx=0 tx=0).stop 2024-04-24T08:20:12.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 msgr2=0x7f7b0419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:12.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7af000bd70 tx=0x7f7af000bda0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 shutdown_connections 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f7ae806c680 0x7f7ae806eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b04106840 0x7f7b0419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 --2- 172.21.15.90:0/375505399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04107a40 0x7f7b0419c5f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.701+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 >> 172.21.15.90:0/375505399 conn(0x7f7b04075800 msgr2=0x7f7b0410ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.702+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 shutdown_connections 2024-04-24T08:20:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:12.702+0000 7f7b08c00700 1 -- 172.21.15.90:0/375505399 wait complete. 2024-04-24T08:20:12.705 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: Deploying daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: pgmap v225: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.90:6826/3029914438,v1:172.21.15.90:6827/3029914438] up:boot 2024-04-24T08:20:13.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: daemon mds.cephfs.smithi090.qbyvpv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: Cluster is now healthy 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: fsmap cephfs:0 1 up:standby 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:creating} 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/375505399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:20:13.012 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:12 smithi149 ceph-mon[31282]: daemon mds.cephfs.smithi090.qbyvpv is now active in filesystem cephfs as rank 0 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: Deploying daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: pgmap v225: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.90:6826/3029914438,v1:172.21.15.90:6827/3029914438] up:boot 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: daemon mds.cephfs.smithi090.qbyvpv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T08:20:13.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: Cluster is now healthy 2024-04-24T08:20:13.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: fsmap cephfs:0 1 up:standby 2024-04-24T08:20:13.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:20:13.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:creating} 2024-04-24T08:20:13.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/375505399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:20:13.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:12 smithi090 ceph-mon[25016]: daemon mds.cephfs.smithi090.qbyvpv is now active in filesystem cephfs as rank 0 2024-04-24T08:20:13.546 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:20:13.557 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:13.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-24T08:20:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:14 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.90:6826/3029914438,v1:172.21.15.90:6827/3029914438] up:active 2024-04-24T08:20:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:14 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 2024-04-24T08:20:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:14 smithi090 ceph-mon[25016]: pgmap v226: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:14.894 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:14 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.90:6826/3029914438,v1:172.21.15.90:6827/3029914438] up:active 2024-04-24T08:20:14.894 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:14 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 2024-04-24T08:20:14.894 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:14 smithi149 ceph-mon[31282]: pgmap v226: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.259+0000 7f99a8000700 1 -- 172.21.15.90:0/1864965934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a00ff130 msgr2=0x7f99a00ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.259+0000 7f99a8000700 1 --2- 172.21.15.90:0/1864965934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a00ff130 0x7f99a00ff540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9994009910 tx=0x7f9994009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 -- 172.21.15.90:0/1864965934 shutdown_connections 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 --2- 172.21.15.90:0/1864965934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a01003d0 0x7f99a0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 --2- 172.21.15.90:0/1864965934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a00ff130 0x7f99a00ff540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 -- 172.21.15.90:0/1864965934 >> 172.21.15.90:0/1864965934 conn(0x7f99a00fa780 msgr2=0x7f99a00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 -- 172.21.15.90:0/1864965934 shutdown_connections 2024-04-24T08:20:16.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.260+0000 7f99a8000700 1 -- 172.21.15.90:0/1864965934 wait complete. 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 Processor -- start 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 -- start start 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a00ff130 0x7f99a0195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0196600 con 0x7f99a01003d0 2024-04-24T08:20:16.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.261+0000 7f99a8000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0196740 con 0x7f99a00ff130 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60212/0 (socket says 172.21.15.90:60212) 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 -- 172.21.15.90:0/3293117551 learned_addr learned my addr 172.21.15.90:0/3293117551 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a5600700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a00ff130 0x7f99a0195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 -- 172.21.15.90:0/3293117551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a00ff130 msgr2=0x7f99a0195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a00ff130 0x7f99a0195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.262+0000 7f99a4c00700 1 -- 172.21.15.90:0/3293117551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940095f0 con 0x7f99a01003d0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f99a4c00700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f998c00d8d0 tx=0x7f998c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998c009940 con 0x7f99a01003d0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a019b1f0 con 0x7f99a01003d0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f998c010460 con 0x7f99a01003d0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998c00f5f0 con 0x7f99a01003d0 2024-04-24T08:20:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.263+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a019b740 con 0x7f99a01003d0 2024-04-24T08:20:16.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.264+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0107c10 con 0x7f99a01003d0 2024-04-24T08:20:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.266+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f998c0105d0 con 0x7f99a01003d0 2024-04-24T08:20:16.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.267+0000 7f999e000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 0x7f998806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:16.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.267+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f998c08b780 con 0x7f99a01003d0 2024-04-24T08:20:16.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.267+0000 7f99a5600700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 0x7f998806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:16.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.268+0000 7f99a5600700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 0x7f998806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9994005b40 tx=0x7f9994005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:16.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.270+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f998c059ac0 con 0x7f99a01003d0 2024-04-24T08:20:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.562+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f99a019c1d0 con 0x7f99a01003d0 2024-04-24T08:20:16.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.835+0000 7f999e000700 1 -- 172.21.15.90:0/3293117551 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f998c059650 con 0x7f99a01003d0 2024-04-24T08:20:16.838 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:boot 2024-04-24T08:20:16.838 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:16.838 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:20:16.838 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: Deploying daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 3 op/s 2024-04-24T08:20:16.839 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:16 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3293117551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T08:20:16.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 msgr2=0x7f998806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:16.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 0x7f998806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9994005b40 tx=0x7f9994005a90 comp rx=0 tx=0).stop 2024-04-24T08:20:16.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 msgr2=0x7f99a0195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:16.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f998c00d8d0 tx=0x7f998c00dc90 comp rx=0 tx=0).stop 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 shutdown_connections 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f998806c680 0x7f998806eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a00ff130 0x7f99a0195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 --2- 172.21.15.90:0/3293117551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99a01003d0 0x7f99a0195fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.840+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 >> 172.21.15.90:0/3293117551 conn(0x7f99a00fa780 msgr2=0x7f99a0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.841+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 shutdown_connections 2024-04-24T08:20:16.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:16.841+0000 7f99a8000700 1 -- 172.21.15.90:0/3293117551 wait complete. 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:boot 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:17.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:17.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:17.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:17.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: Deploying daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:20:17.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 3 op/s 2024-04-24T08:20:17.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:16 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3293117551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T08:20:17.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:20:17.847 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:17.847 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-24T08:20:17.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:17 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3293117551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T08:20:17.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:17 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:18.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:17 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3293117551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T08:20:18.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:17 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby 2024-04-24T08:20:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:19.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:18 smithi090 ceph-mon[25016]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-24T08:20:19.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:18 smithi149 ceph-mon[31282]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-24T08:20:20.496 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:21.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:21 smithi090 ceph-mon[25016]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T08:20:21.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:21 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:21.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:21 smithi149 ceph-mon[31282]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T08:20:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:21 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 -- 172.21.15.90:0/1268083890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc100cd0 msgr2=0x7fe4dc1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1268083890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc1030b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009910 tx=0x7fe4cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 -- 172.21.15.90:0/1268083890 shutdown_connections 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1268083890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1268083890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc1030b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 -- 172.21.15.90:0/1268083890 >> 172.21.15.90:0/1268083890 conn(0x7fe4dc0fa6d0 msgr2=0x7fe4dc0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 -- 172.21.15.90:0/1268083890 shutdown_connections 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.525+0000 7fe4e2000700 1 -- 172.21.15.90:0/1268083890 wait complete. 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.526+0000 7fe4e2000700 1 Processor -- start 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.526+0000 7fe4e2000700 1 -- start start 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4e2000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4e2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4e2000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1987d0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4e2000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc19d1e0 con 0x7fe4dc100cd0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56798/0 (socket says 172.21.15.90:56798) 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4daa00700 1 -- 172.21.15.90:0/1112773963 learned_addr learned my addr 172.21.15.90:0/1112773963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.527+0000 7fe4db400700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4daa00700 1 -- 172.21.15.90:0/1112773963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc100cd0 msgr2=0x7fe4dc197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4daa00700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4daa00700 1 -- 172.21.15.90:0/1112773963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc0095f0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4daa00700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d8d0 tx=0x7fe4c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c8009940 con 0x7fe4dc1035f0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8010460 con 0x7fe4dc1035f0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c800f5f0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.528+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc19d3e0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.529+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc19d900 con 0x7fe4dc1035f0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.530+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc0fc2b0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.531+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe4c80105d0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.532+0000 7fe4d3e00700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 0x7fe4c006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.533+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe4c808b7b0 con 0x7fe4dc1035f0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.533+0000 7fe4db400700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 0x7fe4c006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.534+0000 7fe4db400700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 0x7fe4c006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc006010 tx=0x7fe4cc00b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.536+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe4c8055f10 con 0x7fe4dc1035f0 2024-04-24T08:20:21.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:21.825+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe4dc02cc30 con 0x7fe4dc1035f0 2024-04-24T08:20:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.092+0000 7fe4d3e00700 1 -- 172.21.15.90:0/1112773963 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe4c805de60 con 0x7fe4dc1035f0 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.095+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 msgr2=0x7fe4c006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.095+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 0x7fe4c006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc006010 tx=0x7fe4cc00b580 comp rx=0 tx=0).stop 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.095+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 msgr2=0x7fe4dc1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.095+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d8d0 tx=0x7fe4c800dc90 comp rx=0 tx=0).stop 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 shutdown_connections 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe4c006c680 0x7fe4c006eb30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4dc100cd0 0x7fe4dc197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 --2- 172.21.15.90:0/1112773963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4dc1035f0 0x7fe4dc1981b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 >> 172.21.15.90:0/1112773963 conn(0x7fe4dc0fa6d0 msgr2=0x7fe4dc104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 shutdown_connections 2024-04-24T08:20:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:22.096+0000 7fe4e2000700 1 -- 172.21.15.90:0/1112773963 wait complete. 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: Deploying daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:20:22.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1112773963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] up:boot 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1112773963' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 2 up:standby 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:20:22.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:22 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 1 up:standby 2024-04-24T08:20:22.848 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.848 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: Deploying daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1112773963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] up:boot 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1112773963' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 2 up:standby 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:20:22.849 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:22 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 1 up:standby 2024-04-24T08:20:23.015 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:20:23.025 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:23.025 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-24T08:20:23.444 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:25.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:25 smithi090 ceph-mon[25016]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-24T08:20:25.378 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:25 smithi149 ceph-mon[31282]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-24T08:20:25.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.634+0000 7f3618e00700 1 -- 172.21.15.90:0/1111249815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 msgr2=0x7f36140731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.634+0000 7f3618e00700 1 --2- 172.21.15.90:0/1111249815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f36140731e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3604009910 tx=0x7f3604009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 -- 172.21.15.90:0/1111249815 shutdown_connections 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 --2- 172.21.15.90:0/1111249815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f3614073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 --2- 172.21.15.90:0/1111249815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f36140731e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 -- 172.21.15.90:0/1111249815 >> 172.21.15.90:0/1111249815 conn(0x7f36140fb7f0 msgr2=0x7f36140fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 -- 172.21.15.90:0/1111249815 shutdown_connections 2024-04-24T08:20:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.635+0000 7f3618e00700 1 -- 172.21.15.90:0/1111249815 wait complete. 2024-04-24T08:20:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.636+0000 7f3618e00700 1 Processor -- start 2024-04-24T08:20:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.636+0000 7f3618e00700 1 -- start start 2024-04-24T08:20:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3618e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f361419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3618e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3618e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361419cc50 con 0x7f3614074d80 2024-04-24T08:20:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3618e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361419cd90 con 0x7f36140737b0 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3611400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3611400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56832/0 (socket says 172.21.15.90:56832) 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3611400700 1 -- 172.21.15.90:0/607659424 learned_addr learned my addr 172.21.15.90:0/607659424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.637+0000 7f3611e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f361419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.638+0000 7f3611400700 1 -- 172.21.15.90:0/607659424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 msgr2=0x7f361419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.638+0000 7f3611400700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f361419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.638+0000 7f3611400700 1 -- 172.21.15.90:0/607659424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040095f0 con 0x7f3614074d80 2024-04-24T08:20:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.638+0000 7f3611e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f361419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f3611400700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f360000b4b0 tx=0x7f360000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3600011800 con 0x7f3614074d80 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36141a1840 con 0x7f3614074d80 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3600011e10 con 0x7f3614074d80 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360000f2e0 con 0x7f3614074d80 2024-04-24T08:20:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.639+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36141a1d90 con 0x7f3614074d80 2024-04-24T08:20:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.640+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361410f3f0 con 0x7f3614074d80 2024-04-24T08:20:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.642+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3600010380 con 0x7f3614074d80 2024-04-24T08:20:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.642+0000 7f360aa00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 0x7f35f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.643+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f360008b590 con 0x7f3614074d80 2024-04-24T08:20:25.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.643+0000 7f3611e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 0x7f35f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:25.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.644+0000 7f3611e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 0x7f35f806eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3604006010 tx=0x7f360400b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:25.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.648+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3600059660 con 0x7f3614074d80 2024-04-24T08:20:25.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:25.920+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f361410f5e0 con 0x7f3614074d80 2024-04-24T08:20:26.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.089+0000 7f360aa00700 1 -- 172.21.15.90:0/607659424 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7f36000591f0 con 0x7f3614074d80 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.094+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 msgr2=0x7f35f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 0x7f35f806eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3604006010 tx=0x7f360400b560 comp rx=0 tx=0).stop 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 msgr2=0x7f361419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f360000b4b0 tx=0x7f360000b870 comp rx=0 tx=0).stop 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 shutdown_connections 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f806c680 0x7f35f806eb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36140737b0 0x7f361419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 --2- 172.21.15.90:0/607659424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3614074d80 0x7f361419c630 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 >> 172.21.15.90:0/607659424 conn(0x7f36140fb7f0 msgr2=0x7f3614101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.095+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 shutdown_connections 2024-04-24T08:20:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:26.096+0000 7f3618e00700 1 -- 172.21.15.90:0/607659424 wait complete. 2024-04-24T08:20:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:inline data enabled 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/607659424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T08:20:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-24T08:20:26.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:boot 2024-04-24T08:20:26.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/607659424' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T08:20:26.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:26.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:26 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:20:26.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/607659424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:boot 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/607659424' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:26.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:26 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:20:26.964 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:20:26.978 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:26.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T08:20:27.678 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:27 smithi090 ceph-mon[25016]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T08:20:28.024 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:27 smithi149 ceph-mon[31282]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T08:20:29.737 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s 2024-04-24T08:20:29.737 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:standby-replay 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: Dropping low affinity active daemon mds.cephfs.smithi090.qbyvpv in favor of higher affinity standby. 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: Replacing daemon mds.cephfs.smithi090.qbyvpv as rank 0 with standby daemon mds.cephfs.smithi149.eazcmc 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:replay} 2 up:standby 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:29.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:29 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:standby-replay 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: Dropping low affinity active daemon mds.cephfs.smithi090.qbyvpv in favor of higher affinity standby. 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: Replacing daemon mds.cephfs.smithi090.qbyvpv as rank 0 with standby daemon mds.cephfs.smithi149.eazcmc 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T08:20:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:replay} 2 up:standby 2024-04-24T08:20:30.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:30.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:29 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 -- 172.21.15.90:0/2881430447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 msgr2=0x7fd190105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 --2- 172.21.15.90:0/2881430447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd190105aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd178009910 tx=0x7fd178009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 -- 172.21.15.90:0/2881430447 shutdown_connections 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 --2- 172.21.15.90:0/2881430447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd190105aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 --2- 172.21.15.90:0/2881430447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 0x7fd190103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 -- 172.21.15.90:0/2881430447 >> 172.21.15.90:0/2881430447 conn(0x7fd1900fa780 msgr2=0x7fd1900fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 -- 172.21.15.90:0/2881430447 shutdown_connections 2024-04-24T08:20:30.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.638+0000 7fd195000700 1 -- 172.21.15.90:0/2881430447 wait complete. 2024-04-24T08:20:30.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.639+0000 7fd195000700 1 Processor -- start 2024-04-24T08:20:30.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.639+0000 7fd195000700 1 -- start start 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd195000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 0x7fd190100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd195000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd195000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900ff290 con 0x7fd1901036c0 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd195000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900ff3d0 con 0x7fd190100da0 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd187e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd187e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52186/0 (socket says 172.21.15.90:52186) 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd187e00700 1 -- 172.21.15.90:0/3719356633 learned_addr learned my addr 172.21.15.90:0/3719356633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.640+0000 7fd18de00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 0x7fd190100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd187e00700 1 -- 172.21.15.90:0/3719356633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 msgr2=0x7fd190100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd187e00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 0x7fd190100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd187e00700 1 -- 172.21.15.90:0/3719356633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780095f0 con 0x7fd1901036c0 2024-04-24T08:20:30.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd187e00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd178005b40 tx=0x7fd17800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17801d070 con 0x7fd1901036c0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd17800f460 con 0x7fd1901036c0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900ff650 con 0x7fd1901036c0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.641+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd178021640 con 0x7fd1901036c0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.642+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900ffb40 con 0x7fd1901036c0 2024-04-24T08:20:30.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.643+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19018fbe0 con 0x7fd1901036c0 2024-04-24T08:20:30.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.645+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fd17800fa90 con 0x7fd1901036c0 2024-04-24T08:20:30.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.646+0000 7fd186a00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 0x7fd17406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:30.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.646+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd17808cc90 con 0x7fd1901036c0 2024-04-24T08:20:30.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.646+0000 7fd18de00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 0x7fd17406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:30.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.647+0000 7fd18de00700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 0x7fd17406eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd180005950 tx=0x7fd1800058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:30.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.651+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd17805ad00 con 0x7fd1901036c0 2024-04-24T08:20:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] up:boot 2024-04-24T08:20:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:reconnect 2024-04-24T08:20:30.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:standby 2024-04-24T08:20:30.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] up:standby 2024-04-24T08:20:30.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:reconnect} 3 up:standby 2024-04-24T08:20:30.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:20:30.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:30 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.921+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd19002d0c0 con 0x7fd1901036c0 2024-04-24T08:20:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.922+0000 7fd186a00700 1 -- 172.21.15.90:0/3719356633 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fd178026750 con 0x7fd1901036c0 2024-04-24T08:20:30.923 INFO:teuthology.orchestra.run.smithi090.stdout:e15 2024-04-24T08:20:30.923 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 15 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:30.575945+0000 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:20:30.924 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:20:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:rejoin seq 6 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{-1:14482} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:20:30.926 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:20:30.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 msgr2=0x7fd17406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:30.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 0x7fd17406eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd180005950 tx=0x7fd1800058e0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 msgr2=0x7fd1900fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd178005b40 tx=0x7fd17800be20 comp rx=0 tx=0).stop 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 shutdown_connections 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd17406c680 0x7fd17406eb30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd190100da0 0x7fd190100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 --2- 172.21.15.90:0/3719356633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901036c0 0x7fd1900fed50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.924+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 >> 172.21.15.90:0/3719356633 conn(0x7fd1900fa780 msgr2=0x7fd1900fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.925+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 shutdown_connections 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:30.925+0000 7fd195000700 1 -- 172.21.15.90:0/3719356633 wait complete. 2024-04-24T08:20:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 15 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] up:boot 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:reconnect 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:standby 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] up:standby 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:reconnect} 3 up:standby 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:20:31.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:30 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:31.843 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:31 smithi090 ceph-mon[25016]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-24T08:20:31.843 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:31 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:rejoin 2024-04-24T08:20:31.843 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:31 smithi090 ceph-mon[25016]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:rejoin} 3 up:standby 2024-04-24T08:20:31.843 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:31 smithi090 ceph-mon[25016]: daemon mds.cephfs.smithi149.eazcmc is now active in filesystem cephfs as rank 0 2024-04-24T08:20:31.843 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:31 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3719356633' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:20:31.881 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-24T08:20:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:31 smithi149 ceph-mon[31282]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-24T08:20:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:31 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:rejoin 2024-04-24T08:20:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:31 smithi149 ceph-mon[31282]: fsmap cephfs:1/1 {0=cephfs.smithi149.eazcmc=up:rejoin} 3 up:standby 2024-04-24T08:20:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:31 smithi149 ceph-mon[31282]: daemon mds.cephfs.smithi149.eazcmc is now active in filesystem cephfs as rank 0 2024-04-24T08:20:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:31 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3719356633' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:20:32.321 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:32 smithi090 ceph-mon[25016]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:20:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:32 smithi090 ceph-mon[25016]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:active 2024-04-24T08:20:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:32 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:20:32.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:32 smithi090 ceph-mon[25016]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:32.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:32 smithi090 ceph-mon[25016]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-24T08:20:32.963 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:32 smithi149 ceph-mon[31282]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:20:32.963 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:32 smithi149 ceph-mon[31282]: mds.? [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] up:active 2024-04-24T08:20:32.963 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:32 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:20:32.963 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:32 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:20:32.963 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:32 smithi149 ceph-mon[31282]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-24T08:20:35.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:35 smithi090 ceph-mon[25016]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-24T08:20:35.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:35.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:35 smithi149 ceph-mon[31282]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-24T08:20:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:35.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 -- 172.21.15.90:0/2955141156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 msgr2=0x7f3830101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 --2- 172.21.15.90:0/2955141156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830101cb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3820009910 tx=0x7f3820009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 -- 172.21.15.90:0/2955141156 shutdown_connections 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 --2- 172.21.15.90:0/2955141156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 0x7f38300fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 --2- 172.21.15.90:0/2955141156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830101cb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 -- 172.21.15.90:0/2955141156 >> 172.21.15.90:0/2955141156 conn(0x7f38300f9f10 msgr2=0x7f38300fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:35.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.613+0000 7f3834a00700 1 -- 172.21.15.90:0/2955141156 shutdown_connections 2024-04-24T08:20:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.614+0000 7f3834a00700 1 -- 172.21.15.90:0/2955141156 wait complete. 2024-04-24T08:20:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 Processor -- start 2024-04-24T08:20:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 -- start start 2024-04-24T08:20:35.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 0x7f3830072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300729a0 con 0x7f38301018a0 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.615+0000 7f3834a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300ff850 con 0x7f38301021f0 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52210/0 (socket says 172.21.15.90:52210) 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 -- 172.21.15.90:0/2146462327 learned_addr learned my addr 172.21.15.90:0/2146462327 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382d400700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 0x7f3830072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 -- 172.21.15.90:0/2146462327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 msgr2=0x7f3830072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 0x7f3830072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 -- 172.21.15.90:0/2146462327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200095f0 con 0x7f38301018a0 2024-04-24T08:20:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.616+0000 7f382de00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3820005b40 tx=0x7f382000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.617+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382001d070 con 0x7f38301018a0 2024-04-24T08:20:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.617+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3820022470 con 0x7f38301018a0 2024-04-24T08:20:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.617+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38300ffad0 con 0x7f38301018a0 2024-04-24T08:20:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.617+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382000f690 con 0x7f38301018a0 2024-04-24T08:20:35.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.617+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38300fffc0 con 0x7f38301018a0 2024-04-24T08:20:35.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.618+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3830108db0 con 0x7f38301018a0 2024-04-24T08:20:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.620+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f382000f880 con 0x7f38301018a0 2024-04-24T08:20:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.621+0000 7f3826a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 0x7f381406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.621+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f382008d100 con 0x7f38301018a0 2024-04-24T08:20:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.621+0000 7f382d400700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 0x7f381406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.622+0000 7f382d400700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 0x7f381406eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.626+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3820057870 con 0x7f38301018a0 2024-04-24T08:20:35.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.907+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3830108fd0 con 0x7f38301018a0 2024-04-24T08:20:35.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.908+0000 7f3826a00700 1 -- 172.21.15.90:0/2146462327 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f382005ae90 con 0x7f38301018a0 2024-04-24T08:20:35.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 msgr2=0x7f381406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:35.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 0x7f381406eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).stop 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 msgr2=0x7f3830071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3820005b40 tx=0x7f382000be20 comp rx=0 tx=0).stop 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 shutdown_connections 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f381406c680 0x7f381406eb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38301018a0 0x7f3830071e40 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3820005b40 tx=0x7f382000be20 comp rx=0 tx=0).stop 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 --2- 172.21.15.90:0/2146462327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38301021f0 0x7f3830072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 >> 172.21.15.90:0/2146462327 conn(0x7f38300f9f10 msgr2=0x7f3830104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:35.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 shutdown_connections 2024-04-24T08:20:35.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:35.911+0000 7f3834a00700 1 -- 172.21.15.90:0/2146462327 wait complete. 2024-04-24T08:20:35.913 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:20:35.929 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:20:36.270 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:36 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2146462327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:20:36.462 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:36 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2146462327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:20:36.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-24T08:20:37.128 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T08:20:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:37.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:37.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:37 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:37 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:20:39.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:39 smithi090 ceph-mon[25016]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T08:20:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:39 smithi149 ceph-mon[31282]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 -- 172.21.15.90:0/2189441730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088103640 msgr2=0x7fe088103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 --2- 172.21.15.90:0/2189441730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088103640 0x7fe088103a90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe074009910 tx=0x7fe074009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 -- 172.21.15.90:0/2189441730 shutdown_connections 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 --2- 172.21.15.90:0/2189441730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088103640 0x7fe088103a90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 --2- 172.21.15.90:0/2189441730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088102440 0x7fe088102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 -- 172.21.15.90:0/2189441730 >> 172.21.15.90:0/2189441730 conn(0x7fe0880fd9f0 msgr2=0x7fe0880ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 -- 172.21.15.90:0/2189441730 shutdown_connections 2024-04-24T08:20:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.669+0000 7fe08d200700 1 -- 172.21.15.90:0/2189441730 wait complete. 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.670+0000 7fe08d200700 1 Processor -- start 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.670+0000 7fe08d200700 1 -- start start 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe08d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe08d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088103640 0x7fe0881981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe08d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0881987d0 con 0x7fe088102440 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe08d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088198940 con 0x7fe088103640 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52222/0 (socket says 172.21.15.90:52222) 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 -- 172.21.15.90:0/527049259 learned_addr learned my addr 172.21.15.90:0/527049259 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 -- 172.21.15.90:0/527049259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088103640 msgr2=0x7fe0881981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088103640 0x7fe0881981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.671+0000 7fe085e00700 1 -- 172.21.15.90:0/527049259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740095f0 con 0x7fe088102440 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe085e00700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe07800d900 tx=0x7fe07800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0780041f0 con 0x7fe088102440 2024-04-24T08:20:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe078004350 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08819d3d0 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe078003da0 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.672+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08819d920 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.673+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08810af90 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.674+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe078009730 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.675+0000 7fe07ea00700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 0x7fe06c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.675+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe07808ba60 con 0x7fe088102440 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.676+0000 7fe085400700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 0x7fe06c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.676+0000 7fe085400700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 0x7fe06c06eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b5c0 tx=0x7fe074005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.680+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe0780560c0 con 0x7fe088102440 2024-04-24T08:20:39.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.978+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fe08810b180 con 0x7fe088102440 2024-04-24T08:20:39.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.979+0000 7fe07ea00700 1 -- 172.21.15.90:0/527049259 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7fe08810b180 con 0x7fe088102440 2024-04-24T08:20:39.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 msgr2=0x7fe06c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:39.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 0x7fe06c06eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b5c0 tx=0x7fe074005fd0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 msgr2=0x7fe088197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:39.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe07800d900 tx=0x7fe07800dcc0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 shutdown_connections 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe06c06c680 0x7fe06c06eb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe088102440 0x7fe088197c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 --2- 172.21.15.90:0/527049259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088103640 0x7fe0881981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.981+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 >> 172.21.15.90:0/527049259 conn(0x7fe0880fd9f0 msgr2=0x7fe088106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.982+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 shutdown_connections 2024-04-24T08:20:39.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:39.982+0000 7fe08d200700 1 -- 172.21.15.90:0/527049259 wait complete. 2024-04-24T08:20:39.996 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:20:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:40 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/527049259' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T08:20:40.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:40.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:40 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:40 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/527049259' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T08:20:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:40 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:40.802 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-24T08:20:40.817 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.022+0000 7f1e19a00640 1 -- 172.21.15.90:0/242727911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e140742a0 msgr2=0x7f1e140726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 --2- 172.21.15.90:0/242727911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e140742a0 0x7f1e140726f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009c20 tx=0x7f1e0402c1b0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 -- 172.21.15.90:0/242727911 shutdown_connections 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 --2- 172.21.15.90:0/242727911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e14072cc0 0x7f1e14073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 --2- 172.21.15.90:0/242727911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e140742a0 0x7f1e140726f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 -- 172.21.15.90:0/242727911 >> 172.21.15.90:0/242727911 conn(0x7f1e140faa50 msgr2=0x7f1e140fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 -- 172.21.15.90:0/242727911 shutdown_connections 2024-04-24T08:20:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.023+0000 7f1e19a00640 1 -- 172.21.15.90:0/242727911 wait complete. 2024-04-24T08:20:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 Processor -- start 2024-04-24T08:20:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 -- start start 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e140742a0 0x7f1e1419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419bf40 con 0x7f1e14072cc0 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e19a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419c0b0 con 0x7f1e140742a0 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e13e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e13e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40974/0 (socket says 172.21.15.90:40974) 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.024+0000 7f1e13e00640 1 -- 172.21.15.90:0/4049449167 learned_addr learned my addr 172.21.15.90:0/4049449167 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e0b400640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e140742a0 0x7f1e1419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e13e00640 1 -- 172.21.15.90:0/4049449167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e140742a0 msgr2=0x7f1e1419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e13e00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e140742a0 0x7f1e1419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e13e00640 1 -- 172.21.15.90:0/4049449167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009900 con 0x7f1e14072cc0 2024-04-24T08:20:41.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e13e00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e04000f80 tx=0x7f1e04032af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e04032ed0 con 0x7f1e14072cc0 2024-04-24T08:20:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1e04003e20 con 0x7f1e14072cc0 2024-04-24T08:20:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e141a0ae0 con 0x7f1e14072cc0 2024-04-24T08:20:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0403f440 con 0x7f1e14072cc0 2024-04-24T08:20:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.025+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e141a0fe0 con 0x7f1e14072cc0 2024-04-24T08:20:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.026+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e14195390 con 0x7f1e14072cc0 2024-04-24T08:20:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.027+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1e0403b040 con 0x7f1e14072cc0 2024-04-24T08:20:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.028+0000 7f1e11600640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 0x7f1df806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.028+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1e040ae750 con 0x7f1e14072cc0 2024-04-24T08:20:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.028+0000 7f1e0b400640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 0x7f1df806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.029+0000 7f1e0b400640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 0x7f1df806eb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc005fd0 tx=0x7f1dfc005c70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.032+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1e04078d30 con 0x7f1e14072cc0 2024-04-24T08:20:41.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.231+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e1410e760 con 0x7f1e14072cc0 2024-04-24T08:20:41.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.233+0000 7f1e11600640 1 -- 172.21.15.90:0/4049449167 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15105 (secure 0 0 0) 0x7f1e0407c350 con 0x7f1e14072cc0 2024-04-24T08:20:41.234 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:41.234 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":47,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:20:28.563621+0000","last_up_change":"2024-04-24T08:18:31.586535+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T08:20:06.040754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T08:20:06.663119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6803","nonce":3550641031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6805","nonce":3550641031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6809","nonce":3550641031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3550641031},{"type":"v1","addr":"172.21.15.90:6807","nonce":3550641031}]},"public_addr":"172.21.15.90:6803/3550641031","cluster_addr":"172.21.15.90:6805/3550641031","heartbeat_back_addr":"172.21.15.90:6809/3550641031","heartbeat_front_addr":"172.21.15.90:6807/3550641031","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6811","nonce":2560478642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6813","nonce":2560478642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6817","nonce":2560478642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2560478642},{"type":"v1","addr":"172.21.15.90:6815","nonce":2560478642}]},"public_addr":"172.21.15.90:6811/2560478642","cluster_addr":"172.21.15.90:6813/2560478642","heartbeat_back_addr":"172.21.15.90:6817/2560478642","heartbeat_front_addr":"172.21.15.90:6815/2560478642","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6819","nonce":358354662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6821","nonce":358354662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6825","nonce":358354662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":358354662},{"type":"v1","addr":"172.21.15.90:6823","nonce":358354662}]},"public_addr":"172.21.15.90:6819/358354662","cluster_addr":"172.21.15.90:6821/358354662","heartbeat_back_addr":"172.21.15.90:6825/358354662","heartbeat_front_addr":"172.21.15.90:6823/358354662","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6801","nonce":1453378074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6803","nonce":1453378074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6807","nonce":1453378074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1453378074},{"type":"v1","addr":"172.21.15.149:6805","nonce":1453378074}]},"public_addr":"172.21.15.149:6801/1453378074","cluster_addr":"172.21.15.149:6803/1453378074","heartbeat_back_addr":"172.21.15.149:6807/1453378074","heartbeat_front_addr":"172.21.15.149:6805/1453378074","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6809","nonce":197987858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6811","nonce":197987858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6815","nonce":197987858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":197987858},{"type":"v1","addr":"172.21.15.149:6813","nonce":197987858}]},"public_addr":"172.21.15.149:6809/197987858","cluster_addr":"172.21.15.149:6811/197987858","heartbeat_back_addr":"172.21.15.149:6815/197987858","heartbeat_front_addr":"172.21.15.149:6813/197987858","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6817","nonce":142618710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6819","nonce":142618710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6823","nonce":142618710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":142618710},{"type":"v1","addr":"172.21.15.149:6821","nonce":142618710}]},"public_addr":"172.21.15.149:6817/142618710","cluster_addr":"172.21.15.149:6819/142618710","heartbeat_back_addr":"172.21.15.149:6823/142618710","heartbeat_front_addr":"172.21.15.149:6821/142618710","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6827/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:6826/3029914438":"2024-04-25T08:20:28.563578+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:20:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 msgr2=0x7f1df806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 0x7f1df806eb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc005fd0 tx=0x7f1dfc005c70 comp rx=0 tx=0).stop 2024-04-24T08:20:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 msgr2=0x7f1e1419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e04000f80 tx=0x7f1e04032af0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 shutdown_connections 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f1df806c6b0 0x7f1df806eb60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e140742a0 0x7f1e1419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 --2- 172.21.15.90:0/4049449167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e14072cc0 0x7f1e1419b430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 >> 172.21.15.90:0/4049449167 conn(0x7f1e140faa50 msgr2=0x7f1e140fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 shutdown_connections 2024-04-24T08:20:41.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.235+0000 7f1e19a00640 1 -- 172.21.15.90:0/4049449167 wait complete. 2024-04-24T08:20:41.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:16:48.618419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T08:20:06.040754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T08:20:06.663119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:20:41.256 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 -- 172.21.15.90:0/1045035452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 msgr2=0x7fdc541007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1045035452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541007a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009880 tx=0x7fdc3c02bce0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 -- 172.21.15.90:0/1045035452 shutdown_connections 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1045035452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541007a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1045035452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc540fd730 0x7fdc540fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 -- 172.21.15.90:0/1045035452 >> 172.21.15.90:0/1045035452 conn(0x7fdc540f90d0 msgr2=0x7fdc540fb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 -- 172.21.15.90:0/1045035452 shutdown_connections 2024-04-24T08:20:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.444+0000 7fdc53e00640 1 -- 172.21.15.90:0/1045035452 wait complete. 2024-04-24T08:20:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 Processor -- start 2024-04-24T08:20:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 -- start start 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc540fd730 0x7fdc54194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc541958c0 con 0x7fdc540fe080 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.445+0000 7fdc53e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54195a30 con 0x7fdc540fd730 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40984/0 (socket says 172.21.15.90:40984) 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 -- 172.21.15.90:0/1365081431 learned_addr learned my addr 172.21.15.90:0/1365081431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 -- 172.21.15.90:0/1365081431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc540fd730 msgr2=0x7fdc54194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc540fd730 0x7fdc54194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 -- 172.21.15.90:0/1365081431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009560 con 0x7fdc540fe080 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.446+0000 7fdc4a000640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c02c220 tx=0x7fdc3c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.447+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3c005000 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.447+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c005610 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.447+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5419a450 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.447+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3c034670 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.447+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5419a950 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.448+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc541080f0 con 0x7fdc540fe080 2024-04-24T08:20:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.449+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fdc3c0039d0 con 0x7fdc540fe080 2024-04-24T08:20:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.450+0000 7fdc58a00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 0x7fdc3806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.450+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdc3c0aded0 con 0x7fdc540fe080 2024-04-24T08:20:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.450+0000 7fdc52a00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 0x7fdc3806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.451+0000 7fdc52a00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 0x7fdc3806eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc54196280 tx=0x7fdc44005bf0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.454+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdc3c078530 con 0x7fdc540fe080 2024-04-24T08:20:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:41 smithi149 ceph-mon[31282]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:41 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/4049449167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:20:41.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:41 smithi090 ceph-mon[25016]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:41.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:41 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/4049449167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:20:41.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.648+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc5419abc0 con 0x7fdc540fe080 2024-04-24T08:20:41.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.649+0000 7fdc58a00640 1 -- 172.21.15.90:0/1365081431 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc3c041020 con 0x7fdc540fe080 2024-04-24T08:20:41.650 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.651+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 msgr2=0x7fdc3806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.651+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 0x7fdc3806eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc54196280 tx=0x7fdc44005bf0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 msgr2=0x7fdc541952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c02c220 tx=0x7fdc3c004f00 comp rx=0 tx=0).stop 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 shutdown_connections 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fdc3806c5f0 0x7fdc3806eaa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc540fe080 0x7fdc541952f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 --2- 172.21.15.90:0/1365081431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc540fd730 0x7fdc54194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 >> 172.21.15.90:0/1365081431 conn(0x7fdc540f90d0 msgr2=0x7fdc541039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 shutdown_connections 2024-04-24T08:20:41.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.652+0000 7fdc53e00640 1 -- 172.21.15.90:0/1365081431 wait complete. 2024-04-24T08:20:41.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T08:20:41.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 -- 172.21.15.90:0/4134763412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9081016a0 msgr2=0x7fd908101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 --2- 172.21.15.90:0/4134763412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9081016a0 0x7fd908101ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009880 tx=0x7fd8fc02bce0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 -- 172.21.15.90:0/4134763412 shutdown_connections 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 --2- 172.21.15.90:0/4134763412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd908102890 0x7fd908102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 --2- 172.21.15.90:0/4134763412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9081016a0 0x7fd908101ab0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.871+0000 7fd90ee00640 1 -- 172.21.15.90:0/4134763412 >> 172.21.15.90:0/4134763412 conn(0x7fd9080fcc30 msgr2=0x7fd9080ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.872+0000 7fd90ee00640 1 -- 172.21.15.90:0/4134763412 shutdown_connections 2024-04-24T08:20:41.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.872+0000 7fd90ee00640 1 -- 172.21.15.90:0/4134763412 wait complete. 2024-04-24T08:20:41.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 Processor -- start 2024-04-24T08:20:41.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 -- start start 2024-04-24T08:20:41.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 0x7fd908196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908197ac0 con 0x7fd908102890 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908197c30 con 0x7fd9081016a0 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41016/0 (socket says 172.21.15.90:41016) 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.873+0000 7fd90d000640 1 -- 172.21.15.90:0/3844887119 learned_addr learned my addr 172.21.15.90:0/3844887119 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90da00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 0x7fd908196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90d000640 1 -- 172.21.15.90:0/3844887119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 msgr2=0x7fd908196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90d000640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 0x7fd908196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90d000640 1 -- 172.21.15.90:0/3844887119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc009560 con 0x7fd908102890 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90da00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 0x7fd908196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:20:41.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90d000640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d6f0 tx=0x7fd8f400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f4009a30 con 0x7fd908102890 2024-04-24T08:20:41.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90819c6c0 con 0x7fd908102890 2024-04-24T08:20:41.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd8f40054d0 con 0x7fd908102890 2024-04-24T08:20:41.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f4010460 con 0x7fd908102890 2024-04-24T08:20:41.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.874+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90819cc20 con 0x7fd908102890 2024-04-24T08:20:41.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.876+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90810a3d0 con 0x7fd908102890 2024-04-24T08:20:41.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.877+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fd8f4009b90 con 0x7fd908102890 2024-04-24T08:20:41.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.878+0000 7fd906000640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 0x7fd8f006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:41.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.879+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd8f408b520 con 0x7fd908102890 2024-04-24T08:20:41.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.879+0000 7fd90da00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 0x7fd8f006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:41.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.880+0000 7fd90da00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 0x7fd8f006eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc036040 tx=0x7fd8fc005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:41.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:41.883+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd8f4055b00 con 0x7fd908102890 2024-04-24T08:20:42.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.072+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd90819ce40 con 0x7fd908102890 2024-04-24T08:20:42.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.073+0000 7fd906000640 1 -- 172.21.15.90:0/3844887119 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd8f4089e60 con 0x7fd908102890 2024-04-24T08:20:42.074 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:20:42.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.075+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 msgr2=0x7fd8f006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.075+0000 7fd90ee00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 0x7fd8f006eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc036040 tx=0x7fd8fc005fd0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.075+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 msgr2=0x7fd9081974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.075+0000 7fd90ee00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d6f0 tx=0x7fd8f400dbb0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 shutdown_connections 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd8f006c5f0 0x7fd8f006eaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd908102890 0x7fd9081974f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 --2- 172.21.15.90:0/3844887119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9081016a0 0x7fd908196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 >> 172.21.15.90:0/3844887119 conn(0x7fd9080fcc30 msgr2=0x7fd908105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 shutdown_connections 2024-04-24T08:20:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.076+0000 7fd90ee00640 1 -- 172.21.15.90:0/3844887119 wait complete. 2024-04-24T08:20:42.094 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.273+0000 7f0bfb400640 1 -- 172.21.15.90:0/3056521443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 msgr2=0x7f0bf4107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.273+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3056521443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf4107360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009880 tx=0x7f0bdc02bce0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 -- 172.21.15.90:0/3056521443 shutdown_connections 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3056521443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf4107360 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3056521443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 0x7f0bf4106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 -- 172.21.15.90:0/3056521443 >> 172.21.15.90:0/3056521443 conn(0x7f0bf4074ef0 msgr2=0x7f0bf4077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 -- 172.21.15.90:0/3056521443 shutdown_connections 2024-04-24T08:20:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.274+0000 7f0bfb400640 1 -- 172.21.15.90:0/3056521443 wait complete. 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 Processor -- start 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 -- start start 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 0x7f0bf419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf419c040 con 0x7f0bf4106ef0 2024-04-24T08:20:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bfb400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf419c1b0 con 0x7f0bf4105c60 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41030/0 (socket says 172.21.15.90:41030) 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.275+0000 7f0bf3e00640 1 -- 172.21.15.90:0/3150845821 learned_addr learned my addr 172.21.15.90:0/3150845821 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf8a00640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 0x7f0bf419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf3e00640 1 -- 172.21.15.90:0/3150845821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 msgr2=0x7f0bf419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf3e00640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 0x7f0bf419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf3e00640 1 -- 172.21.15.90:0/3150845821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009560 con 0x7f0bf4106ef0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf3e00640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc02c220 tx=0x7f0bdc004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc005000 con 0x7f0bf4106ef0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf41a0c20 con 0x7f0bf4106ef0 2024-04-24T08:20:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc005610 con 0x7f0bf4106ef0 2024-04-24T08:20:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.276+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc034670 con 0x7f0bf4106ef0 2024-04-24T08:20:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.277+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf41a10f0 con 0x7f0bf4106ef0 2024-04-24T08:20:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.278+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf410e7a0 con 0x7f0bf4106ef0 2024-04-24T08:20:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.279+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0bdc0039d0 con 0x7f0bf4106ef0 2024-04-24T08:20:42.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.280+0000 7f0bf1600640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 0x7f0bd806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.280+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0bdc0adf80 con 0x7f0bf4106ef0 2024-04-24T08:20:42.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.280+0000 7f0bf8a00640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 0x7f0bd806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.281+0000 7f0bf8a00640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 0x7f0bd806eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0be4007350 tx=0x7f0be40072e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.284+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0bdc0e0720 con 0x7f0bf4106ef0 2024-04-24T08:20:42.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.475+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f0bf410e980 con 0x7f0bf4106ef0 2024-04-24T08:20:42.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.476+0000 7f0bf1600640 1 -- 172.21.15.90:0/3150845821 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0bdc041020 con 0x7f0bf4106ef0 2024-04-24T08:20:42.477 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 msgr2=0x7f0bd806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 0x7f0bd806eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0be4007350 tx=0x7f0be40072e0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 msgr2=0x7f0bf419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc02c220 tx=0x7f0bdc004f00 comp rx=0 tx=0).stop 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 shutdown_connections 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f0bd806c5f0 0x7f0bd806eaa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bf4106ef0 0x7f0bf419bb00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 --2- 172.21.15.90:0/3150845821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4105c60 0x7f0bf419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 >> 172.21.15.90:0/3150845821 conn(0x7f0bf4074ef0 msgr2=0x7f0bf410a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:42.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 shutdown_connections 2024-04-24T08:20:42.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.479+0000 7f0bfb400640 1 -- 172.21.15.90:0/3150845821 wait complete. 2024-04-24T08:20:42.495 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:42 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1365081431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:20:42.495 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:42 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3844887119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:20:42.499 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:20:42.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:42 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1365081431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:20:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:42 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3844887119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:20:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 -- 172.21.15.90:0/2515559067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 msgr2=0x7fa70c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 --2- 172.21.15.90:0/2515559067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c104dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80098b0 tx=0x7fa6f802bcd0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 -- 172.21.15.90:0/2515559067 shutdown_connections 2024-04-24T08:20:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 --2- 172.21.15.90:0/2515559067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c104dd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 --2- 172.21.15.90:0/2515559067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 0x7fa70c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 -- 172.21.15.90:0/2515559067 >> 172.21.15.90:0/2515559067 conn(0x7fa70c0f9af0 msgr2=0x7fa70c0fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 -- 172.21.15.90:0/2515559067 shutdown_connections 2024-04-24T08:20:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.691+0000 7fa713a00640 1 -- 172.21.15.90:0/2515559067 wait complete. 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.692+0000 7fa713a00640 1 Processor -- start 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa713a00640 1 -- start start 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa713a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 0x7fa70c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa713a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa713a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c197c80 con 0x7fa70c1029f0 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa713a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c197df0 con 0x7fa70c1000d0 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa70be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa70be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41056/0 (socket says 172.21.15.90:41056) 2024-04-24T08:20:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa70be00640 1 -- 172.21.15.90:0/2264212723 learned_addr learned my addr 172.21.15.90:0/2264212723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:42.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa711000640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 0x7fa70c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.693+0000 7fa70be00640 1 -- 172.21.15.90:0/2264212723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 msgr2=0x7fa70c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa70be00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 0x7fa70c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa70be00640 1 -- 172.21.15.90:0/2264212723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8009560 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa70be00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6f802c210 tx=0x7fa6f8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6f8004420 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa6f80054b0 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6f803e450 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c19c820 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.694+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c19cd80 con 0x7fa70c1029f0 2024-04-24T08:20:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.696+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c0fb5d0 con 0x7fa70c1029f0 2024-04-24T08:20:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.696+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa6f80037d0 con 0x7fa70c1029f0 2024-04-24T08:20:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.697+0000 7fa709600640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 0x7fa6f406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.697+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa6f80ae520 con 0x7fa70c1029f0 2024-04-24T08:20:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.698+0000 7fa711000640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 0x7fa6f406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:42.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.698+0000 7fa711000640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 0x7fa6f406eb60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa700005fd0 tx=0x7fa700005850 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.702+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa6f8078b80 con 0x7fa70c1029f0 2024-04-24T08:20:42.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.929+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa70c1082f0 con 0x7fa70c1029f0 2024-04-24T08:20:42.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.930+0000 7fa709600640 1 -- 172.21.15.90:0/2264212723 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fa6f80ace60 con 0x7fa70c1029f0 2024-04-24T08:20:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:20:42.931 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14494,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/13627278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":13627278},{"type":"v1","addr":"172.21.15.90:6827","nonce":13627278}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:20:31.598807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi090.lumztb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":3,"addr":"172.21.15.90:6829/752337863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":752337863},{"type":"v1","addr":"172.21.15.90:6829","nonce":752337863}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-24T08:20:42.931 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 msgr2=0x7fa6f406eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 0x7fa6f406eb60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa700005fd0 tx=0x7fa700005850 comp rx=0 tx=0).stop 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 msgr2=0x7fa70c1976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6f802c210 tx=0x7fa6f8004f00 comp rx=0 tx=0).stop 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 shutdown_connections 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fa6f406c6b0 0x7fa6f406eb60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa70c1029f0 0x7fa70c1976b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 --2- 172.21.15.90:0/2264212723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa70c1000d0 0x7fa70c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:42.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 >> 172.21.15.90:0/2264212723 conn(0x7fa70c0f9af0 msgr2=0x7fa70c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:42.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 shutdown_connections 2024-04-24T08:20:42.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:42.932+0000 7fa713a00640 1 -- 172.21.15.90:0/2264212723 wait complete. 2024-04-24T08:20:42.952 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-04-24T08:20:42.952 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T08:20:42.979 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T08:20:42.979 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-24T08:20:42.979 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T08:20:42.979 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T08:20:42.979 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:42.979 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-24T08:20:42.979 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T08:20:42.979 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:42.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:42.980 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-24T08:20:43.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:43.022 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-24T08:20:43.137 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-04-24T08:20:43.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:20:43.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:43.141 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-04-24T08:20:43.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:43.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-04-24T08:20:43.294 INFO:teuthology.orchestra.run.smithi149.stderr:Cannot find device "ceph-brx" 2024-04-24T08:20:43.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:20:43.298 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T08:20:43.298 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T08:20:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:43 smithi149 ceph-mon[31282]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:43 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3150845821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:20:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:43 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2264212723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:20:43.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:43 smithi090 ceph-mon[25016]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:43.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:43 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3150845821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:20:43.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:43 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2264212723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:20:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:45.497 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:45 smithi090 ceph-mon[25016]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-24T08:20:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:45 smithi149 ceph-mon[31282]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-24T08:20:46.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 -- 172.21.15.90:0/3072299234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 msgr2=0x7f865c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 --2- 172.21.15.90:0/3072299234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c108840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8650009910 tx=0x7f8650009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 -- 172.21.15.90:0/3072299234 shutdown_connections 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 --2- 172.21.15.90:0/3072299234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 0x7f865c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 --2- 172.21.15.90:0/3072299234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c108840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 -- 172.21.15.90:0/3072299234 >> 172.21.15.90:0/3072299234 conn(0x7f865c0fdf90 msgr2=0x7f865c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 -- 172.21.15.90:0/3072299234 shutdown_connections 2024-04-24T08:20:46.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.699+0000 7f8663c00700 1 -- 172.21.15.90:0/3072299234 wait complete. 2024-04-24T08:20:46.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.700+0000 7f8663c00700 1 Processor -- start 2024-04-24T08:20:46.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.700+0000 7f8663c00700 1 -- start start 2024-04-24T08:20:46.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f8663c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 0x7f865c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:46.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f8663c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f8663c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c198e50 con 0x7f865c108470 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f8663c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19cbe0 con 0x7f865c102470 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f865be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f865be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41074/0 (socket says 172.21.15.90:41074) 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.701+0000 7f865be00700 1 -- 172.21.15.90:0/2736893268 learned_addr learned my addr 172.21.15.90:0/2736893268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:46.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f8661200700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 0x7f865c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f865be00700 1 -- 172.21.15.90:0/2736893268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 msgr2=0x7f865c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f865be00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 0x7f865c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f865be00700 1 -- 172.21.15.90:0/2736893268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500095f0 con 0x7f865c108470 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f865be00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f864800d8d0 tx=0x7f864800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.702+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8648009940 con 0x7f865c108470 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.703+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8648010460 con 0x7f865c108470 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.703+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c19cec0 con 0x7f865c108470 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.703+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864800f5f0 con 0x7f865c108470 2024-04-24T08:20:46.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.703+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c19d410 con 0x7f865c108470 2024-04-24T08:20:46.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.704+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c10a9e0 con 0x7f865c108470 2024-04-24T08:20:46.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.705+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f86480105d0 con 0x7f865c108470 2024-04-24T08:20:46.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.706+0000 7f8659600700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 0x7f864406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:46.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.706+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f864808b980 con 0x7f865c108470 2024-04-24T08:20:46.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.706+0000 7f8661200700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 0x7f864406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:46.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.707+0000 7f8661200700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 0x7f864406eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f865000b5c0 tx=0x7f865001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:46.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.711+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8648055f60 con 0x7f865c108470 2024-04-24T08:20:46.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.974+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f865c19d630 con 0x7f865c108470 2024-04-24T08:20:46.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.974+0000 7f8659600700 1 -- 172.21.15.90:0/2736893268 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8648059580 con 0x7f865c108470 2024-04-24T08:20:46.976 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.977+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 msgr2=0x7f864406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.977+0000 7f8663c00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 0x7f864406eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f865000b5c0 tx=0x7f865001a040 comp rx=0 tx=0).stop 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 msgr2=0x7f865c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f864800d8d0 tx=0x7f864800dc90 comp rx=0 tx=0).stop 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 shutdown_connections 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f864406c680 0x7f864406eb30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f865c102470 0x7f865c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 --2- 172.21.15.90:0/2736893268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f865c108470 0x7f865c198770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 >> 172.21.15.90:0/2736893268 conn(0x7f865c0fdf90 msgr2=0x7f865c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 shutdown_connections 2024-04-24T08:20:46.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:46.978+0000 7f8663c00700 1 -- 172.21.15.90:0/2736893268 wait complete. 2024-04-24T08:20:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:47 smithi149 ceph-mon[31282]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:47 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:47 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:47 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2736893268' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:20:47.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:47 smithi090 ceph-mon[25016]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T08:20:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:47 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:47 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:47 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2736893268' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T08:20:47.762 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T08:20:47.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:47.763 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-04-24T08:20:47.818 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:47.819 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42047sec preferred_lft 42047sec 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-04-24T08:20:47.820 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:47.821 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T08:20:47.821 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T08:20:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:48.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:48.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:48.364 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T08:20:48.443 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-04-24T08:20:48.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:48.449 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-04-24T08:20:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-04-24T08:20:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T08:20:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T08:20:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T08:20:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T08:20:48.507 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:48.507 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T08:20:48.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T08:20:48.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T08:20:48.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T08:20:48.508 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T08:20:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:20:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:20:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:20:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:49.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:49.313 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-24T08:20:49.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:49.412 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-04-24T08:20:49.465 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:49.465 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T08:20:49.465 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:20:49.465 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T08:20:49.465 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T08:20:49.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:49 smithi090 ceph-mon[25016]: pgmap v244: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4 op/s 2024-04-24T08:20:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:49.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:49.656 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T08:20:49.656 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:49.656 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T08:20:49.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T08:20:49.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T08:20:49.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T08:20:49.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T08:20:49.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T08:20:49.657 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T08:20:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:49 smithi149 ceph-mon[31282]: pgmap v244: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4 op/s 2024-04-24T08:20:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:49.963 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:49.963 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T08:20:49.963 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-04-24T08:20:49.963 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T08:20:49.963 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T08:20:50.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:50.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:50.161 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T08:20:50.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:20:50.161 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:50.203 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T08:20:50.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:20:50.203 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:50.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-04-24T08:20:50.363 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:50.366 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-04-24T08:20:50.412 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-04-24T08:20:50.413 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e59b1e0ea484a2b84454f06526f1f5a21d655ecf1e9f6d8baf054205c299ad72/merged 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/733878ecaeab10a763a3efe249cc80e1d5110de2f8d586f0c46d8152754c3b4f/merged 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:50.414 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/b53bd0d4c7900fe4c2d0e927bc46f85399a141b0dca2799542daec38b035aba0/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a30dbbd6c88ce5d5bf2fe6be650db08b05a290ce5595d9b00d8550c2974b2761/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e2ed15c92ed414813987cea79d1389398a63eefe9f6cf85026e59247aef327bd/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/58d8723fb859cf875a7cbc6a1644db4e7da07d8657307d96360fed8753c3afc3/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dc476758b5ce1c7cf663d727e98b990905e43fdebad9d8a7406139db0b76e83d/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/0b50760ff3a03776ed15edf3f61852b76cb3fb295af78a31337d2a465b86b5a3/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/db9160ab621fb03536b3453520e0c455474fa1277a97bd30895e5c20ac0aa4c3/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/3a8e61652df15156d0340350a4abc32497e6136d81b1aa78c528ef56e4a6e406/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/46cfd6b116b771aa2313ea00e5a6bb1d4389395beb8280f029bae0d21f8abd81/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/514f8255b0a68cc92866520ef52f4fd0d7c0ee89c661631764600e04a7de9e34/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/711e571ab15dee96b29498eaaf7ad0b7e788948af52ac5102b428ef3c0f02b00/merged 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-04-24T08:20:50.415 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:20:50.416 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:20:50.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:50.417 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-04-24T08:20:50.512 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T08:20:50.512 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T08:20:50.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-04-24T08:20:50.587 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:20:50.630 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-04-24T08:20:50.631 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:20:50.632 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e59b1e0ea484a2b84454f06526f1f5a21d655ecf1e9f6d8baf054205c299ad72/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/733878ecaeab10a763a3efe249cc80e1d5110de2f8d586f0c46d8152754c3b4f/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/b53bd0d4c7900fe4c2d0e927bc46f85399a141b0dca2799542daec38b035aba0/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a30dbbd6c88ce5d5bf2fe6be650db08b05a290ce5595d9b00d8550c2974b2761/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e2ed15c92ed414813987cea79d1389398a63eefe9f6cf85026e59247aef327bd/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/58d8723fb859cf875a7cbc6a1644db4e7da07d8657307d96360fed8753c3afc3/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dc476758b5ce1c7cf663d727e98b990905e43fdebad9d8a7406139db0b76e83d/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/0b50760ff3a03776ed15edf3f61852b76cb3fb295af78a31337d2a465b86b5a3/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/db9160ab621fb03536b3453520e0c455474fa1277a97bd30895e5c20ac0aa4c3/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/3a8e61652df15156d0340350a4abc32497e6136d81b1aa78c528ef56e4a6e406/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/46cfd6b116b771aa2313ea00e5a6bb1d4389395beb8280f029bae0d21f8abd81/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/514f8255b0a68cc92866520ef52f4fd0d7c0ee89c661631764600e04a7de9e34/merged 2024-04-24T08:20:50.633 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/711e571ab15dee96b29498eaaf7ad0b7e788948af52ac5102b428ef3c0f02b00/merged 2024-04-24T08:20:50.634 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-04-24T08:20:50.634 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:20:50.634 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:20:50.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:50.635 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-04-24T08:20:50.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:2024-04-24T08:20:50.701+0000 7fa91bdb6480 -1 init, newargv = 0x56035d9268a0 newargc=15 2024-04-24T08:20:50.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[63005]: starting ceph client 2024-04-24T08:20:50.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[63005]: starting fuse 2024-04-24T08:20:50.720 INFO:teuthology.orchestra.run.smithi090.stdout:144 2024-04-24T08:20:50.721 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [144] 2024-04-24T08:20:50.721 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-04-24T08:20:50.721 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> 2024-04-24T08:20:50.722 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.0")) 2024-04-24T08:20:50.723 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-04-24T08:20:50.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:20:50.914 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.0.63005.asok 2024-04-24T08:20:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:50.931 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63005.asok 2024-04-24T08:20:50.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:50.932 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63005.asok status 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "0", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "63005", 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:20:51.163 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 14534, 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 14534 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-04-24T08:20:51.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2302411015 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2302411015 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.14534 192.168.0.1:0/2302411015", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/2302411015", 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 17, 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 47, 2024-04-24T08:20:51.165 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-04-24T08:20:51.166 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-04-24T08:20:51.166 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-04-24T08:20:51.166 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:20:51.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T08:20:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:51 smithi090 ceph-mon[25016]: pgmap v245: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:20:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:20:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:51 smithi149 ceph-mon[31282]: pgmap v245: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:20:52.779 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:52 smithi090 ceph-mon[25016]: pgmap v246: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:20:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:52 smithi149 ceph-mon[31282]: pgmap v246: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:20:53.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.738+0000 7f3611a00700 1 -- 172.21.15.90:0/3323303634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c073a00 msgr2=0x7f360c10c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:53.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.738+0000 7f3611a00700 1 --2- 172.21.15.90:0/3323303634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c073a00 0x7f360c10c5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009910 tx=0x7f35f4009c20 comp rx=0 tx=0).stop 2024-04-24T08:20:53.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 -- 172.21.15.90:0/3323303634 shutdown_connections 2024-04-24T08:20:53.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 --2- 172.21.15.90:0/3323303634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c073a00 0x7f360c10c5d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:53.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 --2- 172.21.15.90:0/3323303634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c0730f0 0x7f360c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:53.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 -- 172.21.15.90:0/3323303634 >> 172.21.15.90:0/3323303634 conn(0x7f360c0fbe10 msgr2=0x7f360c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:53.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 -- 172.21.15.90:0/3323303634 shutdown_connections 2024-04-24T08:20:53.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.739+0000 7f3611a00700 1 -- 172.21.15.90:0/3323303634 wait complete. 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 Processor -- start 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 -- start start 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 0x7f360c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c19c970 con 0x7f360c0730f0 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.740+0000 7f3611a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c19cae0 con 0x7f360c073a00 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48126/0 (socket says 172.21.15.90:48126) 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 -- 172.21.15.90:0/2807970417 learned_addr learned my addr 172.21.15.90:0/2807970417 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:20:53.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f3602000700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 0x7f360c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 -- 172.21.15.90:0/2807970417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 msgr2=0x7f360c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 0x7f360c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f360aa00700 1 -- 172.21.15.90:0/2807970417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f40095f0 con 0x7f360c0730f0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.741+0000 7f3602000700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 0x7f360c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f360aa00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00ba50 tx=0x7f35fc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35fc00ca00 con 0x7f360c0730f0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35fc00cb60 con 0x7f360c0730f0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c19cdc0 con 0x7f360c0730f0 2024-04-24T08:20:53.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35fc007580 con 0x7f360c0730f0 2024-04-24T08:20:53.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.742+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c19d310 con 0x7f360c0730f0 2024-04-24T08:20:53.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.743+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c109cd0 con 0x7f360c0730f0 2024-04-24T08:20:53.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.744+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f35fc00ccd0 con 0x7f360c0730f0 2024-04-24T08:20:53.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.745+0000 7f3603e00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 0x7f35f006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:20:53.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.745+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f35fc006e40 con 0x7f360c0730f0 2024-04-24T08:20:53.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.746+0000 7f3602000700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 0x7f35f006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:20:53.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.746+0000 7f3602000700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 0x7f35f006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006010 tx=0x7f35f401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:20:53.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:53.751+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f35fc04e720 con 0x7f360c0730f0 2024-04-24T08:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.015+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f360c1993c0 con 0x7f360c0730f0 2024-04-24T08:20:54.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.016+0000 7f3603e00700 1 -- 172.21.15.90:0/2807970417 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f35fc059270 con 0x7f360c0730f0 2024-04-24T08:20:54.017 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 msgr2=0x7f35f006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 0x7f35f006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006010 tx=0x7f35f401a040 comp rx=0 tx=0).stop 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 msgr2=0x7f360c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00ba50 tx=0x7f35fc00be10 comp rx=0 tx=0).stop 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 shutdown_connections 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f35f006c680 0x7f35f006eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:54.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f360c0730f0 0x7f360c198010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 --2- 172.21.15.90:0/2807970417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f360c073a00 0x7f360c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 >> 172.21.15.90:0/2807970417 conn(0x7f360c0fbe10 msgr2=0x7f360c106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 shutdown_connections 2024-04-24T08:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:20:54.020+0000 7f3611a00700 1 -- 172.21.15.90:0/2807970417 wait complete. 2024-04-24T08:20:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:54 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2807970417' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:20:54.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:54 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2807970417' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:54.922 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T08:20:54.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T08:20:54.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T08:20:54.923 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T08:20:54.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:54.923 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-04-24T08:20:54.982 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T08:20:54.982 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T08:20:54.982 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T08:20:54.982 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42012sec preferred_lft 42012sec 2024-04-24T08:20:54.983 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link 2024-04-24T08:20:54.984 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:20:54.984 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add name ceph-brx type bridge 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr flush dev ceph-brx 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set ceph-brx up 2024-04-24T08:20:54.984 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T08:20:54.985 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-04-24T08:20:55.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:55 smithi090 ceph-mon[25016]: pgmap v247: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:20:55.365 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:55 smithi149 ceph-mon[31282]: pgmap v247: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:20:55.461 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:55.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:55.471 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T08:20:55.536 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-04-24T08:20:55.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:55.540 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-04-24T08:20:55.600 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-04-24T08:20:55.600 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T08:20:55.600 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T08:20:55.601 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T08:20:55.601 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T08:20:55.601 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-04-24T08:20:55.702 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:56.288 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:56.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:56.298 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-04-24T08:20:56.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:56.342 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-04-24T08:20:56.428 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:56.428 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-04-24T08:20:56.428 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:56.428 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T08:20:56.428 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-04-24T08:20:56.568 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:56.629 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:56.638 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T08:20:56.638 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:56.638 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-04-24T08:20:56.638 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T08:20:56.638 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T08:20:56.639 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T08:20:56.639 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T08:20:56.639 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T08:20:56.639 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-04-24T08:20:56.729 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:56.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:56.933 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:20:56.933 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-04-24T08:20:56.933 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.0 up 2024-04-24T08:20:56.933 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T08:20:56.933 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-04-24T08:20:57.037 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:20:57.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:57 smithi149 ceph-mon[31282]: pgmap v248: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-24T08:20:57.129 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:57.138 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-24T08:20:57.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:20:57.138 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:57.181 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T08:20:57.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:20:57.182 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:57.270 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-04-24T08:20:57.392 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-04-24T08:20:57.443 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-04-24T08:20:57.444 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/0 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/d9dc1cf2f999f18a064f10b2dea1e85ab5b2bc975ea10dac7078f1429d060d45/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/0265723473f61f7ee5f2be5368618f3303242c44b103ad7f3bfccd97dba724da/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ae9ded312b69d1374d4265028c563b86540a55f4d1ab929113642047b84a50e7/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/6792d353747e193148b90c890f393d1d3094c8c3be1a31397923509c80d6240d/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/2bedc338ae5033bcac60cb07bbb3a796b4bdcf963a11b1dd0e026ddf0dec6263/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/a084ecdbc54a0cee457ab542749a39a36f3e5d0b512ccc4bf797664e2aae8700/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/717fe6f70e1649f9a75fea3975350476163866bbfcef15fc4a980e1fb6fc8583/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/8e4361f4009075f5c2e84417ccbdc70099d7694ee1502964a7e2090d2d67344e/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ec6ff5d33be1402d6f3316806fd717f6479c8987546ce5ad4944d2ab5558b4d2/merged 2024-04-24T08:20:57.445 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/995072cf0a93614178a716fc66f23020ed6ab19d27af0c8cb0136a5cb306b1e7/merged 2024-04-24T08:20:57.446 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-04-24T08:20:57.446 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:57.446 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:57.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:57.447 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-04-24T08:20:57.496 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T08:20:57.496 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-24T08:20:57.500 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-04-24T08:20:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:57 smithi090 ceph-mon[25016]: pgmap v248: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-24T08:20:57.565 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:20:57.609 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-04-24T08:20:57.609 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-04-24T08:20:57.609 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-04-24T08:20:57.609 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-04-24T08:20:57.609 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-04-24T08:20:57.610 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-04-24T08:20:57.611 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/0 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/d9dc1cf2f999f18a064f10b2dea1e85ab5b2bc975ea10dac7078f1429d060d45/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/0265723473f61f7ee5f2be5368618f3303242c44b103ad7f3bfccd97dba724da/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ae9ded312b69d1374d4265028c563b86540a55f4d1ab929113642047b84a50e7/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/6792d353747e193148b90c890f393d1d3094c8c3be1a31397923509c80d6240d/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/2bedc338ae5033bcac60cb07bbb3a796b4bdcf963a11b1dd0e026ddf0dec6263/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/a084ecdbc54a0cee457ab542749a39a36f3e5d0b512ccc4bf797664e2aae8700/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/717fe6f70e1649f9a75fea3975350476163866bbfcef15fc4a980e1fb6fc8583/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/8e4361f4009075f5c2e84417ccbdc70099d7694ee1502964a7e2090d2d67344e/merged 2024-04-24T08:20:57.612 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ec6ff5d33be1402d6f3316806fd717f6479c8987546ce5ad4944d2ab5558b4d2/merged 2024-04-24T08:20:57.613 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/995072cf0a93614178a716fc66f23020ed6ab19d27af0c8cb0136a5cb306b1e7/merged 2024-04-24T08:20:57.613 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-04-24T08:20:57.613 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:57.613 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:57.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:57.614 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-04-24T08:20:57.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:2024-04-24T08:20:57.709+0000 7f289683d480 -1 init, newargv = 0x561d59cea830 newargc=15 2024-04-24T08:20:57.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[51295]: starting ceph client 2024-04-24T08:20:57.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[51295]: starting fuse 2024-04-24T08:20:58.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-04-24T08:20:58.897 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-04-24T08:20:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-04-24T08:20:58.952 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/0 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/d9dc1cf2f999f18a064f10b2dea1e85ab5b2bc975ea10dac7078f1429d060d45/merged 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/0265723473f61f7ee5f2be5368618f3303242c44b103ad7f3bfccd97dba724da/merged 2024-04-24T08:20:58.953 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ae9ded312b69d1374d4265028c563b86540a55f4d1ab929113642047b84a50e7/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/6792d353747e193148b90c890f393d1d3094c8c3be1a31397923509c80d6240d/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/2bedc338ae5033bcac60cb07bbb3a796b4bdcf963a11b1dd0e026ddf0dec6263/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/a084ecdbc54a0cee457ab542749a39a36f3e5d0b512ccc4bf797664e2aae8700/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/717fe6f70e1649f9a75fea3975350476163866bbfcef15fc4a980e1fb6fc8583/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/8e4361f4009075f5c2e84417ccbdc70099d7694ee1502964a7e2090d2d67344e/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/ec6ff5d33be1402d6f3316806fd717f6479c8987546ce5ad4944d2ab5558b4d2/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/containers/storage/overlay/995072cf0a93614178a716fc66f23020ed6ab19d27af0c8cb0136a5cb306b1e7/merged 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T08:20:58.954 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:58.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:58.955 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-04-24T08:20:59.001 INFO:teuthology.orchestra.run.smithi149.stdout:122 2024-04-24T08:20:59.001 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-04-24T08:20:59.001 DEBUG:teuthology.orchestra.run.smithi149:> sudo stdin-killer -- python3 -c ' 2024-04-24T08:20:59.001 DEBUG:teuthology.orchestra.run.smithi149:> import glob 2024-04-24T08:20:59.001 DEBUG:teuthology.orchestra.run.smithi149:> import re 2024-04-24T08:20:59.001 DEBUG:teuthology.orchestra.run.smithi149:> import os 2024-04-24T08:20:59.001 DEBUG:teuthology.orchestra.run.smithi149:> import subprocess 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> def _find_admin_socket(client_name): 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> files = glob.glob(asok_path) 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> # Given a non-glob path, it better be there 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> if "*" not in asok_path: 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> assert(len(files) == 1) 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> return files[0] 2024-04-24T08:20:59.002 DEBUG:teuthology.orchestra.run.smithi149:> 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> for f in files: 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> contents = proc_f.read() 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> if mountpoint in contents: 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> return f 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> print(_find_admin_socket("client.1")) 2024-04-24T08:20:59.003 DEBUG:teuthology.orchestra.run.smithi149:> ' 2024-04-24T08:20:59.214 INFO:teuthology.orchestra.run.smithi149.stdout:/var/run/ceph/ceph-client.1.51295.asok 2024-04-24T08:20:59.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T08:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:20:59.228 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:20:59 smithi149 ceph-mon[31282]: pgmap v249: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-24T08:20:59.232 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.51295.asok 2024-04-24T08:20:59.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:59.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.51295.asok status 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "metadata": { 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "entity_id": "1", 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "hostname": "smithi149", 2024-04-24T08:20:59.458 INFO:teuthology.orchestra.run.smithi149.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "pid": "51295", 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "root": "/" 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_count": 0, 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_pinned_count": 0, 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "id": 24275, 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "inst": { 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "name": { 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "client", 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "num": 24275 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-04-24T08:20:59.459 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 4244548942 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 4244548942 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "inst_str": "client.24275 192.168.0.1:0/4244548942", 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "addr_str": "192.168.0.1:0/4244548942", 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "inode_count": 1, 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_epoch": 17, 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch": 47, 2024-04-24T08:20:59.460 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch_barrier": 0, 2024-04-24T08:20:59.461 INFO:teuthology.orchestra.run.smithi149.stdout: "blocklisted": false, 2024-04-24T08:20:59.461 INFO:teuthology.orchestra.run.smithi149.stdout: "fs_name": "cephfs" 2024-04-24T08:20:59.461 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T08:20:59.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:59.473 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-04-24T08:20:59.473 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.533 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-04-24T08:20:59.534 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:59.534 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:20:59 smithi090 ceph-mon[25016]: pgmap v249: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-24T08:20:59.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:59.630 DEBUG:teuthology.orchestra.run.smithi149:> stat --file-system '--printf=%T 2024-04-24T08:20:59.631 DEBUG:teuthology.orchestra.run.smithi149:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:59.688 INFO:teuthology.orchestra.run.smithi149.stdout:fuseblk 2024-04-24T08:20:59.689 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:59.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:20:59.689 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T08:20:59.778 INFO:teuthology.run_tasks:Running task print... 2024-04-24T08:20:59.789 INFO:teuthology.task.print:**** done client 2024-04-24T08:20:59.789 INFO:teuthology.run_tasks:Running task parallel... 2024-04-24T08:20:59.799 INFO:teuthology.task.parallel:starting parallel... 2024-04-24T08:20:59.800 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T08:20:59.800 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T08:20:59.801 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:20:59.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T08:20:59.801 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T08:20:59.801 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-24T08:20:59.805 INFO:tasks.workunit:Pulling workunits from ref 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:20:59.805 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T08:20:59.805 INFO:tasks.workunit:timeout=3h 2024-04-24T08:20:59.805 INFO:tasks.workunit:cleanup=True 2024-04-24T08:20:59.806 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 90h/144d Inode: 1 Links: 2 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T08:20:59.863 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T08:20:59.864 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 08:20:12.687773751 +0000 2024-04-24T08:20:59.864 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 08:20:59.622457140 +0000 2024-04-24T08:20:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-04-24T08:20:59.864 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T08:20:59.864 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T08:20:59.990 DEBUG:teuthology.orchestra.run.smithi149:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 7ah/122d Inode: 1 Links: 3 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T08:21:00.048 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T08:21:00.049 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 08:20:59.975997821 +0000 2024-04-24T08:21:00.049 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 08:20:59.975997821 +0000 2024-04-24T08:21:00.049 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2024-04-24T08:21:00.049 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-24T08:21:00.049 DEBUG:teuthology.orchestra.run.smithi149:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-24T08:21:00.135 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:21:00.136 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:21:00.179 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T08:21:00.192 INFO:tasks.workunit.client.1.smithi149.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-24T08:21:00.195 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:21:01.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:01 smithi090 ceph-mon[25016]: pgmap v250: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-24T08:21:01.429 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:01 smithi149 ceph-mon[31282]: pgmap v250: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-24T08:21:02.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 -- 172.21.15.90:0/3266949300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 msgr2=0x7fab041059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 --2- 172.21.15.90:0/3266949300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041059d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009910 tx=0x7faaf0009c20 comp rx=0 tx=0).stop 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 -- 172.21.15.90:0/3266949300 shutdown_connections 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 --2- 172.21.15.90:0/3266949300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041059d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 --2- 172.21.15.90:0/3266949300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 0x7fab041030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 -- 172.21.15.90:0/3266949300 >> 172.21.15.90:0/3266949300 conn(0x7fab040fa6d0 msgr2=0x7fab040fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 -- 172.21.15.90:0/3266949300 shutdown_connections 2024-04-24T08:21:02.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.575+0000 7fab0a600700 1 -- 172.21.15.90:0/3266949300 wait complete. 2024-04-24T08:21:02.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.576+0000 7fab0a600700 1 Processor -- start 2024-04-24T08:21:02.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab0a600700 1 -- start start 2024-04-24T08:21:02.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab0a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 0x7fab04197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:02.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab0a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab0a600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04198750 con 0x7fab041035f0 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab0a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab041988c0 con 0x7fab04100cd0 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab02a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab02a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56770/0 (socket says 172.21.15.90:56770) 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab02a00700 1 -- 172.21.15.90:0/856667688 learned_addr learned my addr 172.21.15.90:0/856667688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.577+0000 7fab03400700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 0x7fab04197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7fab02a00700 1 -- 172.21.15.90:0/856667688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 msgr2=0x7fab04197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7fab02a00700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 0x7fab04197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7fab02a00700 1 -- 172.21.15.90:0/856667688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00095f0 con 0x7fab041035f0 2024-04-24T08:21:02.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7fab02a00700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005b40 tx=0x7faaf000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:02.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf001d070 con 0x7fab041035f0 2024-04-24T08:21:02.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faaf0022470 con 0x7fab041035f0 2024-04-24T08:21:02.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf000f690 con 0x7fab041035f0 2024-04-24T08:21:02.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.578+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0419d300 con 0x7fab041035f0 2024-04-24T08:21:02.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.579+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0419d7f0 con 0x7fab041035f0 2024-04-24T08:21:02.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.580+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab040fc2b0 con 0x7fab041035f0 2024-04-24T08:21:02.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.581+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7faaf00225e0 con 0x7fab041035f0 2024-04-24T08:21:02.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.581+0000 7faafbe00700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 0x7faae806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:02.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.582+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faaf008cff0 con 0x7fab041035f0 2024-04-24T08:21:02.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.582+0000 7fab03400700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 0x7faae806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:02.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.583+0000 7fab03400700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 0x7faae806eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005ea0 tx=0x7faaf4005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:02.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.587+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faaf0057710 con 0x7fab041035f0 2024-04-24T08:21:02.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.804+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fab0402cc30 con 0x7fab041035f0 2024-04-24T08:21:02.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.805+0000 7faafbe00700 1 -- 172.21.15.90:0/856667688 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7faaf005ad30 con 0x7fab041035f0 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.808+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 msgr2=0x7faae806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.808+0000 7fab0a600700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 0x7faae806eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005ea0 tx=0x7faaf4005e30 comp rx=0 tx=0).stop 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.808+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 msgr2=0x7fab041981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.808+0000 7fab0a600700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005b40 tx=0x7faaf000be20 comp rx=0 tx=0).stop 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 shutdown_connections 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faae806c680 0x7faae806eb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab04100cd0 0x7fab04197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 --2- 172.21.15.90:0/856667688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab041035f0 0x7fab041981e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 >> 172.21.15.90:0/856667688 conn(0x7fab040fa6d0 msgr2=0x7fab04100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:02.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 shutdown_connections 2024-04-24T08:21:02.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:02.809+0000 7fab0a600700 1 -- 172.21.15.90:0/856667688 wait complete. 2024-04-24T08:21:03.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:02 smithi149 ceph-mon[31282]: pgmap v251: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-24T08:21:03.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:02 smithi090 ceph-mon[25016]: pgmap v251: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-24T08:21:03.572 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T08:21:04.043 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:21:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:05 smithi149 ceph-mon[31282]: pgmap v252: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s 2024-04-24T08:21:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:05 smithi090 ceph-mon[25016]: pgmap v252: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s 2024-04-24T08:21:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.393+0000 7f8739c00700 1 -- 172.21.15.90:0/3159714630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 msgr2=0x7f87340fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.393+0000 7f8739c00700 1 --2- 172.21.15.90:0/3159714630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f87340fef60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8724009910 tx=0x7f8724009c20 comp rx=0 tx=0).stop 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 -- 172.21.15.90:0/3159714630 shutdown_connections 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 --2- 172.21.15.90:0/3159714630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 0x7f87340ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 --2- 172.21.15.90:0/3159714630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f87340fef60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 -- 172.21.15.90:0/3159714630 >> 172.21.15.90:0/3159714630 conn(0x7f87340fa780 msgr2=0x7f87340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 -- 172.21.15.90:0/3159714630 shutdown_connections 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.394+0000 7f8739c00700 1 -- 172.21.15.90:0/3159714630 wait complete. 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 Processor -- start 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 -- start start 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 0x7f8734198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734198840 con 0x7f87340feb50 2024-04-24T08:21:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.395+0000 7f8739c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873419d250 con 0x7f87340ff4a0 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56800/0 (socket says 172.21.15.90:56800) 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 -- 172.21.15.90:0/3675268733 learned_addr learned my addr 172.21.15.90:0/3675268733 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732000700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 0x7f8734198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 -- 172.21.15.90:0/3675268733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 msgr2=0x7f8734198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 0x7f8734198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 -- 172.21.15.90:0/3675268733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240095f0 con 0x7f87340feb50 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.396+0000 7f8732a00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f872400bd70 tx=0x7f872400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.397+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872401d070 con 0x7f87340feb50 2024-04-24T08:21:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.397+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f872400f460 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.397+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8724021620 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.397+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873419d3f0 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.397+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873419d860 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.398+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873410aef0 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.399+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f872400fa40 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.400+0000 7f872b400700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 0x7f871806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.400+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f872408cc40 con 0x7f87340feb50 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.400+0000 7f8732000700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 0x7f871806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.401+0000 7f8732000700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 0x7f871806eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8720005950 tx=0x7f87200058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.405+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8724057220 con 0x7f87340feb50 2024-04-24T08:21:06.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.623+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f873410b0e0 con 0x7f87340feb50 2024-04-24T08:21:06.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.624+0000 7f872b400700 1 -- 172.21.15.90:0/3675268733 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f872405a840 con 0x7f87340feb50 2024-04-24T08:21:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 msgr2=0x7f871806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 0x7f871806eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8720005950 tx=0x7f87200058e0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 msgr2=0x7f8734197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f872400bd70 tx=0x7f872400be50 comp rx=0 tx=0).stop 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 shutdown_connections 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f871806c680 0x7f871806eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87340feb50 0x7f8734197ce0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 --2- 172.21.15.90:0/3675268733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87340ff4a0 0x7f8734198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.627+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 >> 172.21.15.90:0/3675268733 conn(0x7f87340fa780 msgr2=0x7f8734107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.628+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 shutdown_connections 2024-04-24T08:21:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:06.628+0000 7f8739c00700 1 -- 172.21.15.90:0/3675268733 wait complete. 2024-04-24T08:21:07.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:07 smithi090 ceph-mon[25016]: pgmap v253: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-24T08:21:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:07 smithi149 ceph-mon[31282]: pgmap v253: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-24T08:21:07.647 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T08:21:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:21:09.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:09 smithi090 ceph-mon[25016]: pgmap v254: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-24T08:21:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:09 smithi149 ceph-mon[31282]: pgmap v254: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 -- 172.21.15.90:0/2144927371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c106840 msgr2=0x7f272c106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 --2- 172.21.15.90:0/2144927371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c106840 0x7f272c106c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f271c00b0a0 tx=0x7f271c00b3b0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 -- 172.21.15.90:0/2144927371 shutdown_connections 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 --2- 172.21.15.90:0/2144927371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c107a40 0x7f272c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 --2- 172.21.15.90:0/2144927371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c106840 0x7f272c106c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 -- 172.21.15.90:0/2144927371 >> 172.21.15.90:0/2144927371 conn(0x7f272c075970 msgr2=0x7f272c077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 -- 172.21.15.90:0/2144927371 shutdown_connections 2024-04-24T08:21:11.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.095+0000 7f2731200700 1 -- 172.21.15.90:0/2144927371 wait complete. 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.096+0000 7f2731200700 1 Processor -- start 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2731200700 1 -- start start 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2731200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c106840 0x7f272c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2731200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2731200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c19ccc0 con 0x7f272c107a40 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2731200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c19ce00 con 0x7f272c106840 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2729400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2729400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42752/0 (socket says 172.21.15.90:42752) 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.097+0000 7f2729400700 1 -- 172.21.15.90:0/3732734446 learned_addr learned my addr 172.21.15.90:0/3732734446 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2729e00700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c106840 0x7f272c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2729400700 1 -- 172.21.15.90:0/3732734446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c106840 msgr2=0x7f272c19c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2729400700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c106840 0x7f272c19c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2729400700 1 -- 172.21.15.90:0/3732734446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c009d00 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2729400700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f271800d8d0 tx=0x7f271800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.098+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2718009940 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.099+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2718010460 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.099+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271800f5f0 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.099+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c1a18b0 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.099+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c1a1e00 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.100+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c10f390 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.101+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f27180105d0 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.101+0000 7f2722a00700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 0x7f271006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.102+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f271808b8b0 con 0x7f272c107a40 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.102+0000 7f2729e00700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 0x7f271006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.102+0000 7f2729e00700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 0x7f271006eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00b7d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:11.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.106+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f27180599a0 con 0x7f272c107a40 2024-04-24T08:21:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.323+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f272c10f580 con 0x7f272c107a40 2024-04-24T08:21:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.324+0000 7f2722a00700 1 -- 172.21.15.90:0/3732734446 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f271805de60 con 0x7f272c107a40 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 msgr2=0x7f271006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 0x7f271006eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00b7d0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 msgr2=0x7f272c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f271800d8d0 tx=0x7f271800dc90 comp rx=0 tx=0).stop 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 shutdown_connections 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.327+0000 7f2731200700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f271006c680 0x7f271006eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.328+0000 7f2731200700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f272c106840 0x7f272c19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.328+0000 7f2731200700 1 --2- 172.21.15.90:0/3732734446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c107a40 0x7f272c19c6a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:11.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.328+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 >> 172.21.15.90:0/3732734446 conn(0x7f272c075970 msgr2=0x7f272c10ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:11.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.328+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 shutdown_connections 2024-04-24T08:21:11.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:11.328+0000 7f2731200700 1 -- 172.21.15.90:0/3732734446 wait complete. 2024-04-24T08:21:11.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:11 smithi090 ceph-mon[25016]: pgmap v255: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-24T08:21:11.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:11 smithi149 ceph-mon[31282]: pgmap v255: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-24T08:21:12.410 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-24T08:21:12.808 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:21:12.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:12 smithi090 ceph-mon[25016]: pgmap v256: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-24T08:21:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:12 smithi149 ceph-mon[31282]: pgmap v256: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-24T08:21:15.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:15 smithi090 ceph-mon[25016]: pgmap v257: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.424+0000 7f06db200700 1 -- 172.21.15.90:0/4138252659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 msgr2=0x7f06dc073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.424+0000 7f06db200700 1 --2- 172.21.15.90:0/4138252659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc073b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009910 tx=0x7f06c4009c20 comp rx=0 tx=0).stop 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 -- 172.21.15.90:0/4138252659 shutdown_connections 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 --2- 172.21.15.90:0/4138252659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc073b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 --2- 172.21.15.90:0/4138252659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 0x7f06dc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 -- 172.21.15.90:0/4138252659 >> 172.21.15.90:0/4138252659 conn(0x7f06dc0fb650 msgr2=0x7f06dc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 -- 172.21.15.90:0/4138252659 shutdown_connections 2024-04-24T08:21:15.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.425+0000 7f06db200700 1 -- 172.21.15.90:0/4138252659 wait complete. 2024-04-24T08:21:15.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.426+0000 7f06db200700 1 Processor -- start 2024-04-24T08:21:15.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.426+0000 7f06db200700 1 -- start start 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06db200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06db200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 0x7f06dc19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06db200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc19ca00 con 0x7f06dc073720 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06db200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc19cb40 con 0x7f06dc074ce0 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42764/0 (socket says 172.21.15.90:42764) 2024-04-24T08:21:15.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06d9e00700 1 -- 172.21.15.90:0/3041887884 learned_addr learned my addr 172.21.15.90:0/3041887884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:21:15.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.427+0000 7f06d1400700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 0x7f06dc19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:15.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d9e00700 1 -- 172.21.15.90:0/3041887884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 msgr2=0x7f06dc19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:21:15.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d9e00700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 0x7f06dc19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d9e00700 1 -- 172.21.15.90:0/3041887884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40095f0 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d9e00700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b9c0 tx=0x7f06cc00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc00c980 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f06cc014440 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc1a15f0 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc012550 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.428+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1a1bc0 con 0x7f06dc073720 2024-04-24T08:21:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.430+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc10f180 con 0x7f06dc073720 2024-04-24T08:21:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.431+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f06cc0145b0 con 0x7f06dc073720 2024-04-24T08:21:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.432+0000 7f06d3400700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 0x7f06c006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:21:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.432+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f06cc006e40 con 0x7f06dc073720 2024-04-24T08:21:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.432+0000 7f06d1400700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 0x7f06c006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:21:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.433+0000 7f06d1400700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 0x7f06c006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000c00 tx=0x7f06c4005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:21:15.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.437+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f06cc04e720 con 0x7f06dc073720 2024-04-24T08:21:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:15 smithi149 ceph-mon[31282]: pgmap v257: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-24T08:21:15.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:21:15.669+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f06dc103170 con 0x7f06c006c680 2024-04-24T08:21:17.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:17 smithi149 ceph-mon[31282]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:21:17.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:17 smithi149 ceph-mon[31282]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:17 smithi090 ceph-mon[25016]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:21:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:17 smithi090 ceph-mon[25016]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:19 smithi149 ceph-mon[31282]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:19 smithi090 ceph-mon[25016]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:21.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:21 smithi090 ceph-mon[25016]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:21 smithi149 ceph-mon[31282]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:22.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:22 smithi090 ceph-mon[25016]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:22 smithi149 ceph-mon[31282]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-24T08:21:25.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:25 smithi149 ceph-mon[31282]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-24T08:21:25.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:25 smithi090 ceph-mon[25016]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-24T08:21:27.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:27 smithi149 ceph-mon[31282]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-24T08:21:27.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:27 smithi090 ceph-mon[25016]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-24T08:21:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:29 smithi149 ceph-mon[31282]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:29 smithi090 ceph-mon[25016]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:31 smithi149 ceph-mon[31282]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:31 smithi090 ceph-mon[25016]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:33.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:32 smithi149 ceph-mon[31282]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:33.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:32 smithi090 ceph-mon[25016]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:35 smithi149 ceph-mon[31282]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:35.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:35 smithi090 ceph-mon[25016]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:35.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:37 smithi149 ceph-mon[31282]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:37 smithi090 ceph-mon[25016]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:39 smithi149 ceph-mon[31282]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:39 smithi090 ceph-mon[25016]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:41 smithi149 ceph-mon[31282]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:41 smithi090 ceph-mon[25016]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:43.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:42 smithi149 ceph-mon[31282]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:43.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:42 smithi090 ceph-mon[25016]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:45 smithi149 ceph-mon[31282]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:45 smithi090 ceph-mon[25016]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:21:47.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:47 smithi149 ceph-mon[31282]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:47.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:47 smithi090 ceph-mon[25016]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:48.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:48 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:21:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:48 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:21:49.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:49 smithi149 ceph-mon[31282]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:49.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:49 smithi090 ceph-mon[25016]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:50.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:50.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:21:51.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:51 smithi149 ceph-mon[31282]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:51 smithi090 ceph-mon[25016]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:53.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:52 smithi090 ceph-mon[25016]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:53.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:52 smithi149 ceph-mon[31282]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:55 smithi149 ceph-mon[31282]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:55 smithi090 ceph-mon[25016]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:57.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:57 smithi149 ceph-mon[31282]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:57 smithi090 ceph-mon[25016]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:21:58.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:21:58.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:21:58.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:58 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:21:58.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:58 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:21:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:58 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:21:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:58 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:21:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:21:59 smithi149 ceph-mon[31282]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:21:59.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:21:59 smithi090 ceph-mon[25016]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:01 smithi149 ceph-mon[31282]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:01 smithi090 ceph-mon[25016]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:03.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:02 smithi149 ceph-mon[31282]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:03.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:02 smithi090 ceph-mon[25016]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:05.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:05 smithi149 ceph-mon[31282]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:05.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:05 smithi090 ceph-mon[25016]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:05.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:07.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:07 smithi090 ceph-mon[25016]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:07.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:07 smithi149 ceph-mon[31282]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:08.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.186+0000 7f06d3400700 1 -- 172.21.15.90:0/3041887884 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f06dc103170 con 0x7f06c006c680 2024-04-24T08:22:08.188 INFO:teuthology.orchestra.run.smithi090.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:22:08.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.189+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 msgr2=0x7f06c006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:08.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.189+0000 7f06db200700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 0x7f06c006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000c00 tx=0x7f06c4005fb0 comp rx=0 tx=0).stop 2024-04-24T08:22:08.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.189+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 msgr2=0x7f06dc19bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.189+0000 7f06db200700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b9c0 tx=0x7f06cc00bd80 comp rx=0 tx=0).stop 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 shutdown_connections 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f06c006c680 0x7f06c006eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06dc073720 0x7f06dc19bea0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 --2- 172.21.15.90:0/3041887884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06dc074ce0 0x7f06dc19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 >> 172.21.15.90:0/3041887884 conn(0x7f06dc0fb650 msgr2=0x7f06dc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.190+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 shutdown_connections 2024-04-24T08:22:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:08.191+0000 7f06db200700 1 -- 172.21.15.90:0/3041887884 wait complete. 2024-04-24T08:22:08.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-24T08:22:09.188 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:09.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:22:09.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:22:09.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.412 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:22:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:22:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:22:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:09 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:22:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:09 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:10.496 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:10 smithi090 ceph-mon[25016]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:22:10.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:10 smithi149 ceph-mon[31282]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:22:11.397 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:11 smithi090 ceph-mon[25016]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:11.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:11 smithi149 ceph-mon[31282]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:11.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 -- 172.21.15.90:0/3357488055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c073750 msgr2=0x7f635c073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:11.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 --2- 172.21.15.90:0/3357488055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c073750 0x7f635c073bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6348009910 tx=0x7f6348009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:11.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 -- 172.21.15.90:0/3357488055 shutdown_connections 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 --2- 172.21.15.90:0/3357488055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c073750 0x7f635c073bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 --2- 172.21.15.90:0/3357488055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c074d20 0x7f635c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 -- 172.21.15.90:0/3357488055 >> 172.21.15.90:0/3357488055 conn(0x7f635c0fb7f0 msgr2=0x7f635c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 -- 172.21.15.90:0/3357488055 shutdown_connections 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.708+0000 7f6362a00700 1 -- 172.21.15.90:0/3357488055 wait complete. 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.709+0000 7f6362a00700 1 Processor -- start 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.710+0000 7f6362a00700 1 -- start start 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.710+0000 7f6362a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c073750 0x7f635c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.710+0000 7f6362a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.710+0000 7f6362a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c1987b0 con 0x7f635c074d20 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.710+0000 7f6362a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c198920 con 0x7f635c073750 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.711+0000 7f635b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.711+0000 7f635b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53760/0 (socket says 172.21.15.90:53760) 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.711+0000 7f635b400700 1 -- 172.21.15.90:0/3262903547 learned_addr learned my addr 172.21.15.90:0/3262903547 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.711+0000 7f635be00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c073750 0x7f635c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.711+0000 7f635b400700 1 -- 172.21.15.90:0/3262903547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c073750 msgr2=0x7f635c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f635b400700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c073750 0x7f635c197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f635b400700 1 -- 172.21.15.90:0/3262903547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63480095f0 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f635b400700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63480092e0 tx=0x7f634800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634801d070 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63480052a0 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6348017630 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.712+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c19d350 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.713+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c19d840 con 0x7f635c074d20 2024-04-24T08:22:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.714+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c10af80 con 0x7f635c074d20 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.717+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f634800fba0 con 0x7f635c074d20 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.719+0000 7f6358c00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 0x7f634006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.719+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f634808cd30 con 0x7f635c074d20 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.721+0000 7f635be00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 0x7f634006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.721+0000 7f635be00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 0x7f634006eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f635c198b50 tx=0x7f634c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:11.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:11.728+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6348057430 con 0x7f635c074d20 2024-04-24T08:22:12.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.011+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f635c1077a0 con 0x7f634006c680 2024-04-24T08:22:12.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.013+0000 7f6358c00700 1 -- 172.21.15.90:0/3262903547 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f635c1077a0 con 0x7f634006c680 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.016+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 msgr2=0x7f634006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.016+0000 7f6362a00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 0x7f634006eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f635c198b50 tx=0x7f634c009260 comp rx=0 tx=0).stop 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.016+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 msgr2=0x7f635c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.016+0000 7f6362a00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63480092e0 tx=0x7f634800f740 comp rx=0 tx=0).stop 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 shutdown_connections 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f634006c680 0x7f634006eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f635c073750 0x7f635c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 --2- 172.21.15.90:0/3262903547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f635c074d20 0x7f635c1981b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 >> 172.21.15.90:0/3262903547 conn(0x7f635c0fb7f0 msgr2=0x7f635c106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:12.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 shutdown_connections 2024-04-24T08:22:12.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.017+0000 7f6362a00700 1 -- 172.21.15.90:0/3262903547 wait complete. 2024-04-24T08:22:12.041 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 -- 172.21.15.90:0/3690447241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 msgr2=0x7f80b0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 --2- 172.21.15.90:0/3690447241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b0073c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 -- 172.21.15.90:0/3690447241 shutdown_connections 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 --2- 172.21.15.90:0/3690447241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b0073c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 --2- 172.21.15.90:0/3690447241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b00731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 -- 172.21.15.90:0/3690447241 >> 172.21.15.90:0/3690447241 conn(0x7f80b00fb7f0 msgr2=0x7f80b00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 -- 172.21.15.90:0/3690447241 shutdown_connections 2024-04-24T08:22:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.183+0000 7f80b6200700 1 -- 172.21.15.90:0/3690447241 wait complete. 2024-04-24T08:22:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 Processor -- start 2024-04-24T08:22:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 -- start start 2024-04-24T08:22:12.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b019cbd0 con 0x7f80b0074d80 2024-04-24T08:22:12.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.185+0000 7f80b6200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b019cd10 con 0x7f80b00737b0 2024-04-24T08:22:12.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.186+0000 7f80af400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.186+0000 7f80aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.186+0000 7f80af400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:34838/0 (socket says 172.21.15.90:34838) 2024-04-24T08:22:12.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.186+0000 7f80af400700 1 -- 172.21.15.90:0/575349300 learned_addr learned my addr 172.21.15.90:0/575349300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:12.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.187+0000 7f80aea00700 1 -- 172.21.15.90:0/575349300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 msgr2=0x7f80b019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.187+0000 7f80aea00700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.187+0000 7f80aea00700 1 -- 172.21.15.90:0/575349300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c0095f0 con 0x7f80b0074d80 2024-04-24T08:22:12.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.187+0000 7f80af400700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:22:12.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.188+0000 7f80aea00700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b019c5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:12.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.188+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c01d070 con 0x7f80b0074d80 2024-04-24T08:22:12.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.188+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f809c005dc0 con 0x7f80b0074d80 2024-04-24T08:22:12.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.188+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c017400 con 0x7f80b0074d80 2024-04-24T08:22:12.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.188+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b01a1760 con 0x7f80b0074d80 2024-04-24T08:22:12.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.189+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b01a1c50 con 0x7f80b0074d80 2024-04-24T08:22:12.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.190+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b010f380 con 0x7f80b0074d80 2024-04-24T08:22:12.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.193+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f809c003810 con 0x7f80b0074d80 2024-04-24T08:22:12.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.194+0000 7f80a7e00700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 0x7f809406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.195+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f809c08cc70 con 0x7f80b0074d80 2024-04-24T08:22:12.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.195+0000 7f80af400700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 0x7f809406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.196+0000 7f80af400700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 0x7f809406eb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011550 tx=0x7f80a0009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:12.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.204+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f809c093080 con 0x7f80b0074d80 2024-04-24T08:22:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.479+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80b01a23a0 con 0x7f809406c6d0 2024-04-24T08:22:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.480+0000 7f80a7e00700 1 -- 172.21.15.90:0/575349300 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f80b01a23a0 con 0x7f809406c6d0 2024-04-24T08:22:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 msgr2=0x7f809406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 0x7f809406eb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011550 tx=0x7f80a0009260 comp rx=0 tx=0).stop 2024-04-24T08:22:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 msgr2=0x7f80b019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b019c5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c00f690 comp rx=0 tx=0).stop 2024-04-24T08:22:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 shutdown_connections 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f809406c6d0 0x7f809406eb80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80b00737b0 0x7f80b019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 --2- 172.21.15.90:0/575349300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b0074d80 0x7f80b019c5b0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c00f690 comp rx=0 tx=0).stop 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 >> 172.21.15.90:0/575349300 conn(0x7f80b00fb7f0 msgr2=0x7f80b0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.483+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 shutdown_connections 2024-04-24T08:22:12.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.484+0000 7f80b6200700 1 -- 172.21.15.90:0/575349300 wait complete. 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 -- 172.21.15.90:0/2049436990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef8100020 msgr2=0x7faef8100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 --2- 172.21.15.90:0/2049436990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef8100020 0x7faef8100430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faee8009910 tx=0x7faee8009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 -- 172.21.15.90:0/2049436990 shutdown_connections 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 --2- 172.21.15.90:0/2049436990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef81012c0 0x7faef8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 --2- 172.21.15.90:0/2049436990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef8100020 0x7faef8100430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 -- 172.21.15.90:0/2049436990 >> 172.21.15.90:0/2049436990 conn(0x7faef80fb650 msgr2=0x7faef80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 -- 172.21.15.90:0/2049436990 shutdown_connections 2024-04-24T08:22:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.684+0000 7faef7e00700 1 -- 172.21.15.90:0/2049436990 wait complete. 2024-04-24T08:22:12.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.685+0000 7faef7e00700 1 Processor -- start 2024-04-24T08:22:12.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.685+0000 7faef7e00700 1 -- start start 2024-04-24T08:22:12.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef81012c0 0x7faef810a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8106960 con 0x7faef81012c0 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8106ad0 con 0x7faef8100020 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:34862/0 (socket says 172.21.15.90:34862) 2024-04-24T08:22:12.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef6a00700 1 -- 172.21.15.90:0/745342187 learned_addr learned my addr 172.21.15.90:0/745342187 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.686+0000 7faef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef81012c0 0x7faef810a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.687+0000 7faef6a00700 1 -- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef81012c0 msgr2=0x7faef810a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.687+0000 7faef6a00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef81012c0 0x7faef810a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.687+0000 7faef6a00700 1 -- 172.21.15.90:0/745342187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80095f0 con 0x7faef8100020 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.687+0000 7faef6a00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faee800b5c0 tx=0x7faee800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.688+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee801d070 con 0x7faef8100020 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.688+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8106d50 con 0x7faef8100020 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.688+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef8107240 con 0x7faef8100020 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.688+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faee800f460 con 0x7faef8100020 2024-04-24T08:22:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.688+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee8021640 con 0x7faef8100020 2024-04-24T08:22:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.689+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8005320 con 0x7faef8100020 2024-04-24T08:22:12.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.691+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7faee8003a60 con 0x7faef8100020 2024-04-24T08:22:12.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.692+0000 7faeef400700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 0x7faedc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:12.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.692+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faee808cee0 con 0x7faef8100020 2024-04-24T08:22:12.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.692+0000 7faef6000700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 0x7faedc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:12.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.694+0000 7faef6000700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 0x7faedc06eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faee4005950 tx=0x7faee40058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:12.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:12.701+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faee805b050 con 0x7faef8100020 2024-04-24T08:22:13.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:12 smithi149 ceph-mon[31282]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:12 smithi149 ceph-mon[31282]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:12 smithi149 ceph-mon[31282]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:13.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:12 smithi090 ceph-mon[25016]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:12 smithi090 ceph-mon[25016]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:12 smithi090 ceph-mon[25016]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:13.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.120+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faed8000bf0 con 0x7faedc06c680 2024-04-24T08:22:13.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.140+0000 7faeef400700 1 -- 172.21.15.90:0/745342187 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7faed8000bf0 con 0x7faedc06c680 2024-04-24T08:22:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 86s ago 9m 13.3M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (9m) 86s ago 9m 7499k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (8m) 93s ago 8m 7411k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (9m) 86s ago 9m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (8m) 93s ago 8m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 86s ago 8m 74.4M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (112s) 86s ago 111s 12.7M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (2m) 86s ago 2m 11.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (117s) 93s ago 117s 14.6M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (108s) 93s ago 107s 11.6M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:9283,8765,8443 running (10m) 86s ago 10m 487M - 18.2.0 dc2bc1663786 7ccc358e4d84 2024-04-24T08:22:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (7m) 93s ago 7m 437M - 18.2.0 dc2bc1663786 30e493581e1d 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (11m) 86s ago 11m 43.3M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (7m) 93s ago 7m 34.1M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 86s ago 9m 7982k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 93s ago 7m 7448k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (6m) 86s ago 6m 47.3M 3947M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (6m) 86s ago 6m 47.2M 3947M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (5m) 86s ago 5m 45.6M 3947M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (4m) 93s ago 4m 45.6M 1213M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (4m) 93s ago 4m 43.5M 1213M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (3m) 93s ago 3m 43.1M 1213M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:22:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 86s ago 8m 54.3M - 2.43.0 a07b618ecd1d 0199ca15cf9f 2024-04-24T08:22:13.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 msgr2=0x7faedc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 0x7faedc06eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faee4005950 tx=0x7faee40058e0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 msgr2=0x7faef8109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faee800b5c0 tx=0x7faee800be20 comp rx=0 tx=0).stop 2024-04-24T08:22:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 shutdown_connections 2024-04-24T08:22:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7faedc06c680 0x7faedc06eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faef8100020 0x7faef8109e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 --2- 172.21.15.90:0/745342187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faef81012c0 0x7faef810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.144+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 >> 172.21.15.90:0/745342187 conn(0x7faef80fb650 msgr2=0x7faef81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.145+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 shutdown_connections 2024-04-24T08:22:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.146+0000 7faef7e00700 1 -- 172.21.15.90:0/745342187 wait complete. 2024-04-24T08:22:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 -- 172.21.15.90:0/775276088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 msgr2=0x7f8c541057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 --2- 172.21.15.90:0/775276088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c541057b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009910 tx=0x7f8c40009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:13.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 -- 172.21.15.90:0/775276088 shutdown_connections 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 --2- 172.21.15.90:0/775276088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c541057b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 --2- 172.21.15.90:0/775276088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 -- 172.21.15.90:0/775276088 >> 172.21.15.90:0/775276088 conn(0x7f8c540fa4d0 msgr2=0x7f8c540fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 -- 172.21.15.90:0/775276088 shutdown_connections 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.372+0000 7f8c53200700 1 -- 172.21.15.90:0/775276088 wait complete. 2024-04-24T08:22:13.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.374+0000 7f8c53200700 1 Processor -- start 2024-04-24T08:22:13.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.374+0000 7f8c53200700 1 -- start start 2024-04-24T08:22:13.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.374+0000 7f8c53200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c53200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c54197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c53200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c541985c0 con 0x7f8c541033d0 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c53200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54198700 con 0x7f8c54100ab0 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c51e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c51400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c54197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c51e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:34876/0 (socket says 172.21.15.90:34876) 2024-04-24T08:22:13.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.375+0000 7f8c51e00700 1 -- 172.21.15.90:0/3093398109 learned_addr learned my addr 172.21.15.90:0/3093398109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.376+0000 7f8c51400700 1 -- 172.21.15.90:0/3093398109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 msgr2=0x7f8c54197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.376+0000 7f8c51400700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.376+0000 7f8c51400700 1 -- 172.21.15.90:0/3093398109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c400095f0 con 0x7f8c541033d0 2024-04-24T08:22:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.376+0000 7f8c51e00700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:22:13.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.376+0000 7f8c51400700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c54197fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009910 tx=0x7f8c4000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.377+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4001d070 con 0x7f8c541033d0 2024-04-24T08:22:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.377+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c40005dc0 con 0x7f8c541033d0 2024-04-24T08:22:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.377+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c40017400 con 0x7f8c541033d0 2024-04-24T08:22:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.377+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5419d150 con 0x7f8c541033d0 2024-04-24T08:22:13.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.377+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5419d640 con 0x7f8c541033d0 2024-04-24T08:22:13.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.378+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540fc090 con 0x7f8c541033d0 2024-04-24T08:22:13.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.381+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8c40003810 con 0x7f8c541033d0 2024-04-24T08:22:13.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.382+0000 7f8c4aa00700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 0x7f8c3806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:13.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.383+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8c4008cd10 con 0x7f8c541033d0 2024-04-24T08:22:13.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.383+0000 7f8c51e00700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 0x7f8c3806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:13.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.384+0000 7f8c51e00700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 0x7f8c3806eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c44011550 tx=0x7f8c44009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:13.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.389+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8c4005ae80 con 0x7f8c541033d0 2024-04-24T08:22:13.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.939+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c5402cc30 con 0x7f8c541033d0 2024-04-24T08:22:13.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.940+0000 7f8c4aa00700 1 -- 172.21.15.90:0/3093398109 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f8c400163b0 con 0x7f8c541033d0 2024-04-24T08:22:13.941 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:22:13.942 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-24T08:22:13.943 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:22:13.943 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 msgr2=0x7f8c3806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 0x7f8c3806eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c44011550 tx=0x7f8c44009260 comp rx=0 tx=0).stop 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 msgr2=0x7f8c54197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c54197fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009910 tx=0x7f8c4000f690 comp rx=0 tx=0).stop 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 shutdown_connections 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f8c3806c680 0x7f8c3806eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54100ab0 0x7f8c54197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 --2- 172.21.15.90:0/3093398109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c541033d0 0x7f8c54197fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:13.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 >> 172.21.15.90:0/3093398109 conn(0x7f8c540fa4d0 msgr2=0x7f8c54100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:13.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.943+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 shutdown_connections 2024-04-24T08:22:13.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:13.944+0000 7f8c53200700 1 -- 172.21.15.90:0/3093398109 wait complete. 2024-04-24T08:22:14.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:13 smithi149 ceph-mon[31282]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:14.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:13 smithi090 ceph-mon[25016]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 -- 172.21.15.90:0/2954864247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880fe8a0 msgr2=0x7ff188100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 --2- 172.21.15.90:0/2954864247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880fe8a0 0x7ff188100cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff178009910 tx=0x7ff178009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 -- 172.21.15.90:0/2954864247 shutdown_connections 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 --2- 172.21.15.90:0/2954864247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff188101200 0x7ff188103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 --2- 172.21.15.90:0/2954864247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880fe8a0 0x7ff188100cc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 -- 172.21.15.90:0/2954864247 >> 172.21.15.90:0/2954864247 conn(0x7ff1880fa4d0 msgr2=0x7ff1880fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:14.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.101+0000 7ff18d800700 1 -- 172.21.15.90:0/2954864247 shutdown_connections 2024-04-24T08:22:14.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.102+0000 7ff18d800700 1 -- 172.21.15.90:0/2954864247 wait complete. 2024-04-24T08:22:14.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.103+0000 7ff18d800700 1 Processor -- start 2024-04-24T08:22:14.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.103+0000 7ff18d800700 1 -- start start 2024-04-24T08:22:14.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff18d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 0x7ff18819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff18d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 0x7ff18819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff18d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18819c990 con 0x7ff188101200 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff18d800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18819cad0 con 0x7ff1880fe8a0 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff187e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 0x7ff18819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff187e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 0x7ff18819be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:34898/0 (socket says 172.21.15.90:34898) 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff187e00700 1 -- 172.21.15.90:0/2885011810 learned_addr learned my addr 172.21.15.90:0/2885011810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.104+0000 7ff187400700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 0x7ff18819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:14.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.105+0000 7ff187400700 1 -- 172.21.15.90:0/2885011810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 msgr2=0x7ff18819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:14.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.105+0000 7ff187400700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 0x7ff18819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.105+0000 7ff187400700 1 -- 172.21.15.90:0/2885011810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780095f0 con 0x7ff188101200 2024-04-24T08:22:14.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.105+0000 7ff187400700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 0x7ff18819c370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e970 tx=0x7ff17400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:14.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.106+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1740096d0 con 0x7ff188101200 2024-04-24T08:22:14.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.106+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff17400cba0 con 0x7ff188101200 2024-04-24T08:22:14.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.106+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff174018650 con 0x7ff188101200 2024-04-24T08:22:14.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.106+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1881a1580 con 0x7ff188101200 2024-04-24T08:22:14.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.106+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1881a1a80 con 0x7ff188101200 2024-04-24T08:22:14.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.107+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff168005320 con 0x7ff188101200 2024-04-24T08:22:14.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.109+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff17400cd10 con 0x7ff188101200 2024-04-24T08:22:14.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.111+0000 7ff184c00700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 0x7ff16c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:14.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.111+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff174014070 con 0x7ff188101200 2024-04-24T08:22:14.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.111+0000 7ff187e00700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 0x7ff16c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:14.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.112+0000 7ff187e00700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 0x7ff16c06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff178005b40 tx=0x7ff178005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:14.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.119+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff174051720 con 0x7ff188101200 2024-04-24T08:22:14.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.525+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff168000bf0 con 0x7ff16c06c680 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.527+0000 7ff184c00700 1 -- 172.21.15.90:0/2885011810 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff168000bf0 con 0x7ff16c06c680 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T08:22:14.528 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T08:22:14.529 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:22:14.529 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:22:14.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.530+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 msgr2=0x7ff16c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.530+0000 7ff18d800700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 0x7ff16c06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff178005b40 tx=0x7ff178005a90 comp rx=0 tx=0).stop 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.530+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 msgr2=0x7ff18819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.530+0000 7ff18d800700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 0x7ff18819c370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e970 tx=0x7ff17400ed30 comp rx=0 tx=0).stop 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 shutdown_connections 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff16c06c680 0x7ff16c06eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1880fe8a0 0x7ff18819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 --2- 172.21.15.90:0/2885011810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188101200 0x7ff18819c370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 >> 172.21.15.90:0/2885011810 conn(0x7ff1880fa4d0 msgr2=0x7ff1880fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:14.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 shutdown_connections 2024-04-24T08:22:14.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:14.531+0000 7ff18d800700 1 -- 172.21.15.90:0/2885011810 wait complete. 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3093398109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: Upgrade: Target is version 19.0.0-3116-g1f81fb39 (unknown) 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, digests ['quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644'] 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:14.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: Upgrade: Need to upgrade myself (mgr.smithi090.jsrfxo) 2024-04-24T08:22:14.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:14 smithi090 ceph-mon[25016]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:15.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3093398109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: Upgrade: Target is version 19.0.0-3116-g1f81fb39 (unknown) 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, digests ['quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644'] 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: Upgrade: Need to upgrade myself (mgr.smithi090.jsrfxo) 2024-04-24T08:22:15.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:14 smithi149 ceph-mon[31282]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:17 smithi149 ceph-mon[31282]: pgmap v288: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:17 smithi149 ceph-mon[31282]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 on smithi149 2024-04-24T08:22:17.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:17 smithi090 ceph-mon[25016]: pgmap v288: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:17 smithi090 ceph-mon[25016]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 on smithi149 2024-04-24T08:22:19.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:19 smithi149 ceph-mon[31282]: pgmap v289: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:19.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:19 smithi090 ceph-mon[25016]: pgmap v289: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:20.513 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:21.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:21 smithi149 ceph-mon[31282]: pgmap v290: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:21.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:21 smithi090 ceph-mon[25016]: pgmap v290: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:22.890 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:22 smithi090 ceph-mon[25016]: pgmap v291: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:23.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:22 smithi149 ceph-mon[31282]: pgmap v291: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:25.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:25 smithi149 ceph-mon[31282]: pgmap v292: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:25.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:25 smithi090 ceph-mon[25016]: pgmap v292: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:27.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:27 smithi149 ceph-mon[31282]: pgmap v293: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:27.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:27 smithi090 ceph-mon[25016]: pgmap v293: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:29.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:29 smithi149 ceph-mon[31282]: pgmap v294: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:29 smithi090 ceph-mon[25016]: pgmap v294: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:31.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:31 smithi149 ceph-mon[31282]: pgmap v295: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:31 smithi090 ceph-mon[25016]: pgmap v295: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:33.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:32 smithi149 ceph-mon[31282]: pgmap v296: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:33.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:32 smithi090 ceph-mon[25016]: pgmap v296: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:35 smithi149 ceph-mon[31282]: pgmap v297: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:35 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:35.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:35 smithi090 ceph-mon[25016]: pgmap v297: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:35.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:35 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:37.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:37 smithi149 ceph-mon[31282]: pgmap v298: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:37 smithi090 ceph-mon[25016]: pgmap v298: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:38.921 INFO:tasks.workunit.client.0.smithi090.stderr:Updating files: 61% (7564/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T08:22:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:39 smithi149 ceph-mon[31282]: pgmap v299: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:39 smithi090 ceph-mon[25016]: pgmap v299: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '1f81fb397ae98da7563d451a78c61574c8f4e6e0'. 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.925 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T08:22:39.926 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 1f81fb397ae DNM: trigger builds 2024-04-24T08:22:39.939 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T08:22:40.003 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T08:22:40.012 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T08:22:40.012 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T08:22:41.168 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T08:22:41.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:41 smithi090 ceph-mon[25016]: pgmap v300: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:41.316 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T08:22:41.402 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T08:22:41.409 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T08:22:41.409 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T08:22:41.487 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T08:22:41.496 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:22:41.496 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T08:22:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:41 smithi149 ceph-mon[31282]: pgmap v300: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:41.544 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-24T08:22:41.546 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T08:22:41.546 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-24T08:22:41.673 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p fsstress 2024-04-24T08:22:41.681 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd fsstress 2024-04-24T08:22:41.686 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T08:22:41.686 INFO:tasks.workunit.client.0.smithi090.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T08:22:43.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:42 smithi149 ceph-mon[31282]: pgmap v301: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:43.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:42 smithi090 ceph-mon[25016]: pgmap v301: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:22:43.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ tar xzf ltp-full.tgz 2024-04-24T08:22:44.637 INFO:tasks.workunit.client.1.smithi149.stderr:Updating files: 50% (6175/12271) Updating files: 51% (6259/12271) Updating files: 52% (6381/12271) Updating files: 53% (6504/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 77% (9495/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T08:22:44.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 -- 172.21.15.90:0/3175941620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 msgr2=0x7fe4480ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 --2- 172.21.15.90:0/3175941620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe4480ff330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009910 tx=0x7fe43c009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 -- 172.21.15.90:0/3175941620 shutdown_connections 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 --2- 172.21.15.90:0/3175941620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 0x7fe4481004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 --2- 172.21.15.90:0/3175941620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe4480ff330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 -- 172.21.15.90:0/3175941620 >> 172.21.15.90:0/3175941620 conn(0x7fe4480fa4d0 msgr2=0x7fe4480fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 -- 172.21.15.90:0/3175941620 shutdown_connections 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.713+0000 7fe44e800700 1 -- 172.21.15.90:0/3175941620 wait complete. 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 Processor -- start 2024-04-24T08:22:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 -- start start 2024-04-24T08:22:44.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 0x7fe4481014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448101a30 con 0x7fe4480fef20 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.715+0000 7fe44e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448101b70 con 0x7fe448100080 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33992/0 (socket says 172.21.15.90:33992) 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 -- 172.21.15.90:0/1162873434 learned_addr learned my addr 172.21.15.90:0/1162873434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44ca00700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 0x7fe4481014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 -- 172.21.15.90:0/1162873434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 msgr2=0x7fe4481014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 0x7fe4481014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.716+0000 7fe44d400700 1 -- 172.21.15.90:0/1162873434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0095f0 con 0x7fe4480fef20 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe44d400700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0092e0 tx=0x7fe43c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:44.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe43c01d070 con 0x7fe4480fef20 2024-04-24T08:22:44.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe43c005dc0 con 0x7fe4480fef20 2024-04-24T08:22:44.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe43c017630 con 0x7fe4480fef20 2024-04-24T08:22:44.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448101df0 con 0x7fe4480fef20 2024-04-24T08:22:44.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.717+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4481022e0 con 0x7fe4480fef20 2024-04-24T08:22:44.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.718+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4481079f0 con 0x7fe4480fef20 2024-04-24T08:22:44.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.720+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe43c003680 con 0x7fe4480fef20 2024-04-24T08:22:44.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.721+0000 7fe446000700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 0x7fe43006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:44.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.721+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe43c08cd30 con 0x7fe4480fef20 2024-04-24T08:22:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.721+0000 7fe44ca00700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 0x7fe43006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.722+0000 7fe44ca00700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 0x7fe43006eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe434009c60 tx=0x7fe434009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:44.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.726+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe43c05ae20 con 0x7fe4480fef20 2024-04-24T08:22:45.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:44.999+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe44819dd30 con 0x7fe43006c680 2024-04-24T08:22:45.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.001+0000 7fe446000700 1 -- 172.21.15.90:0/1162873434 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fe44819dd30 con 0x7fe43006c680 2024-04-24T08:22:45.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.004+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 msgr2=0x7fe43006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.004+0000 7fe44e800700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 0x7fe43006eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe434009c60 tx=0x7fe434009260 comp rx=0 tx=0).stop 2024-04-24T08:22:45.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.004+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 msgr2=0x7fe448102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.004+0000 7fe44e800700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0092e0 tx=0x7fe43c00f690 comp rx=0 tx=0).stop 2024-04-24T08:22:45.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 shutdown_connections 2024-04-24T08:22:45.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fe43006c680 0x7fe43006eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4480fef20 0x7fe448102ea0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 --2- 172.21.15.90:0/1162873434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe448100080 0x7fe4481014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 >> 172.21.15.90:0/1162873434 conn(0x7fe4480fa4d0 msgr2=0x7fe448068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:45.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.005+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 shutdown_connections 2024-04-24T08:22:45.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.006+0000 7fe44e800700 1 -- 172.21.15.90:0/1162873434 wait complete. 2024-04-24T08:22:45.024 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 -- 172.21.15.90:0/3445610501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 msgr2=0x7ff8e0105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/3445610501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0105870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009910 tx=0x7ff8cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 -- 172.21.15.90:0/3445610501 shutdown_connections 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/3445610501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0105870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/3445610501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 -- 172.21.15.90:0/3445610501 >> 172.21.15.90:0/3445610501 conn(0x7ff8e00fa4d0 msgr2=0x7ff8e00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 -- 172.21.15.90:0/3445610501 shutdown_connections 2024-04-24T08:22:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.177+0000 7ff8e5e00700 1 -- 172.21.15.90:0/3445610501 wait complete. 2024-04-24T08:22:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.178+0000 7ff8e5e00700 1 Processor -- start 2024-04-24T08:22:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.178+0000 7ff8e5e00700 1 -- start start 2024-04-24T08:22:45.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8e5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8e5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8e5e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01985c0 con 0x7ff8e0103490 2024-04-24T08:22:45.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8e5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0198700 con 0x7ff8e0100b70 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8d7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8d7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34000/0 (socket says 172.21.15.90:34000) 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8d7e00700 1 -- 172.21.15.90:0/1598315332 learned_addr learned my addr 172.21.15.90:0/1598315332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.179+0000 7ff8e4a00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8d7e00700 1 -- 172.21.15.90:0/1598315332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 msgr2=0x7ff8e0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8d7e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8d7e00700 1 -- 172.21.15.90:0/1598315332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0095f0 con 0x7ff8e0103490 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8e4a00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:22:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8d7e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0092e0 tx=0x7ff8cc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8cc01d070 con 0x7ff8e0103490 2024-04-24T08:22:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff8cc005dc0 con 0x7ff8e0103490 2024-04-24T08:22:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e019d150 con 0x7ff8e0103490 2024-04-24T08:22:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.180+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8cc017630 con 0x7ff8e0103490 2024-04-24T08:22:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.181+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e019d640 con 0x7ff8e0103490 2024-04-24T08:22:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.182+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e00fc090 con 0x7ff8e0103490 2024-04-24T08:22:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.183+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff8cc00fba0 con 0x7ff8e0103490 2024-04-24T08:22:45.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.184+0000 7ff8de000700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 0x7ff8c406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.184+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff8cc08cf60 con 0x7ff8e0103490 2024-04-24T08:22:45.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.184+0000 7ff8e4a00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 0x7ff8c406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.185+0000 7ff8e4a00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 0x7ff8c406eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00093a0 tx=0x7ff8d000b040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:45.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.189+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff8cc0bf720 con 0x7ff8e0103490 2024-04-24T08:22:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:45 smithi090 ceph-mon[25016]: pgmap v302: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:45.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.447+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8e0061190 con 0x7ff8c406c680 2024-04-24T08:22:45.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.451+0000 7ff8de000700 1 -- 172.21.15.90:0/1598315332 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff8e0061190 con 0x7ff8c406c680 2024-04-24T08:22:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 msgr2=0x7ff8c406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 0x7ff8c406eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00093a0 tx=0x7ff8d000b040 comp rx=0 tx=0).stop 2024-04-24T08:22:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 msgr2=0x7ff8e0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0092e0 tx=0x7ff8cc00f740 comp rx=0 tx=0).stop 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 shutdown_connections 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7ff8c406c680 0x7ff8c406eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e0100b70 0x7ff8e0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 --2- 172.21.15.90:0/1598315332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e0103490 0x7ff8e0197fa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 >> 172.21.15.90:0/1598315332 conn(0x7ff8e00fa4d0 msgr2=0x7ff8e01003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.454+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 shutdown_connections 2024-04-24T08:22:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.455+0000 7ff8e5e00700 1 -- 172.21.15.90:0/1598315332 wait complete. 2024-04-24T08:22:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:45 smithi149 ceph-mon[31282]: pgmap v302: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:22:45.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 -- 172.21.15.90:0/1105166777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed581033d0 msgr2=0x7fed581057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 --2- 172.21.15.90:0/1105166777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed581033d0 0x7fed581057b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed44009910 tx=0x7fed44009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:45.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 -- 172.21.15.90:0/1105166777 shutdown_connections 2024-04-24T08:22:45.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 --2- 172.21.15.90:0/1105166777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed581033d0 0x7fed581057b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 --2- 172.21.15.90:0/1105166777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed58100ab0 0x7fed58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 -- 172.21.15.90:0/1105166777 >> 172.21.15.90:0/1105166777 conn(0x7fed580fa4d0 msgr2=0x7fed580fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:45.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 -- 172.21.15.90:0/1105166777 shutdown_connections 2024-04-24T08:22:45.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.625+0000 7fed5ce00700 1 -- 172.21.15.90:0/1105166777 wait complete. 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.626+0000 7fed5ce00700 1 Processor -- start 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed5ce00700 1 -- start start 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed5ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed5ce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed581033d0 0x7fed58195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed5ce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58196400 con 0x7fed58100ab0 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed5ce00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58196540 con 0x7fed581033d0 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed57400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed57400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34028/0 (socket says 172.21.15.90:34028) 2024-04-24T08:22:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed57400700 1 -- 172.21.15.90:0/77002168 learned_addr learned my addr 172.21.15.90:0/77002168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:45.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.627+0000 7fed56a00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed581033d0 0x7fed58195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed57400700 1 -- 172.21.15.90:0/77002168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed581033d0 msgr2=0x7fed58195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed57400700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed581033d0 0x7fed58195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed57400700 1 -- 172.21.15.90:0/77002168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440095f0 con 0x7fed58100ab0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed57400700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b4b0 tx=0x7fed4800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed48011800 con 0x7fed58100ab0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fed48011e10 con 0x7fed58100ab0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4800f2e0 con 0x7fed58100ab0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.628+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5819aff0 con 0x7fed58100ab0 2024-04-24T08:22:45.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.629+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5819b530 con 0x7fed58100ab0 2024-04-24T08:22:45.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.630+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5818fac0 con 0x7fed58100ab0 2024-04-24T08:22:45.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.631+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fed48010380 con 0x7fed58100ab0 2024-04-24T08:22:45.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.632+0000 7fed4fe00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 0x7fed3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:45.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.632+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fed4808b4a0 con 0x7fed58100ab0 2024-04-24T08:22:45.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.632+0000 7fed56a00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 0x7fed3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:45.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.633+0000 7fed56a00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 0x7fed3c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed44006010 tx=0x7fed4400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:45.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.638+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fed48059590 con 0x7fed58100ab0 2024-04-24T08:22:45.685 INFO:tasks.workunit.client.1.smithi149.stderr:Note: switching to '1f81fb397ae98da7563d451a78c61574c8f4e6e0'. 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: git switch -c 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr:Or undo this operation with: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: git switch - 2024-04-24T08:22:45.686 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.687 INFO:tasks.workunit.client.1.smithi149.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T08:22:45.687 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-04-24T08:22:45.687 INFO:tasks.workunit.client.1.smithi149.stderr:HEAD is now at 1f81fb397ae DNM: trigger builds 2024-04-24T08:22:45.709 DEBUG:teuthology.orchestra.run.smithi149:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-24T08:22:45.770 INFO:tasks.workunit.client.1.smithi149.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T08:22:45.777 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T08:22:45.777 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T08:22:45.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.904+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed58061190 con 0x7fed3c06c680 2024-04-24T08:22:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.946+0000 7fed4fe00700 1 -- 172.21.15.90:0/77002168 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fed58061190 con 0x7fed3c06c680 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 119s ago 9m 13.3M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (10m) 119s ago 10m 7499k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (8m) 2m ago 8m 7411k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (10m) 119s ago 10m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (8m) 2m ago 8m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:22:45.948 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 119s ago 9m 74.4M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (2m) 119s ago 2m 12.7M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (2m) 119s ago 2m 11.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (2m) 2m ago 2m 14.6M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (2m) 2m ago 2m 11.6M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:9283,8765,8443 running (11m) 119s ago 11m 487M - 18.2.0 dc2bc1663786 7ccc358e4d84 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (8m) 2m ago 8m 437M - 18.2.0 dc2bc1663786 30e493581e1d 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (11m) 119s ago 11m 43.3M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (8m) 2m ago 8m 34.1M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 119s ago 9m 7982k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (8m) 2m ago 8m 7448k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 119s ago 7m 47.3M 3947M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (6m) 119s ago 6m 47.2M 3947M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (6m) 119s ago 6m 45.6M 3947M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (5m) 2m ago 5m 45.6M 1213M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:22:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (4m) 2m ago 4m 43.5M 1213M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:22:45.950 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (4m) 2m ago 4m 43.1M 1213M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:22:45.950 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 119s ago 9m 54.3M - 2.43.0 a07b618ecd1d 0199ca15cf9f 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.949+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 msgr2=0x7fed3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 0x7fed3c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed44006010 tx=0x7fed4400b540 comp rx=0 tx=0).stop 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 msgr2=0x7fed581958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b4b0 tx=0x7fed4800b870 comp rx=0 tx=0).stop 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 shutdown_connections 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fed3c06c680 0x7fed3c06eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed58100ab0 0x7fed581958a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 --2- 172.21.15.90:0/77002168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed581033d0 0x7fed58195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.950+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 >> 172.21.15.90:0/77002168 conn(0x7fed580fa4d0 msgr2=0x7fed580fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.951+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 shutdown_connections 2024-04-24T08:22:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:45.951+0000 7fed5ce00700 1 -- 172.21.15.90:0/77002168 wait complete. 2024-04-24T08:22:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 -- 172.21.15.90:0/284112942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 msgr2=0x7f4380102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 --2- 172.21.15.90:0/284112942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380102f70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4374009910 tx=0x7f4374009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 -- 172.21.15.90:0/284112942 shutdown_connections 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 --2- 172.21.15.90:0/284112942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 0x7f4380105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 --2- 172.21.15.90:0/284112942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380102f70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 -- 172.21.15.90:0/284112942 >> 172.21.15.90:0/284112942 conn(0x7f43800fa5d0 msgr2=0x7f43800fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 -- 172.21.15.90:0/284112942 shutdown_connections 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.121+0000 7f4385e00700 1 -- 172.21.15.90:0/284112942 wait complete. 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 Processor -- start 2024-04-24T08:22:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 -- start start 2024-04-24T08:22:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 0x7f4380197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801985a0 con 0x7f4380100b90 2024-04-24T08:22:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.123+0000 7f4385e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801986e0 con 0x7f43801034b0 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34054/0 (socket says 172.21.15.90:34054) 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 -- 172.21.15.90:0/1199472008 learned_addr learned my addr 172.21.15.90:0/1199472008 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f437fe00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 0x7f4380197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 -- 172.21.15.90:0/1199472008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 msgr2=0x7f4380197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 0x7f4380197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 -- 172.21.15.90:0/1199472008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740095f0 con 0x7f4380100b90 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.124+0000 7f4384a00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f437400bda0 tx=0x7f437400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.125+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437401d070 con 0x7f4380100b90 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.125+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f437400f460 con 0x7f4380100b90 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.125+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4374021640 con 0x7f4380100b90 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.125+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438019d130 con 0x7f4380100b90 2024-04-24T08:22:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.125+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438019d620 con 0x7f4380100b90 2024-04-24T08:22:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.127+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43800fc140 con 0x7f4380100b90 2024-04-24T08:22:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.128+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f437400f5d0 con 0x7f4380100b90 2024-04-24T08:22:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.129+0000 7f437d600700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 0x7f436806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.129+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f437408cc40 con 0x7f4380100b90 2024-04-24T08:22:46.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.130+0000 7f437fe00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 0x7f436806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.131+0000 7f437fe00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 0x7f436806eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f436c005950 tx=0x7f436c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:46.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.135+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f437405ad50 con 0x7f4380100b90 2024-04-24T08:22:46.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:46 smithi090 ceph-mon[25016]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:46.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:46 smithi090 ceph-mon[25016]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:46.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.482+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f438002cc30 con 0x7f4380100b90 2024-04-24T08:22:46.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.483+0000 7f437d600700 1 -- 172.21.15.90:0/1199472008 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f437405a8e0 con 0x7f4380100b90 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:22:46.485 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:22:46.486 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-24T08:22:46.486 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:22:46.486 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:22:46.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 msgr2=0x7f436806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 0x7f436806eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f436c005950 tx=0x7f436c0058e0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 msgr2=0x7f4380197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f437400bda0 tx=0x7f437400be80 comp rx=0 tx=0).stop 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 shutdown_connections 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f436806c680 0x7f436806eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4380100b90 0x7f4380197a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 --2- 172.21.15.90:0/1199472008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43801034b0 0x7f4380197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.487+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 >> 172.21.15.90:0/1199472008 conn(0x7f43800fa5d0 msgr2=0x7f43800ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.488+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 shutdown_connections 2024-04-24T08:22:46.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.488+0000 7f4385e00700 1 -- 172.21.15.90:0/1199472008 wait complete. 2024-04-24T08:22:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:46 smithi149 ceph-mon[31282]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:46 smithi149 ceph-mon[31282]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:46.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 -- 172.21.15.90:0/1868188619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890100b20 msgr2=0x7f2890102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 --2- 172.21.15.90:0/1868188619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890100b20 0x7f2890102f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2880009910 tx=0x7f2880009c20 comp rx=0 tx=0).stop 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 -- 172.21.15.90:0/1868188619 shutdown_connections 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 --2- 172.21.15.90:0/1868188619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890103440 0x7f2890105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 --2- 172.21.15.90:0/1868188619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890100b20 0x7f2890102f00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.662+0000 7f288f200700 1 -- 172.21.15.90:0/1868188619 >> 172.21.15.90:0/1868188619 conn(0x7f28900fa500 msgr2=0x7f28900fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.663+0000 7f288f200700 1 -- 172.21.15.90:0/1868188619 shutdown_connections 2024-04-24T08:22:46.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.663+0000 7f288f200700 1 -- 172.21.15.90:0/1868188619 wait complete. 2024-04-24T08:22:46.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.664+0000 7f288f200700 1 Processor -- start 2024-04-24T08:22:46.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.664+0000 7f288f200700 1 -- start start 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890100b20 0x7f2890197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890198560 con 0x7f2890103440 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28901986a0 con 0x7f2890100b20 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34070/0 (socket says 172.21.15.90:34070) 2024-04-24T08:22:46.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288d400700 1 -- 172.21.15.90:0/3870011780 learned_addr learned my addr 172.21.15.90:0/3870011780 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:22:46.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.665+0000 7f288de00700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890100b20 0x7f2890197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f288d400700 1 -- 172.21.15.90:0/3870011780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890100b20 msgr2=0x7f2890197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f288d400700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890100b20 0x7f2890197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f288d400700 1 -- 172.21.15.90:0/3870011780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28800095f0 con 0x7f2890103440 2024-04-24T08:22:46.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f288d400700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d8d0 tx=0x7f287c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:46.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287c009940 con 0x7f2890103440 2024-04-24T08:22:46.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f287c010460 con 0x7f2890103440 2024-04-24T08:22:46.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.666+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287c00f5f0 con 0x7f2890103440 2024-04-24T08:22:46.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.667+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289019d150 con 0x7f2890103440 2024-04-24T08:22:46.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.667+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289019d6a0 con 0x7f2890103440 2024-04-24T08:22:46.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.668+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28900fc0c0 con 0x7f2890103440 2024-04-24T08:22:46.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.670+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f287c0105d0 con 0x7f2890103440 2024-04-24T08:22:46.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.670+0000 7f2886a00700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 0x7f287406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:22:46.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.671+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f287c08b8b0 con 0x7f2890103440 2024-04-24T08:22:46.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.671+0000 7f288de00700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 0x7f287406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:22:46.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.672+0000 7f288de00700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 0x7f287406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2880006010 tx=0x7f288000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:22:46.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.676+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f287c0599a0 con 0x7f2890103440 2024-04-24T08:22:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.950+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2890061190 con 0x7f287406c680 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.953+0000 7f2886a00700 1 -- 172.21.15.90:0/3870011780 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f2890061190 con 0x7f287406c680 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 image on host smithi149", 2024-04-24T08:22:46.954 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:22:46.955 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:22:46.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 msgr2=0x7f287406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 0x7f287406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2880006010 tx=0x7f288000b540 comp rx=0 tx=0).stop 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 msgr2=0x7f2890197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d8d0 tx=0x7f287c00dc90 comp rx=0 tx=0).stop 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 shutdown_connections 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f287406c680 0x7f287406eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2890100b20 0x7f2890197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 --2- 172.21.15.90:0/3870011780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2890103440 0x7f2890197f40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.956+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 >> 172.21.15.90:0/3870011780 conn(0x7f28900fa500 msgr2=0x7f28900ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.957+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 shutdown_connections 2024-04-24T08:22:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:22:46.957+0000 7f288f200700 1 -- 172.21.15.90:0/3870011780 wait complete. 2024-04-24T08:22:47.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:47 smithi090 ceph-mon[25016]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:47.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:47 smithi090 ceph-mon[25016]: pgmap v303: 65 pgs: 65 active+clean; 7.4 MiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 584 KiB/s wr, 7 op/s 2024-04-24T08:22:47.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:47 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1199472008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:22:47.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:47 smithi149 ceph-mon[31282]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:47 smithi149 ceph-mon[31282]: pgmap v303: 65 pgs: 65 active+clean; 7.4 MiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 584 KiB/s wr, 7 op/s 2024-04-24T08:22:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:47 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1199472008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:22:48.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:48 smithi090 ceph-mon[25016]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:48 smithi149 ceph-mon[31282]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:22:49.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:49 smithi149 ceph-mon[31282]: pgmap v304: 65 pgs: 65 active+clean; 12 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 951 KiB/s wr, 10 op/s 2024-04-24T08:22:49.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:49 smithi090 ceph-mon[25016]: pgmap v304: 65 pgs: 65 active+clean; 12 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 951 KiB/s wr, 10 op/s 2024-04-24T08:22:50.409 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T08:22:50.442 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:50 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:50 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:22:50.638 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T08:22:50.734 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T08:22:50.739 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T08:22:50.739 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T08:22:50.826 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T08:22:50.837 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T08:22:50.837 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-24T08:22:50.881 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-24T08:22:50.882 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T08:22:50.882 DEBUG:teuthology.orchestra.run.smithi149:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-24T08:22:51.002 INFO:tasks.workunit.client.1.smithi149.stderr:+ mkdir -p fsstress 2024-04-24T08:22:51.008 INFO:tasks.workunit.client.1.smithi149.stderr:+ pushd fsstress 2024-04-24T08:22:51.011 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T08:22:51.011 INFO:tasks.workunit.client.1.smithi149.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T08:22:51.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:51 smithi090 ceph-mon[25016]: pgmap v305: 65 pgs: 65 active+clean; 12 MiB data, 197 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 983 KiB/s wr, 14 op/s 2024-04-24T08:22:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:51 smithi149 ceph-mon[31282]: pgmap v305: 65 pgs: 65 active+clean; 12 MiB data, 197 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 983 KiB/s wr, 14 op/s 2024-04-24T08:22:52.292 INFO:tasks.workunit.client.1.smithi149.stderr:+ tar xzf ltp-full.tgz 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: Upgrade: Updating mgr.smithi149.vavftl 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:22:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:52 smithi149 ceph-mon[31282]: pgmap v306: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 28 op/s 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: Upgrade: Updating mgr.smithi149.vavftl 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:22:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:52 smithi090 ceph-mon[25016]: pgmap v306: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 28 op/s 2024-04-24T08:22:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:55 smithi149 ceph-mon[31282]: pgmap v307: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-04-24T08:22:55.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:55 smithi090 ceph-mon[25016]: pgmap v307: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-04-24T08:22:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:57 smithi149 ceph-mon[31282]: pgmap v308: 65 pgs: 65 active+clean; 25 MiB data, 235 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.1 MiB/s wr, 47 op/s 2024-04-24T08:22:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:57 smithi090 ceph-mon[25016]: pgmap v308: 65 pgs: 65 active+clean; 25 MiB data, 235 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.1 MiB/s wr, 47 op/s 2024-04-24T08:22:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:22:59 smithi149 ceph-mon[31282]: pgmap v309: 65 pgs: 65 active+clean; 30 MiB data, 250 MiB used, 536 GiB / 536 GiB avail; 340 KiB/s rd, 1.9 MiB/s wr, 52 op/s 2024-04-24T08:22:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:22:59 smithi090 ceph-mon[25016]: pgmap v309: 65 pgs: 65 active+clean; 30 MiB data, 250 MiB used, 536 GiB / 536 GiB avail; 340 KiB/s rd, 1.9 MiB/s wr, 52 op/s 2024-04-24T08:23:01.455 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:01 smithi149 ceph-mon[31282]: pgmap v310: 65 pgs: 65 active+clean; 30 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 52 op/s 2024-04-24T08:23:01.490 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:01 smithi090 ceph-mon[25016]: pgmap v310: 65 pgs: 65 active+clean; 30 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 52 op/s 2024-04-24T08:23:02.976 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:02 smithi149 ceph-mon[31282]: pgmap v311: 65 pgs: 65 active+clean; 34 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-04-24T08:23:03.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:02 smithi090 ceph-mon[25016]: pgmap v311: 65 pgs: 65 active+clean; 34 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-04-24T08:23:04.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:04 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:04.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:04 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:04.481 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:04 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:04.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:04 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:04.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:04 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:04.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:04 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:05.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:05 smithi149 ceph-mon[31282]: pgmap v312: 65 pgs: 65 active+clean; 34 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 642 KiB/s rd, 1.7 MiB/s wr, 54 op/s 2024-04-24T08:23:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:05 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:05 smithi090 ceph-mon[25016]: pgmap v312: 65 pgs: 65 active+clean; 34 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 642 KiB/s rd, 1.7 MiB/s wr, 54 op/s 2024-04-24T08:23:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:05 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:07.224 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:07 smithi149 ceph-mon[31282]: pgmap v313: 65 pgs: 65 active+clean; 38 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-04-24T08:23:07.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:07 smithi090 ceph-mon[25016]: pgmap v313: 65 pgs: 65 active+clean; 38 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-04-24T08:23:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:09 smithi149 ceph-mon[31282]: pgmap v314: 65 pgs: 65 active+clean; 39 MiB data, 284 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-04-24T08:23:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:09 smithi090 ceph-mon[25016]: pgmap v314: 65 pgs: 65 active+clean; 39 MiB data, 284 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-04-24T08:23:10.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:10 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:10.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:10 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:11.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:10 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:11.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:10 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:11.914 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:11 smithi149 ceph-mon[31282]: pgmap v315: 65 pgs: 65 active+clean; 43 MiB data, 293 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-24T08:23:11.914 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:11 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:11.914 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:11 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:12.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:11 smithi090 ceph-mon[25016]: pgmap v315: 65 pgs: 65 active+clean; 43 MiB data, 293 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-24T08:23:12.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:11 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:12.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:11 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:13.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:12 smithi149 ceph-mon[31282]: pgmap v316: 65 pgs: 65 active+clean; 45 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-04-24T08:23:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:12 smithi090 ceph-mon[25016]: pgmap v316: 65 pgs: 65 active+clean; 45 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-04-24T08:23:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:15 smithi149 ceph-mon[31282]: pgmap v317: 65 pgs: 65 active+clean; 45 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 705 KiB/s rd, 1.0 MiB/s wr, 71 op/s 2024-04-24T08:23:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:15 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:23:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:15 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:23:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:15 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:23:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:15 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:23:15.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:15 smithi090 ceph-mon[25016]: pgmap v317: 65 pgs: 65 active+clean; 45 MiB data, 327 MiB used, 536 GiB / 536 GiB avail; 705 KiB/s rd, 1.0 MiB/s wr, 71 op/s 2024-04-24T08:23:15.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:15 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:23:15.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:15 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:23:15.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:15 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:23:15.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:15 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:23:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:16 smithi149 ceph-mon[31282]: mgrmap e20: smithi090.jsrfxo(active, since 9m), standbys: smithi149.vavftl 2024-04-24T08:23:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:23:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:23:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:16 smithi090 ceph-mon[25016]: mgrmap e20: smithi090.jsrfxo(active, since 9m), standbys: smithi149.vavftl 2024-04-24T08:23:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:16 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:23:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:16 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 -- 172.21.15.90:0/3113540405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 msgr2=0x7f4544106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 --2- 172.21.15.90:0/3113540405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f4544106c50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4534009910 tx=0x7f4534009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 -- 172.21.15.90:0/3113540405 shutdown_connections 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 --2- 172.21.15.90:0/3113540405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f4544107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 --2- 172.21.15.90:0/3113540405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f4544106c50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 -- 172.21.15.90:0/3113540405 >> 172.21.15.90:0/3113540405 conn(0x7f4544075970 msgr2=0x7f4544077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 -- 172.21.15.90:0/3113540405 shutdown_connections 2024-04-24T08:23:17.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.142+0000 7f454ba00700 1 -- 172.21.15.90:0/3113540405 wait complete. 2024-04-24T08:23:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 Processor -- start 2024-04-24T08:23:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 -- start start 2024-04-24T08:23:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f454419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419cc60 con 0x7f4544106840 2024-04-24T08:23:17.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.144+0000 7f454ba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419cda0 con 0x7f4544107a40 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f453be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f454419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f4549000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f453be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f454419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:37734/0 (socket says 172.21.15.90:37734) 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f4549000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45664/0 (socket says 172.21.15.90:45664) 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f453be00700 1 -- 172.21.15.90:0/1187986689 learned_addr learned my addr 172.21.15.90:0/1187986689 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f4549000700 1 -- 172.21.15.90:0/1187986689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 msgr2=0x7f454419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f4549000700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f454419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.145+0000 7f4549000700 1 -- 172.21.15.90:0/1187986689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340095f0 con 0x7f4544106840 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f4549000700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4534000c00 tx=0x7f453400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:17.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f453401d070 con 0x7f4544106840 2024-04-24T08:23:17.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4534005dc0 con 0x7f4544106840 2024-04-24T08:23:17.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45441a17f0 con 0x7f4544106840 2024-04-24T08:23:17.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4534021410 con 0x7f4544106840 2024-04-24T08:23:17.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.146+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45441a1ce0 con 0x7f4544106840 2024-04-24T08:23:17.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.148+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454410f390 con 0x7f4544106840 2024-04-24T08:23:17.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.148+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 91256+0+0 (secure 0 0 0) 0x7f453400f7d0 con 0x7f4544106840 2024-04-24T08:23:17.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.149+0000 7f4542000700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 0x7f452806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.149+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f453408e250 con 0x7f4544106840 2024-04-24T08:23:17.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.150+0000 7f453be00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 0x7f452806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.150+0000 7f453be00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 0x7f452806fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4530007c90 tx=0x7f4530007480 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:17.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.155+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f453405c2c0 con 0x7f4544106840 2024-04-24T08:23:17.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:17 smithi090 ceph-mon[25016]: pgmap v318: 65 pgs: 65 active+clean; 52 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-04-24T08:23:17.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.402+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f454410c390 con 0x7f452806d7e0 2024-04-24T08:23:17.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.404+0000 7f4542000700 1 -- 172.21.15.90:0/1187986689 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f454410c390 con 0x7f452806d7e0 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 msgr2=0x7f452806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 0x7f452806fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4530007c90 tx=0x7f4530007480 comp rx=0 tx=0).stop 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 msgr2=0x7f454419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4534000c00 tx=0x7f453400bd70 comp rx=0 tx=0).stop 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 shutdown_connections 2024-04-24T08:23:17.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f452806d7e0 0x7f452806fc90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4544106840 0x7f454419c100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 --2- 172.21.15.90:0/1187986689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4544107a40 0x7f454419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.407+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 >> 172.21.15.90:0/1187986689 conn(0x7f4544075970 msgr2=0x7f454410ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:17.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.408+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 shutdown_connections 2024-04-24T08:23:17.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.408+0000 7f454ba00700 1 -- 172.21.15.90:0/1187986689 wait complete. 2024-04-24T08:23:17.424 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:23:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:17 smithi149 ceph-mon[31282]: pgmap v318: 65 pgs: 65 active+clean; 52 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-04-24T08:23:17.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.573+0000 7efd1b000700 1 -- 172.21.15.90:0/91622062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14103350 msgr2=0x7efd14105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.573+0000 7efd1b000700 1 --2- 172.21.15.90:0/91622062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14103350 0x7efd14105790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009910 tx=0x7efcfc009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 -- 172.21.15.90:0/91622062 shutdown_connections 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 --2- 172.21.15.90:0/91622062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14103350 0x7efd14105790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 --2- 172.21.15.90:0/91622062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069230 0x7efd14102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 -- 172.21.15.90:0/91622062 >> 172.21.15.90:0/91622062 conn(0x7efd140fa780 msgr2=0x7efd140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 -- 172.21.15.90:0/91622062 shutdown_connections 2024-04-24T08:23:17.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.574+0000 7efd1b000700 1 -- 172.21.15.90:0/91622062 wait complete. 2024-04-24T08:23:17.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.575+0000 7efd1b000700 1 Processor -- start 2024-04-24T08:23:17.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.575+0000 7efd1b000700 1 -- start start 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd1b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd1b000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14103350 0x7efd14198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd1b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd14198840 con 0x7efd14069230 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd1b000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1419d250 con 0x7efd14103350 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd13e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd0be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14103350 0x7efd14198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd13e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45676/0 (socket says 172.21.15.90:45676) 2024-04-24T08:23:17.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.576+0000 7efd13e00700 1 -- 172.21.15.90:0/3285809149 learned_addr learned my addr 172.21.15.90:0/3285809149 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:17.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd13e00700 1 -- 172.21.15.90:0/3285809149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14103350 msgr2=0x7efd14198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd13e00700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14103350 0x7efd14198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd13e00700 1 -- 172.21.15.90:0/3285809149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0095f0 con 0x7efd14069230 2024-04-24T08:23:17.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd13e00700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd0400e8c0 tx=0x7efd0400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:17.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd040041f0 con 0x7efd14069230 2024-04-24T08:23:17.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd0400cc90 con 0x7efd14069230 2024-04-24T08:23:17.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1419d450 con 0x7efd14069230 2024-04-24T08:23:17.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd04003d60 con 0x7efd14069230 2024-04-24T08:23:17.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.577+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1419d970 con 0x7efd14069230 2024-04-24T08:23:17.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.579+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd14108d10 con 0x7efd14069230 2024-04-24T08:23:17.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.580+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 91256+0+0 (secure 0 0 0) 0x7efd0400c7e0 con 0x7efd14069230 2024-04-24T08:23:17.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.581+0000 7efd11600700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 0x7efcf806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:17.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.581+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efd04014070 con 0x7efd14069230 2024-04-24T08:23:17.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.581+0000 7efd0be00700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 0x7efcf806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:17.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.582+0000 7efd0be00700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 0x7efcf806fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009fd0 tx=0x7efcfc005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:17.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.586+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd040579a0 con 0x7efd14069230 2024-04-24T08:23:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.857+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd140ff300 con 0x7efcf806d7e0 2024-04-24T08:23:17.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.858+0000 7efd11600700 1 -- 172.21.15.90:0/3285809149 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7efd140ff300 con 0x7efcf806d7e0 2024-04-24T08:23:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 msgr2=0x7efcf806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 0x7efcf806fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009fd0 tx=0x7efcfc005fd0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 msgr2=0x7efd14197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd0400e8c0 tx=0x7efd0400ebd0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 shutdown_connections 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7efcf806d7e0 0x7efcf806fc90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14069230 0x7efd14197ce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 --2- 172.21.15.90:0/3285809149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14103350 0x7efd14198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 >> 172.21.15.90:0/3285809149 conn(0x7efd140fa780 msgr2=0x7efd140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 shutdown_connections 2024-04-24T08:23:17.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:17.862+0000 7efd1b000700 1 -- 172.21.15.90:0/3285809149 wait complete. 2024-04-24T08:23:18.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.029+0000 7f6ae1600700 1 -- 172.21.15.90:0/3023911751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 msgr2=0x7f6adc103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.029+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3023911751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc103b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009910 tx=0x7f6ac4009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.029+0000 7f6ae1600700 1 -- 172.21.15.90:0/3023911751 shutdown_connections 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.029+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3023911751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc103b40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.030+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3023911751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 0x7f6adc102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.030+0000 7f6ae1600700 1 -- 172.21.15.90:0/3023911751 >> 172.21.15.90:0/3023911751 conn(0x7f6adc0fda80 msgr2=0x7f6adc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.030+0000 7f6ae1600700 1 -- 172.21.15.90:0/3023911751 shutdown_connections 2024-04-24T08:23:18.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.030+0000 7f6ae1600700 1 -- 172.21.15.90:0/3023911751 wait complete. 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 Processor -- start 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 -- start start 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 0x7f6adc197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc198880 con 0x7f6adc1036d0 2024-04-24T08:23:18.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.031+0000 7f6ae1600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc19d290 con 0x7f6adc102430 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45688/0 (socket says 172.21.15.90:45688) 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 -- 172.21.15.90:0/3083734456 learned_addr learned my addr 172.21.15.90:0/3083734456 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6adaa00700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 0x7f6adc197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 -- 172.21.15.90:0/3083734456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 msgr2=0x7f6adc197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 0x7f6adc197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.032+0000 7f6ad3e00700 1 -- 172.21.15.90:0/3083734456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40095f0 con 0x7f6adc1036d0 2024-04-24T08:23:18.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.033+0000 7f6ad3e00700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400bd70 tx=0x7f6ac400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:18.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.033+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac401d070 con 0x7f6adc1036d0 2024-04-24T08:23:18.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.033+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6ac4022470 con 0x7f6adc1036d0 2024-04-24T08:23:18.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.033+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac400f670 con 0x7f6adc1036d0 2024-04-24T08:23:18.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.033+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc19d430 con 0x7f6adc1036d0 2024-04-24T08:23:18.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.034+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc19d8f0 con 0x7f6adc1036d0 2024-04-24T08:23:18.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.035+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc10af90 con 0x7f6adc1036d0 2024-04-24T08:23:18.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.035+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 91256+0+0 (secure 0 0 0) 0x7f6ac4022a50 con 0x7f6adc1036d0 2024-04-24T08:23:18.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.036+0000 7f6ad3400700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 0x7f6ac006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.036+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6ac408e640 con 0x7f6adc1036d0 2024-04-24T08:23:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.037+0000 7f6adaa00700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 0x7f6ac006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.037+0000 7f6adaa00700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 0x7f6ac006fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006fd0 tx=0x7f6acc008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:18.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.041+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6ac4058d50 con 0x7f6adc1036d0 2024-04-24T08:23:18.274 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:18 smithi149 ceph-mon[31282]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:18.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.281+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6adc19df50 con 0x7f6ac006d7e0 2024-04-24T08:23:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.314+0000 7f6ad3400700 1 -- 172.21.15.90:0/3083734456 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6adc19df50 con 0x7f6ac006d7e0 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 2m ago 10m 13.3M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (10m) 2m ago 10m 7499k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (9m) 8s ago 9m 7662k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (10m) 2m ago 10m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (9m) 8s ago 9m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (8m) 2m ago 9m 74.4M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:23:18.316 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (2m) 2m ago 2m 12.7M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (3m) 2m ago 3m 11.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (3m) 8s ago 3m 67.5M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (2m) 8s ago 2m 12.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:9283,8765,8443 running (12m) 2m ago 12m 487M - 18.2.0 dc2bc1663786 7ccc358e4d84 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (15s) 8s ago 8m 49.6M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 839fd9b0cdf9 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (12m) 2m ago 12m 43.3M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (8m) 8s ago 8m 34.9M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (10m) 2m ago 10m 7982k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (9m) 8s ago 9m 8186k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 2m ago 7m 47.3M 3947M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 2m ago 7m 47.2M 3947M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (6m) 2m ago 6m 45.6M 3947M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (6m) 8s ago 6m 62.7M 1213M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (5m) 8s ago 5m 57.0M 1213M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (4m) 8s ago 4m 57.2M 1213M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:23:18.317 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 2m ago 9m 54.3M - 2.43.0 a07b618ecd1d 0199ca15cf9f 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 msgr2=0x7f6ac006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 0x7f6ac006fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006fd0 tx=0x7f6acc008040 comp rx=0 tx=0).stop 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 msgr2=0x7f6adc198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400bd70 tx=0x7f6ac400be50 comp rx=0 tx=0).stop 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 shutdown_connections 2024-04-24T08:23:18.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f6ac006d7e0 0x7f6ac006fc90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6adc102430 0x7f6adc197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 --2- 172.21.15.90:0/3083734456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6adc1036d0 0x7f6adc198260 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.318+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 >> 172.21.15.90:0/3083734456 conn(0x7f6adc0fda80 msgr2=0x7f6adc106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.319+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 shutdown_connections 2024-04-24T08:23:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.319+0000 7f6ae1600700 1 -- 172.21.15.90:0/3083734456 wait complete. 2024-04-24T08:23:18.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:18 smithi090 ceph-mon[25016]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:18.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 -- 172.21.15.90:0/3239258842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 msgr2=0x7fd5901038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 --2- 172.21.15.90:0/3239258842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd5901038b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009910 tx=0x7fd57c009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:18.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 -- 172.21.15.90:0/3239258842 shutdown_connections 2024-04-24T08:23:18.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 --2- 172.21.15.90:0/3239258842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd5901038b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 --2- 172.21.15.90:0/3239258842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590102260 0x7fd590102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 -- 172.21.15.90:0/3239258842 >> 172.21.15.90:0/3239258842 conn(0x7fd5900fd8d0 msgr2=0x7fd5900ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:18.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 -- 172.21.15.90:0/3239258842 shutdown_connections 2024-04-24T08:23:18.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.494+0000 7fd58f200700 1 -- 172.21.15.90:0/3239258842 wait complete. 2024-04-24T08:23:18.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 Processor -- start 2024-04-24T08:23:18.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 -- start start 2024-04-24T08:23:18.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590102260 0x7fd590078b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900755f0 con 0x7fd590103460 2024-04-24T08:23:18.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.496+0000 7fd58f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590075760 con 0x7fd590102260 2024-04-24T08:23:18.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:18.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45702/0 (socket says 172.21.15.90:45702) 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 -- 172.21.15.90:0/3684446863 learned_addr learned my addr 172.21.15.90:0/3684446863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 -- 172.21.15.90:0/3684446863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590102260 msgr2=0x7fd590078b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590102260 0x7fd590078b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.497+0000 7fd58d400700 1 -- 172.21.15.90:0/3684446863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c0095f0 con 0x7fd590103460 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd58d400700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005b40 tx=0x7fd57c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c01d070 con 0x7fd590103460 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd57c00f460 con 0x7fd590103460 2024-04-24T08:23:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c021640 con 0x7fd590103460 2024-04-24T08:23:18.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5900759e0 con 0x7fd590103460 2024-04-24T08:23:18.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.498+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd590075ed0 con 0x7fd590103460 2024-04-24T08:23:18.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.500+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5900761b0 con 0x7fd590103460 2024-04-24T08:23:18.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.502+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 91256+0+0 (secure 0 0 0) 0x7fd57c00fa90 con 0x7fd590103460 2024-04-24T08:23:18.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.503+0000 7fd586a00700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 0x7fd57406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:18.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.503+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd57c08e520 con 0x7fd590103460 2024-04-24T08:23:18.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.503+0000 7fd58de00700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 0x7fd57406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:18.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.504+0000 7fd58de00700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 0x7fd57406fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd5901032c0 tx=0x7fd580009470 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:18.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.510+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd57c05c690 con 0x7fd590103460 2024-04-24T08:23:18.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.845+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5901a1ed0 con 0x7fd590103460 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.846+0000 7fd586a00700 1 -- 172.21.15.90:0/3684446863 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fd57c05c220 con 0x7fd590103460 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:18.847 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:23:18.848 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:23:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 msgr2=0x7fd57406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 0x7fd57406fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd5901032c0 tx=0x7fd580009470 comp rx=0 tx=0).stop 2024-04-24T08:23:18.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 msgr2=0x7fd590079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:18.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005b40 tx=0x7fd57c00be20 comp rx=0 tx=0).stop 2024-04-24T08:23:18.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 shutdown_connections 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7fd57406d7e0 0x7fd57406fc90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590102260 0x7fd590078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 --2- 172.21.15.90:0/3684446863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd590103460 0x7fd590079040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.850+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 >> 172.21.15.90:0/3684446863 conn(0x7fd5900fd8d0 msgr2=0x7fd590106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.851+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 shutdown_connections 2024-04-24T08:23:18.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:18.851+0000 7fd58f200700 1 -- 172.21.15.90:0/3684446863 wait complete. 2024-04-24T08:23:19.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 -- 172.21.15.90:0/3242212722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec100b00 msgr2=0x7f27ec102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 --2- 172.21.15.90:0/3242212722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec100b00 0x7f27ec102ee0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009910 tx=0x7f27e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 -- 172.21.15.90:0/3242212722 shutdown_connections 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 --2- 172.21.15.90:0/3242212722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec103420 0x7f27ec105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 --2- 172.21.15.90:0/3242212722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec100b00 0x7f27ec102ee0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.017+0000 7f27f2c00700 1 -- 172.21.15.90:0/3242212722 >> 172.21.15.90:0/3242212722 conn(0x7f27ec0fa520 msgr2=0x7f27ec0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.018+0000 7f27f2c00700 1 -- 172.21.15.90:0/3242212722 shutdown_connections 2024-04-24T08:23:19.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.018+0000 7f27f2c00700 1 -- 172.21.15.90:0/3242212722 wait complete. 2024-04-24T08:23:19.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 Processor -- start 2024-04-24T08:23:19.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 -- start start 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec100b00 0x7f27ec197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec198560 con 0x7f27ec103420 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.019+0000 7f27f2c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1986a0 con 0x7f27ec100b00 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45728/0 (socket says 172.21.15.90:45728) 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 -- 172.21.15.90:0/1308510231 learned_addr learned my addr 172.21.15.90:0/1308510231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:19.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f1800700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec100b00 0x7f27ec197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:19.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 -- 172.21.15.90:0/1308510231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec100b00 msgr2=0x7f27ec197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:19.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec100b00 0x7f27ec197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.020+0000 7f27f0e00700 1 -- 172.21.15.90:0/1308510231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00095f0 con 0x7f27ec103420 2024-04-24T08:23:19.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27f0e00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c8a0 tx=0x7f27d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:19.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d800cea0 con 0x7f27ec103420 2024-04-24T08:23:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f27d8004830 con 0x7f27ec103420 2024-04-24T08:23:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d80055b0 con 0x7f27ec103420 2024-04-24T08:23:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec19d150 con 0x7f27ec103420 2024-04-24T08:23:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.021+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec19d6a0 con 0x7f27ec103420 2024-04-24T08:23:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.022+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec0fc0e0 con 0x7f27ec103420 2024-04-24T08:23:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.023+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 91256+0+0 (secure 0 0 0) 0x7f27d8007ad0 con 0x7f27ec103420 2024-04-24T08:23:19.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.024+0000 7f27ea000700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 0x7f27d406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:19.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.025+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f27d808ccd0 con 0x7f27ec103420 2024-04-24T08:23:19.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.025+0000 7f27f1800700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 0x7f27d406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:19.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.025+0000 7f27f1800700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 0x7f27d406fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27e0006010 tx=0x7f27e000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:19.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.030+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f27d805adc0 con 0x7f27ec103420 2024-04-24T08:23:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.296+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 --> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27ec061190 con 0x7f27d406d7e0 2024-04-24T08:23:19.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.298+0000 7f27ea000700 1 -- 172.21.15.90:0/1308510231 <== mgr.14215 v2:172.21.15.90:6800/1367639192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f27ec061190 con 0x7f27d406d7e0 2024-04-24T08:23:19.299 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:23:19.300 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.301+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 msgr2=0x7f27d406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.301+0000 7f27f2c00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 0x7f27d406fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27e0006010 tx=0x7f27e000b540 comp rx=0 tx=0).stop 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.301+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 msgr2=0x7f27ec197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c8a0 tx=0x7f27d800cc60 comp rx=0 tx=0).stop 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 shutdown_connections 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:6800/1367639192,v1:172.21.15.90:6801/1367639192] conn(0x7f27d406d7e0 0x7f27d406fc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27ec100b00 0x7f27ec197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 --2- 172.21.15.90:0/1308510231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27ec103420 0x7f27ec197f40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:19.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 >> 172.21.15.90:0/1308510231 conn(0x7f27ec0fa520 msgr2=0x7f27ec0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 shutdown_connections 2024-04-24T08:23:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:19.302+0000 7f27f2c00700 1 -- 172.21.15.90:0/1308510231 wait complete. 2024-04-24T08:23:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:19 smithi149 ceph-mon[31282]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:19 smithi149 ceph-mon[31282]: pgmap v319: 65 pgs: 65 active+clean; 57 MiB data, 362 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-04-24T08:23:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:19 smithi149 ceph-mon[31282]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:19 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3684446863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:23:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:19 smithi090 ceph-mon[25016]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:19 smithi090 ceph-mon[25016]: pgmap v319: 65 pgs: 65 active+clean; 57 MiB data, 362 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-04-24T08:23:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:19 smithi090 ceph-mon[25016]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:19 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3684446863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:23:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:20 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:23:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:20 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: pgmap v320: 65 pgs: 65 active+clean; 57 MiB data, 367 MiB used, 536 GiB / 536 GiB avail; 1002 KiB/s rd, 1.5 MiB/s wr, 74 op/s 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: Upgrade: Need to upgrade myself (mgr.smithi090.jsrfxo) 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr fail", "who": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "mgr fail", "who": "smithi090.jsrfxo"}]': finished 2024-04-24T08:23:21.530 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: mgrmap e21: smithi149.vavftl(active, starting, since 0.017771s) 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:23:21.531 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: Manager daemon smithi149.vavftl is now available 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/trash_purge_schedule"}]: dispatch 2024-04-24T08:23:21.532 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:21 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/trash_purge_schedule"}]: dispatch 2024-04-24T08:23:21.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: pgmap v320: 65 pgs: 65 active+clean; 57 MiB data, 367 MiB used, 536 GiB / 536 GiB avail; 1002 KiB/s rd, 1.5 MiB/s wr, 74 op/s 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: Upgrade: Need to upgrade myself (mgr.smithi090.jsrfxo) 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr fail", "who": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.14215 172.21.15.90:0/3958252801' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "mgr fail", "who": "smithi090.jsrfxo"}]': finished 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: mgrmap e21: smithi149.vavftl(active, starting, since 0.017771s) 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:23:21.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: Manager daemon smithi149.vavftl is now available 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:23:21.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/trash_purge_schedule"}]: dispatch 2024-04-24T08:23:21.546 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:21 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.vavftl/trash_purge_schedule"}]: dispatch 2024-04-24T08:23:23.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: mgrmap e22: smithi149.vavftl(active, since 1.01741s) 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: Deploying cephadm binary to smithi090 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: Deploying cephadm binary to smithi149 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:23:21] ENGINE Bus STARTING 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:23:21] ENGINE Serving on http://172.21.15.149:8765 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:23:22] ENGINE Serving on https://172.21.15.149:7150 2024-04-24T08:23:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:23:22] ENGINE Bus STARTED 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: mgrmap e22: smithi149.vavftl(active, since 1.01741s) 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: Deploying cephadm binary to smithi090 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: Deploying cephadm binary to smithi149 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:23:21] ENGINE Bus STARTING 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:23:21] ENGINE Serving on http://172.21.15.149:8765 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:23:22] ENGINE Serving on https://172.21.15.149:7150 2024-04-24T08:23:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:23:22] ENGINE Bus STARTED 2024-04-24T08:23:23.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:23 smithi090 ceph-mon[25016]: pgmap v4: 65 pgs: 65 active+clean; 66 MiB data, 385 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:23:24.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:23 smithi149 ceph-mon[31282]: pgmap v4: 65 pgs: 65 active+clean; 66 MiB data, 385 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:23:24.909 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:24 smithi090 ceph-mon[25016]: mgrmap e23: smithi149.vavftl(active, since 3s) 2024-04-24T08:23:25.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:24 smithi149 ceph-mon[31282]: mgrmap e23: smithi149.vavftl(active, since 3s) 2024-04-24T08:23:25.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:25 smithi149 ceph-mon[31282]: pgmap v5: 65 pgs: 65 active+clean; 66 MiB data, 385 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:23:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:25 smithi090 ceph-mon[25016]: pgmap v5: 65 pgs: 65 active+clean; 66 MiB data, 385 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:23:26.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:26 smithi090 ceph-mon[25016]: mgrmap e24: smithi149.vavftl(active, since 5s) 2024-04-24T08:23:27.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:26 smithi149 ceph-mon[31282]: mgrmap e24: smithi149.vavftl(active, since 5s) 2024-04-24T08:23:28.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:27 smithi149 ceph-mon[31282]: pgmap v6: 65 pgs: 65 active+clean; 82 MiB data, 443 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-04-24T08:23:28.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:27 smithi090 ceph-mon[25016]: pgmap v6: 65 pgs: 65 active+clean; 82 MiB data, 443 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-04-24T08:23:29.954 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:29 smithi090 ceph-mon[25016]: pgmap v7: 65 pgs: 65 active+clean; 86 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 788 KiB/s rd, 3.0 MiB/s wr, 153 op/s 2024-04-24T08:23:30.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:29 smithi149 ceph-mon[31282]: pgmap v7: 65 pgs: 65 active+clean; 86 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 788 KiB/s rd, 3.0 MiB/s wr, 153 op/s 2024-04-24T08:23:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: pgmap v8: 65 pgs: 65 active+clean; 86 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 613 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2024-04-24T08:23:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: Standby manager daemon smithi090.jsrfxo started 2024-04-24T08:23:32.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/crt"}]: dispatch 2024-04-24T08:23:32.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:23:32.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/key"}]: dispatch 2024-04-24T08:23:32.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:31 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:23:32.150 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: pgmap v8: 65 pgs: 65 active+clean; 86 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 613 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2024-04-24T08:23:32.150 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: Standby manager daemon smithi090.jsrfxo started 2024-04-24T08:23:32.151 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/crt"}]: dispatch 2024-04-24T08:23:32.151 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:23:32.151 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/key"}]: dispatch 2024-04-24T08:23:32.151 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:31 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/3782323492' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:23:33.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:32 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:23:33.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:32 smithi090 ceph-mon[25016]: mgrmap e25: smithi149.vavftl(active, since 11s), standbys: smithi090.jsrfxo 2024-04-24T08:23:33.170 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:32 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:23:33.170 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:32 smithi149 ceph-mon[31282]: mgrmap e25: smithi149.vavftl(active, since 11s), standbys: smithi090.jsrfxo 2024-04-24T08:23:33.969 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:33 smithi149 ceph-mon[31282]: pgmap v9: 65 pgs: 65 active+clean; 99 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 146 op/s 2024-04-24T08:23:34.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:33 smithi090 ceph-mon[25016]: pgmap v9: 65 pgs: 65 active+clean; 99 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 146 op/s 2024-04-24T08:23:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:35 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:35 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:35.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:35 smithi149 ceph-mon[31282]: pgmap v10: 65 pgs: 65 active+clean; 99 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 134 op/s 2024-04-24T08:23:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:35 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:35 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:35.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:35 smithi090 ceph-mon[25016]: pgmap v10: 65 pgs: 65 active+clean; 99 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 134 op/s 2024-04-24T08:23:37.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:36 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:36 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:36 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:37.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:36 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:36 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:36 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:37 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:37 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:37 smithi090 ceph-mon[25016]: pgmap v11: 65 pgs: 65 active+clean; 102 MiB data, 516 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-04-24T08:23:37.927 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:37 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.927 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:37 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:37.927 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:37 smithi149 ceph-mon[31282]: pgmap v11: 65 pgs: 65 active+clean; 102 MiB data, 516 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-04-24T08:23:38.940 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:38.940 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: Adjusting osd_memory_target on smithi090 to 1216M 2024-04-24T08:23:39.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:38 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:39.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:39.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:39.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: Adjusting osd_memory_target on smithi090 to 1216M 2024-04-24T08:23:39.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:38 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:40.141 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:39 smithi149 ceph-mon[31282]: pgmap v12: 65 pgs: 65 active+clean; 105 MiB data, 528 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-04-24T08:23:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:39 smithi090 ceph-mon[25016]: pgmap v12: 65 pgs: 65 active+clean; 105 MiB data, 528 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-04-24T08:23:42.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:41 smithi149 ceph-mon[31282]: pgmap v13: 65 pgs: 65 active+clean; 105 MiB data, 528 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-04-24T08:23:42.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:41 smithi090 ceph-mon[25016]: pgmap v13: 65 pgs: 65 active+clean; 105 MiB data, 528 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: pgmap v14: 65 pgs: 65 active+clean; 113 MiB data, 558 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 117 op/s 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:44 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:23:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: pgmap v14: 65 pgs: 65 active+clean; 113 MiB data, 558 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 117 op/s 2024-04-24T08:23:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:44 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:23:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:45 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:23:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:45 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:23:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:45 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:23:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:45 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:23:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:45 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:23:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:45 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:23:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:45 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:23:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:45 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:23:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: pgmap v15: 65 pgs: 65 active+clean; 113 MiB data, 558 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:46 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: pgmap v15: 65 pgs: 65 active+clean; 113 MiB data, 558 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:46.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:46 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:23:47.487 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:23:47.487 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.487 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.487 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.487 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:47 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:47 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:48.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:48 smithi090 ceph-mon[25016]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:23:48.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:48 smithi090 ceph-mon[25016]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:23:48.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:48 smithi090 ceph-mon[25016]: pgmap v16: 65 pgs: 65 active+clean; 118 MiB data, 582 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-04-24T08:23:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:48 smithi149 ceph-mon[31282]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:23:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:48 smithi149 ceph-mon[31282]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:23:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:48 smithi149 ceph-mon[31282]: pgmap v16: 65 pgs: 65 active+clean; 118 MiB data, 582 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-04-24T08:23:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 -- 172.21.15.90:0/99425297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80ff4a0 msgr2=0x7f1fe80ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 --2- 172.21.15.90:0/99425297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe80ff910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009910 tx=0x7f1fd4009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 -- 172.21.15.90:0/99425297 shutdown_connections 2024-04-24T08:23:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 --2- 172.21.15.90:0/99425297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe80ff910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 --2- 172.21.15.90:0/99425297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80feb50 0x7f1fe80fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 -- 172.21.15.90:0/99425297 >> 172.21.15.90:0/99425297 conn(0x7f1fe80fa780 msgr2=0x7f1fe80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 -- 172.21.15.90:0/99425297 shutdown_connections 2024-04-24T08:23:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.492+0000 7f1fef000700 1 -- 172.21.15.90:0/99425297 wait complete. 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.493+0000 7f1fef000700 1 Processor -- start 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.493+0000 7f1fef000700 1 -- start start 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fef000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fef000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fef000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81987b0 con 0x7f1fe80feb50 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fef000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8198920 con 0x7f1fe80ff4a0 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fe7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fe7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47476/0 (socket says 172.21.15.90:47476) 2024-04-24T08:23:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fe7e00700 1 -- 172.21.15.90:0/86736740 learned_addr learned my addr 172.21.15.90:0/86736740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.494+0000 7f1fe7400700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe7e00700 1 -- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80ff4a0 msgr2=0x7f1fe8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe7e00700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe7e00700 1 -- 172.21.15.90:0/86736740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40095f0 con 0x7f1fe80feb50 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe7e00700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c940 tx=0x7f1fd800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd80041f0 con 0x7f1fe80feb50 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1fd8007d10 con 0x7f1fe80feb50 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd8003cf0 con 0x7f1fe80feb50 2024-04-24T08:23:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe819d370 con 0x7f1fe80feb50 2024-04-24T08:23:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.495+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe819d870 con 0x7f1fe80feb50 2024-04-24T08:23:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.497+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe810aef0 con 0x7f1fe80feb50 2024-04-24T08:23:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.498+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91360+0+0 (secure 0 0 0) 0x7f1fd8007860 con 0x7f1fe80feb50 2024-04-24T08:23:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.499+0000 7f1fe4c00700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 0x7f1fcc06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.499+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1fd808d0f0 con 0x7f1fe80feb50 2024-04-24T08:23:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.499+0000 7f1fe7400700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 0x7f1fcc06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.501+0000 7f1fe7400700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 0x7f1fcc06fdc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006010 tx=0x7f1fd400b540 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.505+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f1fd8056210 con 0x7f1fe80feb50 2024-04-24T08:23:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.779+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 --> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fe81088a0 con 0x7f1fcc06d910 2024-04-24T08:23:49.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.781+0000 7f1fe4c00700 1 -- 172.21.15.90:0/86736740 <== mgr.24325 v2:172.21.15.149:6828/1162352260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f1fe81088a0 con 0x7f1fcc06d910 2024-04-24T08:23:49.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 msgr2=0x7f1fcc06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 0x7f1fcc06fdc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006010 tx=0x7f1fd400b540 comp rx=0 tx=0).stop 2024-04-24T08:23:49.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 msgr2=0x7f1fe8197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c940 tx=0x7f1fd800cd00 comp rx=0 tx=0).stop 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 shutdown_connections 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1fcc06d910 0x7f1fcc06fdc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fe80feb50 0x7f1fe8197ce0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 --2- 172.21.15.90:0/86736740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fe80ff4a0 0x7f1fe8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.784+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 >> 172.21.15.90:0/86736740 conn(0x7f1fe80fa780 msgr2=0x7f1fe8107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.785+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 shutdown_connections 2024-04-24T08:23:49.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.785+0000 7f1fef000700 1 -- 172.21.15.90:0/86736740 wait complete. 2024-04-24T08:23:49.804 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 -- 172.21.15.90:0/14047330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 msgr2=0x7fa208105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 --2- 172.21.15.90:0/14047330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208105a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009910 tx=0x7fa1f4009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 -- 172.21.15.90:0/14047330 shutdown_connections 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 --2- 172.21.15.90:0/14047330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208105a30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 --2- 172.21.15.90:0/14047330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 0x7fa208103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.967+0000 7fa20fc00700 1 -- 172.21.15.90:0/14047330 >> 172.21.15.90:0/14047330 conn(0x7fa2080fa6d0 msgr2=0x7fa2080fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:49.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.968+0000 7fa20fc00700 1 -- 172.21.15.90:0/14047330 shutdown_connections 2024-04-24T08:23:49.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.968+0000 7fa20fc00700 1 -- 172.21.15.90:0/14047330 wait complete. 2024-04-24T08:23:49.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 Processor -- start 2024-04-24T08:23:49.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 -- start start 2024-04-24T08:23:49.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 0x7fa208197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081988a0 con 0x7fa208103650 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.970+0000 7fa20fc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20819d2b0 con 0x7fa208100d30 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.971+0000 7fa207e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.971+0000 7fa207e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46814/0 (socket says 172.21.15.90:46814) 2024-04-24T08:23:49.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.971+0000 7fa207e00700 1 -- 172.21.15.90:0/3194941931 learned_addr learned my addr 172.21.15.90:0/3194941931 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:49.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.971+0000 7fa20d200700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 0x7fa208197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.972+0000 7fa207e00700 1 -- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 msgr2=0x7fa208197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:49.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.972+0000 7fa207e00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 0x7fa208197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:49.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.972+0000 7fa207e00700 1 -- 172.21.15.90:0/3194941931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40095f0 con 0x7fa208103650 2024-04-24T08:23:49.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa207e00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005b40 tx=0x7fa1f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:49.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f401d070 con 0x7fa208103650 2024-04-24T08:23:49.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa1f4022470 con 0x7fa208103650 2024-04-24T08:23:49.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f400f690 con 0x7fa208103650 2024-04-24T08:23:49.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20819d450 con 0x7fa208103650 2024-04-24T08:23:49.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.973+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20819d910 con 0x7fa208103650 2024-04-24T08:23:49.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.974+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2080fc2b0 con 0x7fa208103650 2024-04-24T08:23:49.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.977+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91360+0+0 (secure 0 0 0) 0x7fa1f4022a10 con 0x7fa208103650 2024-04-24T08:23:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.978+0000 7fa205600700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 0x7fa1f006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.978+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa1f408f0b0 con 0x7fa208103650 2024-04-24T08:23:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.979+0000 7fa20d200700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 0x7fa1f006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.980+0000 7fa20d200700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 0x7fa1f006fdc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa208199220 tx=0x7fa1fc009190 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:49.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:49.982+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fa1f4058390 con 0x7fa208103650 2024-04-24T08:23:50.280 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:50 smithi090 ceph-mon[25016]: pgmap v17: 65 pgs: 65 active+clean; 122 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-04-24T08:23:50.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.297+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 --> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa208061190 con 0x7fa1f006d910 2024-04-24T08:23:50.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.301+0000 7fa205600700 1 -- 172.21.15.90:0/3194941931 <== mgr.24325 v2:172.21.15.149:6828/1162352260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa208061190 con 0x7fa1f006d910 2024-04-24T08:23:50.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 msgr2=0x7fa1f006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 0x7fa1f006fdc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa208199220 tx=0x7fa1fc009190 comp rx=0 tx=0).stop 2024-04-24T08:23:50.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 msgr2=0x7fa208198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005b40 tx=0x7fa1f400bd70 comp rx=0 tx=0).stop 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 shutdown_connections 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fa1f006d910 0x7fa1f006fdc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa208100d30 0x7fa208197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.305+0000 7fa20fc00700 1 --2- 172.21.15.90:0/3194941931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa208103650 0x7fa208198280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.306+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 >> 172.21.15.90:0/3194941931 conn(0x7fa2080fa6d0 msgr2=0x7fa2080ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.306+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 shutdown_connections 2024-04-24T08:23:50.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.306+0000 7fa20fc00700 1 -- 172.21.15.90:0/3194941931 wait complete. 2024-04-24T08:23:50.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 -- 172.21.15.90:0/1452609041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c0737b0 msgr2=0x7fd35c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 --2- 172.21.15.90:0/1452609041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c0737b0 0x7fd35c073c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd348009910 tx=0x7fd348009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:50.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 -- 172.21.15.90:0/1452609041 shutdown_connections 2024-04-24T08:23:50.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 --2- 172.21.15.90:0/1452609041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c0737b0 0x7fd35c073c20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 --2- 172.21.15.90:0/1452609041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c074d80 0x7fd35c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 -- 172.21.15.90:0/1452609041 >> 172.21.15.90:0/1452609041 conn(0x7fd35c0fb7f0 msgr2=0x7fd35c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 -- 172.21.15.90:0/1452609041 shutdown_connections 2024-04-24T08:23:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.479+0000 7fd361800700 1 -- 172.21.15.90:0/1452609041 wait complete. 2024-04-24T08:23:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.480+0000 7fd361800700 1 Processor -- start 2024-04-24T08:23:50.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd361800700 1 -- start start 2024-04-24T08:23:50.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd361800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c0737b0 0x7fd35c19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd361800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd361800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c19cc70 con 0x7fd35c074d80 2024-04-24T08:23:50.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd361800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c19cdb0 con 0x7fd35c0737b0 2024-04-24T08:23:50.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd35a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd35a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46840/0 (socket says 172.21.15.90:46840) 2024-04-24T08:23:50.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.481+0000 7fd35a000700 1 -- 172.21.15.90:0/2392604743 learned_addr learned my addr 172.21.15.90:0/2392604743 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.482+0000 7fd35aa00700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c0737b0 0x7fd35c19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.482+0000 7fd35a000700 1 -- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c0737b0 msgr2=0x7fd35c19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.482+0000 7fd35a000700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c0737b0 0x7fd35c19c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.482+0000 7fd35a000700 1 -- 172.21.15.90:0/2392604743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480095f0 con 0x7fd35c074d80 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.482+0000 7fd35a000700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd348005b40 tx=0x7fd34800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.483+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd34801d070 con 0x7fd35c074d80 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.483+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd348022470 con 0x7fd35c074d80 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.483+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c1a1800 con 0x7fd35c074d80 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.483+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd34800f690 con 0x7fd35c074d80 2024-04-24T08:23:50.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.483+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c1a1cf0 con 0x7fd35c074d80 2024-04-24T08:23:50.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.484+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c10f3f0 con 0x7fd35c074d80 2024-04-24T08:23:50.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.485+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91360+0+0 (secure 0 0 0) 0x7fd3480225e0 con 0x7fd35c074d80 2024-04-24T08:23:50.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.487+0000 7fd353400700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 0x7fd34006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.487+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd34808eba0 con 0x7fd35c074d80 2024-04-24T08:23:50.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.488+0000 7fd35aa00700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 0x7fd34006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.489+0000 7fd35aa00700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 0x7fd34006fdc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd34c006fd0 tx=0x7fd34c008040 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.492+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fd34805b910 con 0x7fd35c074d80 2024-04-24T08:23:50.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:50 smithi149 ceph-mon[31282]: pgmap v17: 65 pgs: 65 active+clean; 122 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-04-24T08:23:50.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.756+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 --> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd35c1a23f0 con 0x7fd34006d910 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.776+0000 7fd353400700 1 -- 172.21.15.90:0/2392604743 <== mgr.24325 v2:172.21.15.149:6828/1162352260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd35c1a23f0 con 0x7fd34006d910 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 15s ago 10m 14.2M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (11m) 15s ago 11m 7772k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (9m) 16s ago 9m 7696k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (11m) 15s ago 11m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:23:50.777 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (9m) 16s ago 9m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (8m) 15s ago 10m 76.6M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (3m) 15s ago 3m 94.5M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (3m) 15s ago 3m 13.5M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (3m) 16s ago 3m 106M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (3m) 16s ago 3m 13.4M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:9283,8765,8443 running (12m) 15s ago 12m 375M - 18.2.0 dc2bc1663786 7ccc358e4d84 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (48s) 16s ago 9m 488M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 839fd9b0cdf9 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (12m) 15s ago 12m 49.5M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (9m) 16s ago 9m 38.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (11m) 15s ago 11m 8208k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (9m) 16s ago 9m 7784k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:23:50.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 15s ago 8m 70.4M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 15s ago 7m 69.0M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 15s ago 7m 65.8M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (6m) 16s ago 6m 76.2M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (6m) 16s ago 6m 70.5M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (5m) 16s ago 5m 65.2M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:23:50.779 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (8m) 15s ago 10m 59.6M - 2.43.0 a07b618ecd1d 0199ca15cf9f 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 msgr2=0x7fd34006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 0x7fd34006fdc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd34c006fd0 tx=0x7fd34c008040 comp rx=0 tx=0).stop 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 msgr2=0x7fd35c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd348005b40 tx=0x7fd34800be20 comp rx=0 tx=0).stop 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 shutdown_connections 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7fd34006d910 0x7fd34006fdc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c0737b0 0x7fd35c19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 --2- 172.21.15.90:0/2392604743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35c074d80 0x7fd35c19c650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 >> 172.21.15.90:0/2392604743 conn(0x7fd35c0fb7f0 msgr2=0x7fd35c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:50.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 shutdown_connections 2024-04-24T08:23:50.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.779+0000 7fd361800700 1 -- 172.21.15.90:0/2392604743 wait complete. 2024-04-24T08:23:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.932+0000 7f1e87600700 1 -- 172.21.15.90:0/2833987854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 msgr2=0x7f1e80073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.932+0000 7f1e87600700 1 --2- 172.21.15.90:0/2833987854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e80073c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009910 tx=0x7f1e6c009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 -- 172.21.15.90:0/2833987854 shutdown_connections 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 --2- 172.21.15.90:0/2833987854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e80073c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 --2- 172.21.15.90:0/2833987854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 0x7f1e800731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 -- 172.21.15.90:0/2833987854 >> 172.21.15.90:0/2833987854 conn(0x7f1e800fb7f0 msgr2=0x7f1e800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 -- 172.21.15.90:0/2833987854 shutdown_connections 2024-04-24T08:23:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.933+0000 7f1e87600700 1 -- 172.21.15.90:0/2833987854 wait complete. 2024-04-24T08:23:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.934+0000 7f1e87600700 1 Processor -- start 2024-04-24T08:23:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.934+0000 7f1e87600700 1 -- start start 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e87600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e87600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 0x7f1e8019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e87600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8019cbd0 con 0x7f1e800737b0 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e87600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8019cd10 con 0x7f1e80074d80 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e84c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e84c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46856/0 (socket says 172.21.15.90:46856) 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e84c00700 1 -- 172.21.15.90:0/540446087 learned_addr learned my addr 172.21.15.90:0/540446087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.935+0000 7f1e7fe00700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 0x7f1e8019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e84c00700 1 -- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 msgr2=0x7f1e8019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e84c00700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 0x7f1e8019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e84c00700 1 -- 172.21.15.90:0/540446087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0095f0 con 0x7f1e800737b0 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e84c00700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e920 tx=0x7f1e7400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e7400c9d0 con 0x7f1e800737b0 2024-04-24T08:23:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1e74010430 con 0x7f1e800737b0 2024-04-24T08:23:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.936+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e801a17c0 con 0x7f1e800737b0 2024-04-24T08:23:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.937+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e801a1d10 con 0x7f1e800737b0 2024-04-24T08:23:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.937+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e74018650 con 0x7f1e800737b0 2024-04-24T08:23:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.938+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e60005320 con 0x7f1e800737b0 2024-04-24T08:23:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.939+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91360+0+0 (secure 0 0 0) 0x7f1e74010a60 con 0x7f1e800737b0 2024-04-24T08:23:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.940+0000 7f1e7d600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 0x7f1e6806fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.940+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1e74014070 con 0x7f1e800737b0 2024-04-24T08:23:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.940+0000 7f1e7fe00700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 0x7f1e6806fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.941+0000 7f1e7fe00700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 0x7f1e6806fdd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fd0 tx=0x7f1e6c005fd0 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:50.944+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f1e74053ca0 con 0x7f1e800737b0 2024-04-24T08:23:51.231 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:51 smithi090 ceph-mon[25016]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.231 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:51 smithi090 ceph-mon[25016]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.231 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:51 smithi090 ceph-mon[25016]: pgmap v18: 65 pgs: 65 active+clean; 122 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-04-24T08:23:51.231 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:51 smithi090 ceph-mon[25016]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.231 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:51 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:51.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:51 smithi149 ceph-mon[31282]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:51 smithi149 ceph-mon[31282]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:51 smithi149 ceph-mon[31282]: pgmap v18: 65 pgs: 65 active+clean; 122 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-04-24T08:23:51.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:51 smithi149 ceph-mon[31282]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:51.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:51 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:23:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.323+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e60005cc0 con 0x7f1e800737b0 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.325+0000 7f1e7d600700 1 -- 172.21.15.90:0/540446087 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f1e7405a380 con 0x7f1e800737b0 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:23:51.326 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:23:51.327 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 msgr2=0x7f1e6806fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 0x7f1e6806fdd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fd0 tx=0x7f1e6c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 msgr2=0x7f1e8019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e920 tx=0x7f1e7400ec30 comp rx=0 tx=0).stop 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 shutdown_connections 2024-04-24T08:23:51.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f1e6806d920 0x7f1e6806fdd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e800737b0 0x7f1e8019c070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 --2- 172.21.15.90:0/540446087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e80074d80 0x7f1e8019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 >> 172.21.15.90:0/540446087 conn(0x7f1e800fb7f0 msgr2=0x7f1e80101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:51.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 shutdown_connections 2024-04-24T08:23:51.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.328+0000 7f1e87600700 1 -- 172.21.15.90:0/540446087 wait complete. 2024-04-24T08:23:51.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.494+0000 7f41eee00700 1 -- 172.21.15.90:0/1121558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8101440 msgr2=0x7f41e8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.494+0000 7f41eee00700 1 --2- 172.21.15.90:0/1121558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8101440 0x7f41e8101890 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009910 tx=0x7f41d4009c20 comp rx=0 tx=0).stop 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 -- 172.21.15.90:0/1121558826 shutdown_connections 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 --2- 172.21.15.90:0/1121558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8101440 0x7f41e8101890 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 --2- 172.21.15.90:0/1121558826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8100240 0x7f41e8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 -- 172.21.15.90:0/1121558826 >> 172.21.15.90:0/1121558826 conn(0x7f41e80fb7f0 msgr2=0x7f41e80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 -- 172.21.15.90:0/1121558826 shutdown_connections 2024-04-24T08:23:51.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.495+0000 7f41eee00700 1 -- 172.21.15.90:0/1121558826 wait complete. 2024-04-24T08:23:51.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.496+0000 7f41eee00700 1 Processor -- start 2024-04-24T08:23:51.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41eee00700 1 -- start start 2024-04-24T08:23:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41eee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:51.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41eee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 0x7f41e8195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41eee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8196610 con 0x7f41e8100240 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41eee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8196750 con 0x7f41e8101440 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46880/0 (socket says 172.21.15.90:46880) 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.497+0000 7f41e7e00700 1 -- 172.21.15.90:0/2796853928 learned_addr learned my addr 172.21.15.90:0/2796853928 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:23:51.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7400700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 0x7f41e8195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7e00700 1 -- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 msgr2=0x7f41e8195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7e00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 0x7f41e8195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7e00700 1 -- 172.21.15.90:0/2796853928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40095f0 con 0x7f41e8100240 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7400700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 0x7f41e8195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.498+0000 7f41e7e00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41d800d900 tx=0x7f41d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.499+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d80041f0 con 0x7f41e8100240 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.499+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41d8004350 con 0x7f41e8100240 2024-04-24T08:23:51.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.499+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d8003da0 con 0x7f41e8100240 2024-04-24T08:23:51.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.499+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e819b200 con 0x7f41e8100240 2024-04-24T08:23:51.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.499+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e819b750 con 0x7f41e8100240 2024-04-24T08:23:51.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.501+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e8108d90 con 0x7f41e8100240 2024-04-24T08:23:51.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.502+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91360+0+0 (secure 0 0 0) 0x7f41d8009730 con 0x7f41e8100240 2024-04-24T08:23:51.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.502+0000 7f41e4c00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 0x7f41cc06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:23:51.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.503+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f41d808d390 con 0x7f41e8100240 2024-04-24T08:23:51.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.503+0000 7f41e7400700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 0x7f41cc06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:23:51.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.504+0000 7f41e7400700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 0x7f41cc06fdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41d4006010 tx=0x7f41d400b560 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:23:51.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.508+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f41d8059fe0 con 0x7f41e8100240 2024-04-24T08:23:51.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.867+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 --> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41e819be50 con 0x7f41cc06d910 2024-04-24T08:23:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.871+0000 7f41e4c00700 1 -- 172.21.15.90:0/2796853928 <== mgr.24325 v2:172.21.15.149:6828/1162352260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f41e819be50 con 0x7f41cc06d910 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:23:51.872 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.874+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 msgr2=0x7f41cc06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.874+0000 7f41eee00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 0x7f41cc06fdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41d4006010 tx=0x7f41d400b560 comp rx=0 tx=0).stop 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.874+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 msgr2=0x7f41e8195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.874+0000 7f41eee00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41d800d900 tx=0x7f41d800dcc0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 shutdown_connections 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:6828/1162352260,v1:172.21.15.149:6829/1162352260] conn(0x7f41cc06d910 0x7f41cc06fdc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41e8100240 0x7f41e8195ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 --2- 172.21.15.90:0/2796853928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41e8101440 0x7f41e8195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:23:51.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 >> 172.21.15.90:0/2796853928 conn(0x7f41e80fb7f0 msgr2=0x7f41e8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:23:51.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 shutdown_connections 2024-04-24T08:23:51.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:23:51.875+0000 7f41eee00700 1 -- 172.21.15.90:0/2796853928 wait complete. 2024-04-24T08:23:52.403 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:52 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/540446087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:23:52.404 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:52 smithi090 ceph-mon[25016]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:52 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/540446087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:23:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:52 smithi149 ceph-mon[31282]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:23:53.242 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:53 smithi090 ceph-mon[25016]: pgmap v19: 65 pgs: 65 active+clean; 131 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-04-24T08:23:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:53 smithi149 ceph-mon[31282]: pgmap v19: 65 pgs: 65 active+clean; 131 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-04-24T08:23:56.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:55 smithi149 ceph-mon[31282]: pgmap v20: 65 pgs: 65 active+clean; 131 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-04-24T08:23:56.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:55 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:56.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:55 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:56.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:55 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:23:56.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:55 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:56.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:55 smithi090 ceph-mon[25016]: pgmap v20: 65 pgs: 65 active+clean; 131 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-04-24T08:23:56.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:55 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:56.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:55 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:56.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:55 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:23:56.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:55 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:23:57.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:56 smithi149 ceph-mon[31282]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:23:57.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:56 smithi149 ceph-mon[31282]: Upgrade: Need to upgrade myself (mgr.smithi149.vavftl) 2024-04-24T08:23:57.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:56 smithi149 ceph-mon[31282]: Upgrade: Need to upgrade myself (mgr.smithi149.vavftl) 2024-04-24T08:23:57.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:56 smithi090 ceph-mon[25016]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:23:57.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:56 smithi090 ceph-mon[25016]: Upgrade: Need to upgrade myself (mgr.smithi149.vavftl) 2024-04-24T08:23:57.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:56 smithi090 ceph-mon[25016]: Upgrade: Need to upgrade myself (mgr.smithi149.vavftl) 2024-04-24T08:23:58.028 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:57 smithi090 ceph-mon[25016]: pgmap v21: 65 pgs: 65 active+clean; 137 MiB data, 639 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-04-24T08:23:58.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:57 smithi149 ceph-mon[31282]: pgmap v21: 65 pgs: 65 active+clean; 137 MiB data, 639 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 98 op/s 2024-04-24T08:23:59.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: Upgrade: Updating mgr.smithi090.jsrfxo 2024-04-24T08:23:59.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:59.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:23:59.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:23:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:23:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:58 smithi090 ceph-mon[25016]: Deploying daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: Upgrade: Updating mgr.smithi090.jsrfxo 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:23:59.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:58 smithi149 ceph-mon[31282]: Deploying daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:24:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:23:59 smithi090 ceph-mon[25016]: pgmap v22: 65 pgs: 65 active+clean; 138 MiB data, 644 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-04-24T08:24:00.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:23:59 smithi149 ceph-mon[31282]: pgmap v22: 65 pgs: 65 active+clean; 138 MiB data, 644 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-04-24T08:24:01.997 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:01 smithi090 ceph-mon[25016]: pgmap v23: 65 pgs: 65 active+clean; 138 MiB data, 644 MiB used, 536 GiB / 536 GiB avail; 953 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-04-24T08:24:02.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:01 smithi149 ceph-mon[31282]: pgmap v23: 65 pgs: 65 active+clean; 138 MiB data, 644 MiB used, 536 GiB / 536 GiB avail; 953 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-04-24T08:24:04.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:03 smithi149 ceph-mon[31282]: pgmap v24: 65 pgs: 65 active+clean; 149 MiB data, 687 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 144 op/s 2024-04-24T08:24:04.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:03 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:04.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:03 smithi090 ceph-mon[25016]: pgmap v24: 65 pgs: 65 active+clean; 149 MiB data, 687 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 144 op/s 2024-04-24T08:24:04.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:03 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:05.208 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:04 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:05.209 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:04 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:05.209 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:04 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:04 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:04 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:04 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:06.138 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:05 smithi149 ceph-mon[31282]: pgmap v25: 65 pgs: 65 active+clean; 149 MiB data, 687 MiB used, 536 GiB / 536 GiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-24T08:24:06.138 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:05 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:06.138 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:05 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:06.254 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:05 smithi090 ceph-mon[25016]: pgmap v25: 65 pgs: 65 active+clean; 149 MiB data, 687 MiB used, 536 GiB / 536 GiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-24T08:24:06.254 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:05 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:06.254 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:05 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:08.198 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:07 smithi090 ceph-mon[25016]: pgmap v26: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-04-24T08:24:08.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:07 smithi149 ceph-mon[31282]: pgmap v26: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-04-24T08:24:10.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:09 smithi149 ceph-mon[31282]: pgmap v27: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 812 KiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-04-24T08:24:10.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:09 smithi090 ceph-mon[25016]: pgmap v27: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 812 KiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-04-24T08:24:12.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:11 smithi090 ceph-mon[25016]: pgmap v28: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 811 KiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-04-24T08:24:12.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:11 smithi149 ceph-mon[31282]: pgmap v28: 65 pgs: 65 active+clean; 159 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 811 KiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-04-24T08:24:14.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:13 smithi149 ceph-mon[31282]: pgmap v29: 65 pgs: 65 active+clean; 169 MiB data, 753 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-04-24T08:24:14.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:13 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:14.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:13 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:14.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:13 smithi090 ceph-mon[25016]: pgmap v29: 65 pgs: 65 active+clean; 169 MiB data, 753 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-04-24T08:24:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:13 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:13 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:15.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: Standby manager daemon smithi090.jsrfxo restarted 2024-04-24T08:24:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: Standby manager daemon smithi090.jsrfxo started 2024-04-24T08:24:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/crt"}]: dispatch 2024-04-24T08:24:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:24:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/key"}]: dispatch 2024-04-24T08:24:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:14 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:24:15.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: Standby manager daemon smithi090.jsrfxo restarted 2024-04-24T08:24:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: Standby manager daemon smithi090.jsrfxo started 2024-04-24T08:24:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/crt"}]: dispatch 2024-04-24T08:24:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:24:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.jsrfxo/key"}]: dispatch 2024-04-24T08:24:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:14 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:24:16.220 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:15 smithi090 ceph-mon[25016]: pgmap v30: 65 pgs: 65 active+clean; 169 MiB data, 753 MiB used, 536 GiB / 536 GiB avail; 858 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-04-24T08:24:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:15 smithi090 ceph-mon[25016]: mgrmap e26: smithi149.vavftl(active, since 54s), standbys: smithi090.jsrfxo 2024-04-24T08:24:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:15 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:15 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:16.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:15 smithi149 ceph-mon[31282]: pgmap v30: 65 pgs: 65 active+clean; 169 MiB data, 753 MiB used, 536 GiB / 536 GiB avail; 858 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-04-24T08:24:16.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:15 smithi149 ceph-mon[31282]: mgrmap e26: smithi149.vavftl(active, since 54s), standbys: smithi090.jsrfxo 2024-04-24T08:24:16.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:15 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:16.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:15 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:18.095 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:17 smithi090 ceph-mon[25016]: pgmap v31: 65 pgs: 65 active+clean; 177 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-04-24T08:24:18.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:17 smithi149 ceph-mon[31282]: pgmap v31: 65 pgs: 65 active+clean; 177 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-04-24T08:24:20.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:19 smithi149 ceph-mon[31282]: pgmap v32: 65 pgs: 65 active+clean; 177 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-24T08:24:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:19 smithi090 ceph-mon[25016]: pgmap v32: 65 pgs: 65 active+clean; 177 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-24T08:24:21.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd='[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]': finished 2024-04-24T08:24:21.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: mgrmap e27: smithi090.jsrfxo(active, starting, since 0.0189565s) 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:24:21.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:24:21.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:24:21.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 172.21.15.149:0/3242675526' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.24325 ' entity='mgr.smithi149.vavftl' cmd='[{"prefix": "mgr fail", "who": "smithi149.vavftl"}]': finished 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: mgrmap e27: smithi090.jsrfxo(active, starting, since 0.0189565s) 2024-04-24T08:24:21.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:24:21.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:24:21.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 -- 172.21.15.90:0/4185314665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058073730 msgr2=0x7f7058073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 --2- 172.21.15.90:0/4185314665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058073730 0x7f7058073ba0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7044009910 tx=0x7f7044009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 -- 172.21.15.90:0/4185314665 shutdown_connections 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 --2- 172.21.15.90:0/4185314665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058073730 0x7f7058073ba0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 --2- 172.21.15.90:0/4185314665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058074d00 0x7f7058073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 -- 172.21.15.90:0/4185314665 >> 172.21.15.90:0/4185314665 conn(0x7f70580fb7f0 msgr2=0x7f70580fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 -- 172.21.15.90:0/4185314665 shutdown_connections 2024-04-24T08:24:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.060+0000 7f705f200700 1 -- 172.21.15.90:0/4185314665 wait complete. 2024-04-24T08:24:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.062+0000 7f705f200700 1 Processor -- start 2024-04-24T08:24:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.062+0000 7f705f200700 1 -- start start 2024-04-24T08:24:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f705f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058073730 0x7f7058197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f705f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f705f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70581987b0 con 0x7f7058074d00 2024-04-24T08:24:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f705f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058198920 con 0x7f7058073730 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f7057400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f7057400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55394/0 (socket says 172.21.15.90:55394) 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f7057400700 1 -- 172.21.15.90:0/3582231625 learned_addr learned my addr 172.21.15.90:0/3582231625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.063+0000 7f7057e00700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058073730 0x7f7058197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7057400700 1 -- 172.21.15.90:0/3582231625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058073730 msgr2=0x7f7058197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7057400700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058073730 0x7f7058197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7057400700 1 -- 172.21.15.90:0/3582231625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440095f0 con 0x7f7058074d00 2024-04-24T08:24:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7057400700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7044000c00 tx=0x7f704400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:22.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704401d070 con 0x7f7058074d00 2024-04-24T08:24:22.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7044005dc0 con 0x7f7058074d00 2024-04-24T08:24:22.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705819d350 con 0x7f7058074d00 2024-04-24T08:24:22.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.064+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70440226a0 con 0x7f7058074d00 2024-04-24T08:24:22.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.065+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705819d840 con 0x7f7058074d00 2024-04-24T08:24:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.066+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 46347+0+0 (secure 0 0 0) 0x7f7044017400 con 0x7f7058074d00 2024-04-24T08:24:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.066+0000 7f7054c00700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 0x7f703c03b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.066+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705810af70 con 0x7f7058074d00 2024-04-24T08:24:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.066+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f704404ebc0 con 0x7f7058074d00 2024-04-24T08:24:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.067+0000 7f7057e00700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 0x7f703c03b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.067+0000 7f7057e00700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 0x7f703c03b970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7058198b50 tx=0x7f7048009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:22.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.073+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f7044017cb0 con 0x7f7058074d00 2024-04-24T08:24:22.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.434+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70581077c0 con 0x7f703c0394c0 2024-04-24T08:24:22.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.437+0000 7f7054c00700 1 -- 172.21.15.90:0/3582231625 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f70581077c0 con 0x7f703c0394c0 2024-04-24T08:24:22.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 msgr2=0x7f703c03b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 0x7f703c03b970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7058198b50 tx=0x7f7048009260 comp rx=0 tx=0).stop 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 msgr2=0x7f70581981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7044000c00 tx=0x7f704400f740 comp rx=0 tx=0).stop 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 shutdown_connections 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f703c0394c0 0x7f703c03b970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7058073730 0x7f7058197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 --2- 172.21.15.90:0/3582231625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7058074d00 0x7f70581981b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 >> 172.21.15.90:0/3582231625 conn(0x7f70580fb7f0 msgr2=0x7f70581060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 shutdown_connections 2024-04-24T08:24:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.440+0000 7f705f200700 1 -- 172.21.15.90:0/3582231625 wait complete. 2024-04-24T08:24:22.463 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:24:22.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.645+0000 7fb5af800700 1 -- 172.21.15.90:0/2410810777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8100240 msgr2=0x7fb5a8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.645+0000 7fb5af800700 1 --2- 172.21.15.90:0/2410810777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8100240 0x7fb5a8100650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009910 tx=0x7fb59c009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:24:21] ENGINE Bus STARTING 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:24:21] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: mgrmap e28: smithi090.jsrfxo(active, since 1.01831s) 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:24:21] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: [24/Apr/2024:08:24:21] ENGINE Bus STARTED 2024-04-24T08:24:22.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:22 smithi090 ceph-mon[25016]: pgmap v3: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:22.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 -- 172.21.15.90:0/2410810777 shutdown_connections 2024-04-24T08:24:22.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 --2- 172.21.15.90:0/2410810777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8101440 0x7fb5a8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 --2- 172.21.15.90:0/2410810777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8100240 0x7fb5a8100650 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 -- 172.21.15.90:0/2410810777 >> 172.21.15.90:0/2410810777 conn(0x7fb5a80fb7f0 msgr2=0x7fb5a80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:22.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 -- 172.21.15.90:0/2410810777 shutdown_connections 2024-04-24T08:24:22.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.646+0000 7fb5af800700 1 -- 172.21.15.90:0/2410810777 wait complete. 2024-04-24T08:24:22.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.648+0000 7fb5af800700 1 Processor -- start 2024-04-24T08:24:22.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.650+0000 7fb5af800700 1 -- start start 2024-04-24T08:24:22.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.650+0000 7fb5af800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 0x7fb5a8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.650+0000 7fb5af800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.650+0000 7fb5af800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8073680 con 0x7fb5a8101440 2024-04-24T08:24:22.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.650+0000 7fb5af800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80737c0 con 0x7fb5a8100240 2024-04-24T08:24:22.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.651+0000 7fb5ace00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 0x7fb5a8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.651+0000 7fb5a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.651+0000 7fb5a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55438/0 (socket says 172.21.15.90:55438) 2024-04-24T08:24:22.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.651+0000 7fb5a7e00700 1 -- 172.21.15.90:0/1325225369 learned_addr learned my addr 172.21.15.90:0/1325225369 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:22.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.652+0000 7fb5ace00700 1 -- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 msgr2=0x7fb5a8073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:22.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.652+0000 7fb5ace00700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:22.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.652+0000 7fb5ace00700 1 -- 172.21.15.90:0/1325225369 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0095f0 con 0x7fb5a8100240 2024-04-24T08:24:22.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.652+0000 7fb5a7e00700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:24:22.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.652+0000 7fb5ace00700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 0x7fb5a8074af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb59c000c00 tx=0x7fb59c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.653+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59c01d070 con 0x7fb5a8100240 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.653+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb59c005dc0 con 0x7fb5a8100240 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.653+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59c022470 con 0x7fb5a8100240 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.653+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a8073a40 con 0x7fb5a8100240 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.653+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a8073f00 con 0x7fb5a8100240 2024-04-24T08:24:22.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.654+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 28) v1 ==== 46347+0+0 (secure 0 0 0) 0x7fb59c0225d0 con 0x7fb5a8100240 2024-04-24T08:24:22.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.655+0000 7fb5a5600700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 0x7fb59003b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:22.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.655+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb59c04f9c0 con 0x7fb5a8100240 2024-04-24T08:24:22.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.655+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb588005320 con 0x7fb5a8100240 2024-04-24T08:24:22.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.655+0000 7fb5a7e00700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 0x7fb59003b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:22.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.656+0000 7fb5a7e00700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 0x7fb59003b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb594011550 tx=0x7fb594009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:22.663+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fb59c022920 con 0x7fb5a8100240 2024-04-24T08:24:22.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:24:21] ENGINE Bus STARTING 2024-04-24T08:24:22.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:24:21] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:24:22.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: mgrmap e28: smithi090.jsrfxo(active, since 1.01831s) 2024-04-24T08:24:22.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:24:21] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:24:22.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: [24/Apr/2024:08:24:21] ENGINE Bus STARTED 2024-04-24T08:24:22.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:22 smithi149 ceph-mon[31282]: pgmap v3: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.001+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb588000bf0 con 0x7fb5900394c0 2024-04-24T08:24:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.004+0000 7fb5a5600700 1 -- 172.21.15.90:0/1325225369 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fb588000bf0 con 0x7fb5900394c0 2024-04-24T08:24:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.007+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 msgr2=0x7fb59003b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.007+0000 7fb5af800700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 0x7fb59003b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb594011550 tx=0x7fb594009260 comp rx=0 tx=0).stop 2024-04-24T08:24:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.007+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 msgr2=0x7fb5a8074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.007+0000 7fb5af800700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 0x7fb5a8074af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb59c000c00 tx=0x7fb59c00fa00 comp rx=0 tx=0).stop 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 shutdown_connections 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fb5900394c0 0x7fb59003b970 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a8100240 0x7fb5a8074af0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 --2- 172.21.15.90:0/1325225369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a8101440 0x7fb5a8073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 >> 172.21.15.90:0/1325225369 conn(0x7fb5a80fb7f0 msgr2=0x7fb5a8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 shutdown_connections 2024-04-24T08:24:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.008+0000 7fb5af800700 1 -- 172.21.15.90:0/1325225369 wait complete. 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.187+0000 7fbb79a00700 1 -- 172.21.15.90:0/2867996323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74103640 msgr2=0x7fbb74103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.187+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2867996323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74103640 0x7fbb74103a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009910 tx=0x7fbb60009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.187+0000 7fbb79a00700 1 -- 172.21.15.90:0/2867996323 shutdown_connections 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.187+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2867996323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74103640 0x7fbb74103a90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.187+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2867996323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74102490 0x7fbb741028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.188+0000 7fbb79a00700 1 -- 172.21.15.90:0/2867996323 >> 172.21.15.90:0/2867996323 conn(0x7fbb740fda60 msgr2=0x7fbb740ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:23.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.188+0000 7fbb79a00700 1 -- 172.21.15.90:0/2867996323 shutdown_connections 2024-04-24T08:24:23.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.188+0000 7fbb79a00700 1 -- 172.21.15.90:0/2867996323 wait complete. 2024-04-24T08:24:23.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.189+0000 7fbb79a00700 1 Processor -- start 2024-04-24T08:24:23.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.189+0000 7fbb79a00700 1 -- start start 2024-04-24T08:24:23.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb79a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb79a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74103640 0x7fbb741981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb79a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741987f0 con 0x7fbb74102490 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb79a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74198960 con 0x7fbb74103640 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55446/0 (socket says 172.21.15.90:55446) 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb72a00700 1 -- 172.21.15.90:0/2921663867 learned_addr learned my addr 172.21.15.90:0/2921663867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.190+0000 7fbb72000700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74103640 0x7fbb741981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.191+0000 7fbb72a00700 1 -- 172.21.15.90:0/2921663867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74103640 msgr2=0x7fbb741981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.191+0000 7fbb72a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74103640 0x7fbb741981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.191+0000 7fbb72a00700 1 -- 172.21.15.90:0/2921663867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600095f0 con 0x7fbb74102490 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.191+0000 7fbb72a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400e8f0 tx=0x7fbb6400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.192+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb64009790 con 0x7fbb74102490 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.192+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7419d3f0 con 0x7fbb74102490 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.192+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7419d940 con 0x7fbb74102490 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.192+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb6400c810 con 0x7fbb74102490 2024-04-24T08:24:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.192+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb64003d20 con 0x7fbb74102490 2024-04-24T08:24:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.193+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 46347+0+0 (secure 0 0 0) 0x7fbb64018430 con 0x7fbb74102490 2024-04-24T08:24:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.194+0000 7fbb6b400700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 0x7fbb5803b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.194+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbb64014070 con 0x7fbb74102490 2024-04-24T08:24:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.194+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54005320 con 0x7fbb74102490 2024-04-24T08:24:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.195+0000 7fbb72000700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 0x7fbb5803b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:23.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.195+0000 7fbb72000700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 0x7fbb5803b970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb741991c0 tx=0x7fbb60005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:23.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.201+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fbb64021430 con 0x7fbb74102490 2024-04-24T08:24:23.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.412+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 29) v1 ==== 46641+0+0 (secure 0 0 0) 0x7fbb64017cd0 con 0x7fbb74102490 2024-04-24T08:24:23.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.490+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb54000bf0 con 0x7fbb580394c0 2024-04-24T08:24:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.514+0000 7fbb6b400700 1 -- 172.21.15.90:0/2921663867 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbb54000bf0 con 0x7fbb580394c0 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 10s ago 11m 14.2M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (11m) 10s ago 11m 7818k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (10m) 49s ago 10m 7696k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (11m) 10s ago 11m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (10m) 49s ago 10m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (9m) 10s ago 11m 75.7M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (4m) 10s ago 4m 164M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:24:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (4m) 10s ago 4m 13.8M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (4m) 49s ago 4m 106M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (3m) 49s ago 3m 13.4M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (20s) 10s ago 13m 64.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (80s) 49s ago 9m 488M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 839fd9b0cdf9 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (13m) 10s ago 13m 51.1M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (9m) 49s ago 9m 38.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (11m) 10s ago 11m 7922k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:24:23.517 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (10m) 49s ago 10m 7784k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 10s ago 8m 79.4M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (8m) 10s ago 8m 80.7M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 10s ago 7m 70.6M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (7m) 49s ago 7m 76.2M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (6m) 49s ago 6m 70.5M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (5m) 49s ago 5m 65.2M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:24:23.518 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (29s) 10s ago 10m 40.0M - 2.43.0 a07b618ecd1d 541c9856b6ae 2024-04-24T08:24:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 msgr2=0x7fbb5803b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 0x7fbb5803b970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb741991c0 tx=0x7fbb60005fd0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 msgr2=0x7fbb74197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400e8f0 tx=0x7fbb6400ec00 comp rx=0 tx=0).stop 2024-04-24T08:24:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 shutdown_connections 2024-04-24T08:24:23.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fbb580394c0 0x7fbb5803b970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb74102490 0x7fbb74197cb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 --2- 172.21.15.90:0/2921663867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb74103640 0x7fbb741981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.518+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 >> 172.21.15.90:0/2921663867 conn(0x7fbb740fda60 msgr2=0x7fbb74106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.519+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 shutdown_connections 2024-04-24T08:24:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.519+0000 7fbb79a00700 1 -- 172.21.15.90:0/2921663867 wait complete. 2024-04-24T08:24:23.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 -- 172.21.15.90:0/1885697888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 msgr2=0x7f32e0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:23.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 --2- 172.21.15.90:0/1885697888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0100650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009910 tx=0x7f32d0009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:23.708 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:23 smithi090 ceph-mon[25016]: pgmap v4: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:23.708 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:23 smithi090 ceph-mon[25016]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:23.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 -- 172.21.15.90:0/1885697888 shutdown_connections 2024-04-24T08:24:23.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 --2- 172.21.15.90:0/1885697888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e0101440 0x7f32e0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 --2- 172.21.15.90:0/1885697888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0100650 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.706+0000 7f32e4a00700 1 -- 172.21.15.90:0/1885697888 >> 172.21.15.90:0/1885697888 conn(0x7f32e00fb7f0 msgr2=0x7f32e00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:23.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.709+0000 7f32e4a00700 1 -- 172.21.15.90:0/1885697888 shutdown_connections 2024-04-24T08:24:23.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.709+0000 7f32e4a00700 1 -- 172.21.15.90:0/1885697888 wait complete. 2024-04-24T08:24:23.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.710+0000 7f32e4a00700 1 Processor -- start 2024-04-24T08:24:23.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.711+0000 7f32e4a00700 1 -- start start 2024-04-24T08:24:23.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.711+0000 7f32e4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.711+0000 7f32e4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e0101440 0x7f32e01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.711+0000 7f32e4a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0198800 con 0x7f32e0100240 2024-04-24T08:24:23.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.711+0000 7f32e4a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0198970 con 0x7f32e0101440 2024-04-24T08:24:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55462/0 (socket says 172.21.15.90:55462) 2024-04-24T08:24:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 -- 172.21.15.90:0/2008963285 learned_addr learned my addr 172.21.15.90:0/2008963285 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 -- 172.21.15.90:0/2008963285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e0101440 msgr2=0x7f32e01981b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T08:24:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e0101440 0x7f32e01981b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.712+0000 7f32dde00700 1 -- 172.21.15.90:0/2008963285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00095f0 con 0x7f32e0100240 2024-04-24T08:24:23.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.713+0000 7f32dde00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32d000f690 tx=0x7f32d000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:23.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.713+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32d001d070 con 0x7f32e0100240 2024-04-24T08:24:23.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.713+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e0192630 con 0x7f32e0100240 2024-04-24T08:24:23.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.714+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32d0005dc0 con 0x7f32e0100240 2024-04-24T08:24:23.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.714+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e01a1cc0 con 0x7f32e0100240 2024-04-24T08:24:23.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.714+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32d00177d0 con 0x7f32e0100240 2024-04-24T08:24:23.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.715+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c0005320 con 0x7f32e0100240 2024-04-24T08:24:23.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.715+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 29) v1 ==== 46641+0+0 (secure 0 0 0) 0x7f32d0017930 con 0x7f32e0100240 2024-04-24T08:24:23.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.716+0000 7f32d6a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 0x7f32c403bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:23.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.716+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f32d000e760 con 0x7f32e0100240 2024-04-24T08:24:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.720+0000 7f32dd400700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 0x7f32c403bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.721+0000 7f32dd400700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 0x7f32c403bcf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32e0198da0 tx=0x7f32cc009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:23.727+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f32d0027020 con 0x7f32e0100240 2024-04-24T08:24:23.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:23 smithi149 ceph-mon[31282]: pgmap v4: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:23.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:23 smithi149 ceph-mon[31282]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:24.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.134+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32c0005cc0 con 0x7f32e0100240 2024-04-24T08:24:24.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.135+0000 7f32d6a00700 1 -- 172.21.15.90:0/2008963285 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f32d000e3e0 con 0x7f32e0100240 2024-04-24T08:24:24.136 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:24:24.136 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:24:24.136 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:24:24.137 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:24:24.138 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:24:24.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 msgr2=0x7f32c403bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 0x7f32c403bcf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32e0198da0 tx=0x7f32cc009260 comp rx=0 tx=0).stop 2024-04-24T08:24:24.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 msgr2=0x7f32e0197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32d000f690 tx=0x7f32d000f770 comp rx=0 tx=0).stop 2024-04-24T08:24:24.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 shutdown_connections 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f32c4039840 0x7f32c403bcf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32e0100240 0x7f32e0197c70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 --2- 172.21.15.90:0/2008963285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e0101440 0x7f32e01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 >> 172.21.15.90:0/2008963285 conn(0x7f32e00fb7f0 msgr2=0x7f32e0102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 shutdown_connections 2024-04-24T08:24:24.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.138+0000 7f32e4a00700 1 -- 172.21.15.90:0/2008963285 wait complete. 2024-04-24T08:24:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 -- 172.21.15.90:0/1448663821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878072380 msgr2=0x7f0878107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 --2- 172.21.15.90:0/1448663821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878072380 0x7f0878107d50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0864009910 tx=0x7f0864009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 -- 172.21.15.90:0/1448663821 shutdown_connections 2024-04-24T08:24:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 --2- 172.21.15.90:0/1448663821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878072380 0x7f0878107d50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 --2- 172.21.15.90:0/1448663821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878071a30 0x7f0878071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 -- 172.21.15.90:0/1448663821 >> 172.21.15.90:0/1448663821 conn(0x7f087806d000 msgr2=0x7f087806f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 -- 172.21.15.90:0/1448663821 shutdown_connections 2024-04-24T08:24:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.306+0000 7f087e200700 1 -- 172.21.15.90:0/1448663821 wait complete. 2024-04-24T08:24:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.307+0000 7f087e200700 1 Processor -- start 2024-04-24T08:24:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087e200700 1 -- start start 2024-04-24T08:24:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878072380 0x7f08781a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a0d10 con 0x7f0878071a30 2024-04-24T08:24:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a0e50 con 0x7f0878072380 2024-04-24T08:24:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55472/0 (socket says 172.21.15.90:55472) 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f087ce00700 1 -- 172.21.15.90:0/1471801570 learned_addr learned my addr 172.21.15.90:0/1471801570 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.308+0000 7f0877e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878072380 0x7f08781a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.309+0000 7f087ce00700 1 -- 172.21.15.90:0/1471801570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878072380 msgr2=0x7f08781a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.309+0000 7f087ce00700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878072380 0x7f08781a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.309+0000 7f087ce00700 1 -- 172.21.15.90:0/1471801570 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640095f0 con 0x7f0878071a30 2024-04-24T08:24:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.309+0000 7f087ce00700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c930 tx=0x7f086c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.310+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086c007ad0 con 0x7f0878071a30 2024-04-24T08:24:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.310+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781a5910 con 0x7f0878071a30 2024-04-24T08:24:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.310+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08781a5e30 con 0x7f0878071a30 2024-04-24T08:24:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.310+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f086c00ce80 con 0x7f0878071a30 2024-04-24T08:24:24.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.310+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086c018630 con 0x7f0878071a30 2024-04-24T08:24:24.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.311+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0858005320 con 0x7f0878071a30 2024-04-24T08:24:24.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.311+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 29) v1 ==== 46641+0+0 (secure 0 0 0) 0x7f086c018790 con 0x7f0878071a30 2024-04-24T08:24:24.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.312+0000 7f0875600700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 0x7f086003bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:24.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.312+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f086c04dcb0 con 0x7f0878071a30 2024-04-24T08:24:24.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.312+0000 7f0877e00700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 0x7f086003bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:24.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.313+0000 7f0877e00700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 0x7f086003bcf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0864006010 tx=0x7f086400b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:24.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.318+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f086c0164e0 con 0x7f0878071a30 2024-04-24T08:24:24.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.655+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0858000bf0 con 0x7f0860039840 2024-04-24T08:24:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.657+0000 7f0875600700 1 -- 172.21.15.90:0/1471801570 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f0858000bf0 con 0x7f0860039840 2024-04-24T08:24:24.657 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:24:24.658 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:24:24.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:24 smithi090 ceph-mon[25016]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:24.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:24 smithi090 ceph-mon[25016]: mgrmap e29: smithi090.jsrfxo(active, since 3s) 2024-04-24T08:24:24.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:24 smithi090 ceph-mon[25016]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:24.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:24 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2008963285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:24:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.659+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 msgr2=0x7f086003bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.659+0000 7f087e200700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 0x7f086003bcf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0864006010 tx=0x7f086400b560 comp rx=0 tx=0).stop 2024-04-24T08:24:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.659+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 msgr2=0x7f08781a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.659+0000 7f087e200700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c930 tx=0x7f086c00cc40 comp rx=0 tx=0).stop 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 shutdown_connections 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0860039840 0x7f086003bcf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0878071a30 0x7f08781a0240 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 --2- 172.21.15.90:0/1471801570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0878072380 0x7f08781a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 >> 172.21.15.90:0/1471801570 conn(0x7f087806d000 msgr2=0x7f087810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.660+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 shutdown_connections 2024-04-24T08:24:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:24.661+0000 7f087e200700 1 -- 172.21.15.90:0/1471801570 wait complete. 2024-04-24T08:24:24.685 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:24 smithi149 ceph-mon[31282]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:24.685 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:24 smithi149 ceph-mon[31282]: mgrmap e29: smithi090.jsrfxo(active, since 3s) 2024-04-24T08:24:24.686 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:24 smithi149 ceph-mon[31282]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:24.686 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:24 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2008963285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:24:25.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:25 smithi090 ceph-mon[25016]: pgmap v5: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:25.756 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:25 smithi149 ceph-mon[31282]: pgmap v5: 65 pgs: 65 active+clean; 185 MiB data, 776 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:24:26.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:26 smithi090 ceph-mon[25016]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:26.435 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:26 smithi090 ceph-mon[25016]: mgrmap e30: smithi090.jsrfxo(active, since 5s) 2024-04-24T08:24:26.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:26 smithi149 ceph-mon[31282]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:26.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:26 smithi149 ceph-mon[31282]: mgrmap e30: smithi090.jsrfxo(active, since 5s) 2024-04-24T08:24:27.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:27 smithi090 ceph-mon[25016]: pgmap v6: 65 pgs: 65 active+clean; 186 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 258 KiB/s wr, 62 op/s 2024-04-24T08:24:27.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:27 smithi149 ceph-mon[31282]: pgmap v6: 65 pgs: 65 active+clean; 186 MiB data, 793 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 258 KiB/s wr, 62 op/s 2024-04-24T08:24:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:29 smithi149 ceph-mon[31282]: pgmap v7: 65 pgs: 65 active+clean; 199 MiB data, 813 MiB used, 536 GiB / 536 GiB avail; 617 KiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-04-24T08:24:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:29 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:29 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:29 smithi090 ceph-mon[25016]: pgmap v7: 65 pgs: 65 active+clean; 199 MiB data, 813 MiB used, 536 GiB / 536 GiB avail; 617 KiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-04-24T08:24:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:29 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:29 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:29.836 INFO:tasks.workunit.client.1.smithi149.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T08:24:29.847 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T08:24:29.848 INFO:tasks.workunit.client.1.smithi149.stderr:+ make 2024-04-24T08:24:30.477 INFO:tasks.workunit.client.1.smithi149.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T08:24:31.352 INFO:tasks.workunit.client.1.smithi149.stderr:++ readlink -f fsstress 2024-04-24T08:24:31.356 INFO:tasks.workunit.client.1.smithi149.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T08:24:31.356 INFO:tasks.workunit.client.1.smithi149.stderr:+ popd 2024-04-24T08:24:31.360 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T08:24:31.360 INFO:tasks.workunit.client.1.smithi149.stderr:+ popd 2024-04-24T08:24:31.361 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-24T08:24:31.362 INFO:tasks.workunit.client.1.smithi149.stderr:++ mktemp -d -p . 2024-04-24T08:24:31.367 INFO:tasks.workunit.client.1.smithi149.stderr:+ T=./tmp.o5tDYj6NmN 2024-04-24T08:24:31.367 INFO:tasks.workunit.client.1.smithi149.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.o5tDYj6NmN -l 1 -n 1000 -p 10 -v 2024-04-24T08:24:31.374 INFO:tasks.workunit.client.1.smithi149.stdout:seed = 1714256360 2024-04-24T08:24:31.384 INFO:tasks.workunit.client.1.smithi149.stdout:0/0: mknod c0 0 2024-04-24T08:24:31.384 INFO:tasks.workunit.client.1.smithi149.stdout:7/0: dwrite - no filename 2024-04-24T08:24:31.389 INFO:tasks.workunit.client.1.smithi149.stdout:0/1: rename c0 to c1 0 2024-04-24T08:24:31.389 INFO:tasks.workunit.client.1.smithi149.stdout:0/2: write - no filename 2024-04-24T08:24:31.390 INFO:tasks.workunit.client.1.smithi149.stdout:1/0: link - no file 2024-04-24T08:24:31.390 INFO:tasks.workunit.client.1.smithi149.stdout:1/1: rename - no filename 2024-04-24T08:24:31.390 INFO:tasks.workunit.client.1.smithi149.stdout:1/2: dwrite - no filename 2024-04-24T08:24:31.390 INFO:tasks.workunit.client.1.smithi149.stdout:1/3: unlink - no file 2024-04-24T08:24:31.390 INFO:tasks.workunit.client.1.smithi149.stdout:7/1: creat f0 x:0 0 0 2024-04-24T08:24:31.392 INFO:tasks.workunit.client.1.smithi149.stdout:2/0: chown . 44784 1 2024-04-24T08:24:31.393 INFO:tasks.workunit.client.1.smithi149.stdout:2/1: rename - no filename 2024-04-24T08:24:31.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/3: mkdir d2 0 2024-04-24T08:24:31.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/4: dread - no filename 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/2: chown . 3653246 1 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/3: rename - no filename 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/4: unlink - no file 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/5: chown . 51 1 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/6: fsync - no filename 2024-04-24T08:24:31.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/7: rmdir - no directory 2024-04-24T08:24:31.395 INFO:tasks.workunit.client.1.smithi149.stdout:7/2: rename f0 to f1 0 2024-04-24T08:24:31.396 INFO:tasks.workunit.client.1.smithi149.stdout:7/3: dread - f1 zero size 2024-04-24T08:24:31.399 INFO:tasks.workunit.client.1.smithi149.stdout:1/4: mkdir d0 0 2024-04-24T08:24:31.400 INFO:tasks.workunit.client.1.smithi149.stdout:1/5: chown d0 214300 1 2024-04-24T08:24:31.401 INFO:tasks.workunit.client.1.smithi149.stdout:7/4: mknod c2 0 2024-04-24T08:24:31.405 INFO:tasks.workunit.client.1.smithi149.stdout:2/8: mkdir d0 0 2024-04-24T08:24:31.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/0: fsync - no filename 2024-04-24T08:24:31.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/1: dread - no filename 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/9: chown d0 5 1 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/10: readlink - no filename 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/11: write - no filename 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/12: readlink - no filename 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/13: dwrite - no filename 2024-04-24T08:24:31.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/14: readlink - no filename 2024-04-24T08:24:31.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/5: symlink d2/l3 0 2024-04-24T08:24:31.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/6: dread - no filename 2024-04-24T08:24:31.409 INFO:tasks.workunit.client.1.smithi149.stdout:3/0: mkdir d0 0 2024-04-24T08:24:31.410 INFO:tasks.workunit.client.1.smithi149.stdout:3/1: chown d0 2495 1 2024-04-24T08:24:31.411 INFO:tasks.workunit.client.1.smithi149.stdout:3/2: link - no file 2024-04-24T08:24:31.411 INFO:tasks.workunit.client.1.smithi149.stdout:3/3: write - no filename 2024-04-24T08:24:31.412 INFO:tasks.workunit.client.1.smithi149.stdout:5/0: truncate - no filename 2024-04-24T08:24:31.412 INFO:tasks.workunit.client.1.smithi149.stdout:5/1: dwrite - no filename 2024-04-24T08:24:31.412 INFO:tasks.workunit.client.1.smithi149.stdout:5/2: write - no filename 2024-04-24T08:24:31.413 INFO:tasks.workunit.client.1.smithi149.stdout:1/6: getdents d0 0 2024-04-24T08:24:31.415 INFO:tasks.workunit.client.1.smithi149.stdout:7/5: link f1 f3 0 2024-04-24T08:24:31.418 INFO:tasks.workunit.client.1.smithi149.stdout:0/7: creat d2/f4 x:0 0 0 2024-04-24T08:24:31.419 INFO:tasks.workunit.client.1.smithi149.stdout:4/2: creat f0 x:0 0 0 2024-04-24T08:24:31.422 INFO:tasks.workunit.client.1.smithi149.stdout:2/15: creat d0/f1 x:0 0 0 2024-04-24T08:24:31.425 INFO:tasks.workunit.client.1.smithi149.stdout:6/0: read - no filename 2024-04-24T08:24:31.429 INFO:tasks.workunit.client.1.smithi149.stdout:7/6: mknod c4 0 2024-04-24T08:24:31.433 INFO:tasks.workunit.client.1.smithi149.stdout:3/4: creat d0/f1 x:0 0 0 2024-04-24T08:24:31.433 INFO:tasks.workunit.client.1.smithi149.stdout:4/3: creat f1 x:0 0 0 2024-04-24T08:24:31.434 INFO:tasks.workunit.client.1.smithi149.stdout:2/16: creat d0/f2 x:0 0 0 2024-04-24T08:24:31.435 INFO:tasks.workunit.client.1.smithi149.stdout:5/3: creat f0 x:0 0 0 2024-04-24T08:24:31.435 INFO:tasks.workunit.client.1.smithi149.stdout:5/4: readlink - no filename 2024-04-24T08:24:31.436 INFO:tasks.workunit.client.1.smithi149.stdout:4/4: fdatasync f1 0 2024-04-24T08:24:31.436 INFO:tasks.workunit.client.1.smithi149.stdout:4/5: readlink - no filename 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/7: mkdir d0/d1 0 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/8: truncate - no filename 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/9: link - no file 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/10: dwrite - no filename 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/11: write - no filename 2024-04-24T08:24:31.441 INFO:tasks.workunit.client.1.smithi149.stdout:5/5: truncate f0 540220 0 2024-04-24T08:24:31.442 INFO:tasks.workunit.client.1.smithi149.stdout:2/17: mknod d0/c3 0 2024-04-24T08:24:31.442 INFO:tasks.workunit.client.1.smithi149.stdout:8/0: chown . 988 1 2024-04-24T08:24:31.443 INFO:tasks.workunit.client.1.smithi149.stdout:8/1: unlink - no file 2024-04-24T08:24:31.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/5: rename d0/f1 to d0/f2 0 2024-04-24T08:24:31.449 INFO:tasks.workunit.client.1.smithi149.stdout:6/1: mkdir d0 0 2024-04-24T08:24:31.452 INFO:tasks.workunit.client.1.smithi149.stdout:9/0: write - no filename 2024-04-24T08:24:31.460 INFO:tasks.workunit.client.1.smithi149.stdout:5/6: rename f0 to f1 0 2024-04-24T08:24:31.460 INFO:tasks.workunit.client.1.smithi149.stdout:0/8: dwrite d2/f4 [0,4194304] 0 2024-04-24T08:24:31.463 INFO:tasks.workunit.client.1.smithi149.stdout:0/9: chown d2 56 1 2024-04-24T08:24:31.465 INFO:tasks.workunit.client.1.smithi149.stdout:3/6: write d0/f2 [460172,7371] 0 2024-04-24T08:24:31.465 INFO:tasks.workunit.client.1.smithi149.stdout:7/7: link c2 c5 0 2024-04-24T08:24:31.470 INFO:tasks.workunit.client.1.smithi149.stdout:8/2: symlink l0 0 2024-04-24T08:24:31.470 INFO:tasks.workunit.client.1.smithi149.stdout:8/3: dwrite - no filename 2024-04-24T08:24:31.470 INFO:tasks.workunit.client.1.smithi149.stdout:8/4: write - no filename 2024-04-24T08:24:31.470 INFO:tasks.workunit.client.1.smithi149.stdout:8/5: dwrite - no filename 2024-04-24T08:24:31.472 INFO:tasks.workunit.client.1.smithi149.stdout:7/8: read - f1 zero size 2024-04-24T08:24:31.473 INFO:tasks.workunit.client.1.smithi149.stdout:0/10: write d2/f4 [2745219,46547] 0 2024-04-24T08:24:31.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/18: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:24:31.485 INFO:tasks.workunit.client.1.smithi149.stdout:6/2: creat d0/f1 x:0 0 0 2024-04-24T08:24:31.494 INFO:tasks.workunit.client.1.smithi149.stdout:8/6: mkdir d1 0 2024-04-24T08:24:31.494 INFO:tasks.workunit.client.1.smithi149.stdout:3/7: creat d0/f3 x:0 0 0 2024-04-24T08:24:31.499 INFO:tasks.workunit.client.1.smithi149.stdout:3/8: truncate d0/f2 874211 0 2024-04-24T08:24:31.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/1: getdents . 0 2024-04-24T08:24:31.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/2: readlink - no filename 2024-04-24T08:24:31.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/3: dwrite - no filename 2024-04-24T08:24:31.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/4: readlink - no filename 2024-04-24T08:24:31.507 INFO:tasks.workunit.client.1.smithi149.stdout:1/12: rmdir d0/d1 0 2024-04-24T08:24:31.507 INFO:tasks.workunit.client.1.smithi149.stdout:1/13: readlink - no filename 2024-04-24T08:24:31.507 INFO:tasks.workunit.client.1.smithi149.stdout:1/14: write - no filename 2024-04-24T08:24:31.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:31 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:31 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:31 smithi149 ceph-mon[31282]: pgmap v8: 65 pgs: 65 active+clean; 199 MiB data, 813 MiB used, 536 GiB / 536 GiB avail; 480 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-24T08:24:31.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:31 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.517 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:31 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.527 INFO:tasks.workunit.client.1.smithi149.stdout:0/11: creat d2/f5 x:0 0 0 2024-04-24T08:24:31.534 INFO:tasks.workunit.client.1.smithi149.stdout:7/9: write f1 [134003,31361] 0 2024-04-24T08:24:31.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/15: symlink d0/l2 0 2024-04-24T08:24:31.537 INFO:tasks.workunit.client.1.smithi149.stdout:1/16: truncate - no filename 2024-04-24T08:24:31.537 INFO:tasks.workunit.client.1.smithi149.stdout:0/12: readlink d2/l3 0 2024-04-24T08:24:31.540 INFO:tasks.workunit.client.1.smithi149.stdout:5/7: dwrite f1 [0,4194304] 0 2024-04-24T08:24:31.549 INFO:tasks.workunit.client.1.smithi149.stdout:8/7: creat d1/f2 x:0 0 0 2024-04-24T08:24:31.550 INFO:tasks.workunit.client.1.smithi149.stdout:9/5: mknod c0 0 2024-04-24T08:24:31.552 INFO:tasks.workunit.client.1.smithi149.stdout:2/19: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:31.560 INFO:tasks.workunit.client.1.smithi149.stdout:6/3: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:24:31.570 INFO:tasks.workunit.client.1.smithi149.stdout:7/10: rename c2 to c6 0 2024-04-24T08:24:31.571 INFO:tasks.workunit.client.1.smithi149.stdout:9/6: mkdir d1 0 2024-04-24T08:24:31.571 INFO:tasks.workunit.client.1.smithi149.stdout:9/7: write - no filename 2024-04-24T08:24:31.571 INFO:tasks.workunit.client.1.smithi149.stdout:9/8: dread - no filename 2024-04-24T08:24:31.571 INFO:tasks.workunit.client.1.smithi149.stdout:9/9: write - no filename 2024-04-24T08:24:31.589 INFO:tasks.workunit.client.1.smithi149.stdout:7/11: mknod c7 0 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/17: link d0/l2 d0/l3 0 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/18: dread - no filename 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/19: dread - no filename 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/20: dwrite - no filename 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/21: dread - no filename 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/22: dwrite - no filename 2024-04-24T08:24:31.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/23: dread - no filename 2024-04-24T08:24:31.597 INFO:tasks.workunit.client.1.smithi149.stdout:1/24: fsync - no filename 2024-04-24T08:24:31.599 INFO:tasks.workunit.client.1.smithi149.stdout:2/20: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:24:31.608 INFO:tasks.workunit.client.1.smithi149.stdout:9/10: mkdir d1/d2 0 2024-04-24T08:24:31.609 INFO:tasks.workunit.client.1.smithi149.stdout:9/11: dwrite - no filename 2024-04-24T08:24:31.609 INFO:tasks.workunit.client.1.smithi149.stdout:2/21: write d0/f2 [2880434,24304] 0 2024-04-24T08:24:31.610 INFO:tasks.workunit.client.1.smithi149.stdout:9/12: chown d1 0 1 2024-04-24T08:24:31.610 INFO:tasks.workunit.client.1.smithi149.stdout:9/13: write - no filename 2024-04-24T08:24:31.611 INFO:tasks.workunit.client.1.smithi149.stdout:1/25: creat d0/f4 x:0 0 0 2024-04-24T08:24:31.615 INFO:tasks.workunit.client.1.smithi149.stdout:8/8: dwrite d1/f2 [0,4194304] 0 2024-04-24T08:24:31.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/22: chown d0/c3 16329422 1 2024-04-24T08:24:31.618 INFO:tasks.workunit.client.1.smithi149.stdout:9/14: rename d1 to d1/d3 22 2024-04-24T08:24:31.622 INFO:tasks.workunit.client.1.smithi149.stdout:1/26: mkdir d0/d5 0 2024-04-24T08:24:31.623 INFO:tasks.workunit.client.1.smithi149.stdout:2/23: creat d0/f4 x:0 0 0 2024-04-24T08:24:31.626 INFO:tasks.workunit.client.1.smithi149.stdout:9/15: creat d1/f4 x:0 0 0 2024-04-24T08:24:31.629 INFO:tasks.workunit.client.1.smithi149.stdout:9/16: chown c0 0 1 2024-04-24T08:24:31.629 INFO:tasks.workunit.client.1.smithi149.stdout:1/27: write d0/f4 [375030,68062] 0 2024-04-24T08:24:31.641 INFO:tasks.workunit.client.1.smithi149.stdout:2/24: creat d0/f5 x:0 0 0 2024-04-24T08:24:31.649 INFO:tasks.workunit.client.1.smithi149.stdout:2/25: mkdir d0/d6 0 2024-04-24T08:24:31.655 INFO:tasks.workunit.client.1.smithi149.stdout:2/26: truncate d0/f1 5048956 0 2024-04-24T08:24:31.655 INFO:tasks.workunit.client.1.smithi149.stdout:2/27: readlink - no filename 2024-04-24T08:24:31.666 INFO:tasks.workunit.client.1.smithi149.stdout:9/17: dwrite d1/f4 [0,4194304] 0 2024-04-24T08:24:31.671 INFO:tasks.workunit.client.1.smithi149.stdout:1/28: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:31.671 INFO:tasks.workunit.client.1.smithi149.stdout:9/18: write d1/f4 [3640186,95832] 0 2024-04-24T08:24:31.676 INFO:tasks.workunit.client.1.smithi149.stdout:7/12: fsync f1 0 2024-04-24T08:24:31.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:31 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:31 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:31 smithi090 ceph-mon[25016]: pgmap v8: 65 pgs: 65 active+clean; 199 MiB data, 813 MiB used, 536 GiB / 536 GiB avail; 480 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-24T08:24:31.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:31 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:31 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:31.682 INFO:tasks.workunit.client.1.smithi149.stdout:0/13: fsync d2/f4 0 2024-04-24T08:24:31.683 INFO:tasks.workunit.client.1.smithi149.stdout:0/14: read - d2/f5 zero size 2024-04-24T08:24:31.685 INFO:tasks.workunit.client.1.smithi149.stdout:9/19: mknod d1/d2/c5 0 2024-04-24T08:24:31.692 INFO:tasks.workunit.client.1.smithi149.stdout:4/6: getdents . 0 2024-04-24T08:24:31.692 INFO:tasks.workunit.client.1.smithi149.stdout:9/20: creat d1/d2/f6 x:0 0 0 2024-04-24T08:24:31.698 INFO:tasks.workunit.client.1.smithi149.stdout:9/21: write d1/f4 [3035424,100955] 0 2024-04-24T08:24:31.699 INFO:tasks.workunit.client.1.smithi149.stdout:9/22: chown d1 106011 1 2024-04-24T08:24:31.700 INFO:tasks.workunit.client.1.smithi149.stdout:4/7: unlink f0 0 2024-04-24T08:24:31.701 INFO:tasks.workunit.client.1.smithi149.stdout:4/8: truncate f1 769395 0 2024-04-24T08:24:31.704 INFO:tasks.workunit.client.1.smithi149.stdout:4/9: dread f1 [0,4194304] 0 2024-04-24T08:24:31.705 INFO:tasks.workunit.client.1.smithi149.stdout:9/23: mkdir d1/d2/d7 0 2024-04-24T08:24:31.711 INFO:tasks.workunit.client.1.smithi149.stdout:9/24: creat d1/d2/d7/f8 x:0 0 0 2024-04-24T08:24:31.714 INFO:tasks.workunit.client.1.smithi149.stdout:9/25: unlink d1/d2/c5 0 2024-04-24T08:24:31.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/26: truncate d1/d2/f6 244181 0 2024-04-24T08:24:31.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/9: getdents d0 0 2024-04-24T08:24:31.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/10: unlink d0/f2 0 2024-04-24T08:24:31.731 INFO:tasks.workunit.client.1.smithi149.stdout:0/15: truncate d2/f4 654721 0 2024-04-24T08:24:31.732 INFO:tasks.workunit.client.1.smithi149.stdout:7/13: dwrite f1 [0,4194304] 0 2024-04-24T08:24:31.736 INFO:tasks.workunit.client.1.smithi149.stdout:5/8: truncate f1 3694174 0 2024-04-24T08:24:31.754 INFO:tasks.workunit.client.1.smithi149.stdout:9/27: dwrite d1/d2/d7/f8 [0,4194304] 0 2024-04-24T08:24:31.754 INFO:tasks.workunit.client.1.smithi149.stdout:8/9: truncate d1/f2 2393222 0 2024-04-24T08:24:31.763 INFO:tasks.workunit.client.1.smithi149.stdout:1/29: write d0/f4 [5211824,128022] 0 2024-04-24T08:24:31.767 INFO:tasks.workunit.client.1.smithi149.stdout:7/14: dread f1 [0,4194304] 0 2024-04-24T08:24:31.767 INFO:tasks.workunit.client.1.smithi149.stdout:1/30: chown d0/f4 13513079 1 2024-04-24T08:24:31.772 INFO:tasks.workunit.client.1.smithi149.stdout:9/28: mknod d1/d2/c9 0 2024-04-24T08:24:31.776 INFO:tasks.workunit.client.1.smithi149.stdout:3/11: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:31.779 INFO:tasks.workunit.client.1.smithi149.stdout:1/31: fsync d0/f4 0 2024-04-24T08:24:31.788 INFO:tasks.workunit.client.1.smithi149.stdout:3/12: link d0/f3 d0/f4 0 2024-04-24T08:24:31.792 INFO:tasks.workunit.client.1.smithi149.stdout:1/32: mknod d0/d5/c6 0 2024-04-24T08:24:31.800 INFO:tasks.workunit.client.1.smithi149.stdout:9/29: dread d1/f4 [0,4194304] 0 2024-04-24T08:24:31.802 INFO:tasks.workunit.client.1.smithi149.stdout:3/13: mkdir d0/d5 0 2024-04-24T08:24:31.803 INFO:tasks.workunit.client.1.smithi149.stdout:7/15: dwrite f3 [0,4194304] 0 2024-04-24T08:24:31.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/16: rmdir d2 39 2024-04-24T08:24:31.808 INFO:tasks.workunit.client.1.smithi149.stdout:3/14: creat d0/f6 x:0 0 0 2024-04-24T08:24:31.809 INFO:tasks.workunit.client.1.smithi149.stdout:1/33: chown d0/l2 31516027 1 2024-04-24T08:24:31.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/9: fsync f1 0 2024-04-24T08:24:31.811 INFO:tasks.workunit.client.1.smithi149.stdout:3/15: truncate d0/f6 690455 0 2024-04-24T08:24:31.817 INFO:tasks.workunit.client.1.smithi149.stdout:8/10: write d1/f2 [1995596,72672] 0 2024-04-24T08:24:31.818 INFO:tasks.workunit.client.1.smithi149.stdout:8/11: chown d1 31 1 2024-04-24T08:24:31.825 INFO:tasks.workunit.client.1.smithi149.stdout:3/16: mkdir d0/d7 0 2024-04-24T08:24:31.829 INFO:tasks.workunit.client.1.smithi149.stdout:9/30: dread d1/d2/d7/f8 [0,4194304] 0 2024-04-24T08:24:31.832 INFO:tasks.workunit.client.1.smithi149.stdout:8/12: link d1/f2 d1/f3 0 2024-04-24T08:24:31.836 INFO:tasks.workunit.client.1.smithi149.stdout:1/34: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:31.844 INFO:tasks.workunit.client.1.smithi149.stdout:5/10: dwrite f1 [0,4194304] 0 2024-04-24T08:24:31.847 INFO:tasks.workunit.client.1.smithi149.stdout:9/31: rename d1/d2/d7 to d1/da 0 2024-04-24T08:24:31.850 INFO:tasks.workunit.client.1.smithi149.stdout:1/35: mknod d0/c7 0 2024-04-24T08:24:31.856 INFO:tasks.workunit.client.1.smithi149.stdout:5/11: mknod c2 0 2024-04-24T08:24:31.858 INFO:tasks.workunit.client.1.smithi149.stdout:1/36: write d0/f4 [703420,4692] 0 2024-04-24T08:24:31.858 INFO:tasks.workunit.client.1.smithi149.stdout:9/32: symlink d1/d2/lb 0 2024-04-24T08:24:31.863 INFO:tasks.workunit.client.1.smithi149.stdout:1/37: chown d0/l2 7 1 2024-04-24T08:24:31.879 INFO:tasks.workunit.client.1.smithi149.stdout:9/33: write d1/d2/f6 [664464,10266] 0 2024-04-24T08:24:31.882 INFO:tasks.workunit.client.1.smithi149.stdout:3/17: dwrite d0/f3 [4194304,4194304] 0 2024-04-24T08:24:31.888 INFO:tasks.workunit.client.1.smithi149.stdout:1/38: creat d0/d5/f8 x:0 0 0 2024-04-24T08:24:31.891 INFO:tasks.workunit.client.1.smithi149.stdout:1/39: dread - d0/d5/f8 zero size 2024-04-24T08:24:31.892 INFO:tasks.workunit.client.1.smithi149.stdout:1/40: chown d0/d5 61 1 2024-04-24T08:24:31.893 INFO:tasks.workunit.client.1.smithi149.stdout:5/12: dread f1 [0,4194304] 0 2024-04-24T08:24:31.894 INFO:tasks.workunit.client.1.smithi149.stdout:1/41: dread - d0/d5/f8 zero size 2024-04-24T08:24:31.896 INFO:tasks.workunit.client.1.smithi149.stdout:3/18: creat d0/d5/f8 x:0 0 0 2024-04-24T08:24:31.898 INFO:tasks.workunit.client.1.smithi149.stdout:5/13: rename c2 to c3 0 2024-04-24T08:24:31.900 INFO:tasks.workunit.client.1.smithi149.stdout:3/19: chown d0/d5/f8 9628257 1 2024-04-24T08:24:31.901 INFO:tasks.workunit.client.1.smithi149.stdout:1/42: creat d0/d5/f9 x:0 0 0 2024-04-24T08:24:31.919 INFO:tasks.workunit.client.1.smithi149.stdout:1/43: rmdir d0 39 2024-04-24T08:24:31.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/34: dwrite d1/f4 [4194304,4194304] 0 2024-04-24T08:24:31.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/14: dread f1 [0,4194304] 0 2024-04-24T08:24:31.931 INFO:tasks.workunit.client.1.smithi149.stdout:1/44: dread - d0/d5/f9 zero size 2024-04-24T08:24:31.935 INFO:tasks.workunit.client.1.smithi149.stdout:9/35: creat d1/da/fc x:0 0 0 2024-04-24T08:24:31.941 INFO:tasks.workunit.client.1.smithi149.stdout:9/36: chown d1/d2/f6 58731279 1 2024-04-24T08:24:31.941 INFO:tasks.workunit.client.1.smithi149.stdout:3/20: dwrite d0/f6 [0,4194304] 0 2024-04-24T08:24:31.946 INFO:tasks.workunit.client.1.smithi149.stdout:9/37: mkdir d1/da/dd 0 2024-04-24T08:24:31.947 INFO:tasks.workunit.client.1.smithi149.stdout:1/45: getdents d0/d5 0 2024-04-24T08:24:31.950 INFO:tasks.workunit.client.1.smithi149.stdout:9/38: chown d1/d2/f6 1898701919 1 2024-04-24T08:24:31.951 INFO:tasks.workunit.client.1.smithi149.stdout:1/46: mkdir d0/da 0 2024-04-24T08:24:31.953 INFO:tasks.workunit.client.1.smithi149.stdout:9/39: mkdir d1/de 0 2024-04-24T08:24:31.954 INFO:tasks.workunit.client.1.smithi149.stdout:1/47: write d0/d5/f8 [259347,126304] 0 2024-04-24T08:24:31.963 INFO:tasks.workunit.client.1.smithi149.stdout:1/48: dread - d0/d5/f9 zero size 2024-04-24T08:24:31.963 INFO:tasks.workunit.client.1.smithi149.stdout:5/15: dwrite f1 [0,4194304] 0 2024-04-24T08:24:31.975 INFO:tasks.workunit.client.1.smithi149.stdout:9/40: dread d1/f4 [4194304,4194304] 0 2024-04-24T08:24:31.976 INFO:tasks.workunit.client.1.smithi149.stdout:9/41: chown d1 0 1 2024-04-24T08:24:31.978 INFO:tasks.workunit.client.1.smithi149.stdout:9/42: chown d1/d2/c9 29 1 2024-04-24T08:24:31.979 INFO:tasks.workunit.client.1.smithi149.stdout:5/16: mknod c4 0 2024-04-24T08:24:31.982 INFO:tasks.workunit.client.1.smithi149.stdout:5/17: creat f5 x:0 0 0 2024-04-24T08:24:31.984 INFO:tasks.workunit.client.1.smithi149.stdout:9/43: link d1/da/f8 d1/de/ff 0 2024-04-24T08:24:31.986 INFO:tasks.workunit.client.1.smithi149.stdout:5/18: symlink l6 0 2024-04-24T08:24:31.987 INFO:tasks.workunit.client.1.smithi149.stdout:5/19: write f5 [216969,68284] 0 2024-04-24T08:24:31.988 INFO:tasks.workunit.client.1.smithi149.stdout:5/20: chown f1 1293706 1 2024-04-24T08:24:31.990 INFO:tasks.workunit.client.1.smithi149.stdout:9/44: unlink d1/d2/lb 0 2024-04-24T08:24:31.995 INFO:tasks.workunit.client.1.smithi149.stdout:1/49: truncate d0/d5/f8 287515 0 2024-04-24T08:24:31.999 INFO:tasks.workunit.client.1.smithi149.stdout:5/21: fsync f1 0 2024-04-24T08:24:32.003 INFO:tasks.workunit.client.1.smithi149.stdout:1/50: fdatasync d0/d5/f8 0 2024-04-24T08:24:32.009 INFO:tasks.workunit.client.1.smithi149.stdout:1/51: mkdir d0/d5/db 0 2024-04-24T08:24:32.018 INFO:tasks.workunit.client.1.smithi149.stdout:9/45: dwrite d1/da/f8 [0,4194304] 0 2024-04-24T08:24:32.022 INFO:tasks.workunit.client.1.smithi149.stdout:1/52: dread d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:32.032 INFO:tasks.workunit.client.1.smithi149.stdout:9/46: rename c0 to d1/c10 0 2024-04-24T08:24:32.050 INFO:tasks.workunit.client.1.smithi149.stdout:1/53: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:32.052 INFO:tasks.workunit.client.1.smithi149.stdout:9/47: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:32.052 INFO:tasks.workunit.client.1.smithi149.stdout:1/54: write d0/f4 [1358117,80480] 0 2024-04-24T08:24:32.058 INFO:tasks.workunit.client.1.smithi149.stdout:9/48: write d1/f4 [8099515,99065] 0 2024-04-24T08:24:32.061 INFO:tasks.workunit.client.1.smithi149.stdout:1/55: truncate d0/d5/f8 347905 0 2024-04-24T08:24:32.068 INFO:tasks.workunit.client.1.smithi149.stdout:9/49: creat d1/da/dd/f11 x:0 0 0 2024-04-24T08:24:32.074 INFO:tasks.workunit.client.1.smithi149.stdout:9/50: mkdir d1/de/d12 0 2024-04-24T08:24:32.080 INFO:tasks.workunit.client.1.smithi149.stdout:1/56: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:32.084 INFO:tasks.workunit.client.1.smithi149.stdout:1/57: chown d0/c7 55 1 2024-04-24T08:24:32.088 INFO:tasks.workunit.client.1.smithi149.stdout:9/51: getdents d1/d2 0 2024-04-24T08:24:32.095 INFO:tasks.workunit.client.1.smithi149.stdout:1/58: creat d0/da/fc x:0 0 0 2024-04-24T08:24:32.101 INFO:tasks.workunit.client.1.smithi149.stdout:9/52: dread d1/d2/f6 [0,4194304] 0 2024-04-24T08:24:32.108 INFO:tasks.workunit.client.1.smithi149.stdout:9/53: mkdir d1/da/d13 0 2024-04-24T08:24:32.117 INFO:tasks.workunit.client.1.smithi149.stdout:1/59: dwrite d0/da/fc [0,4194304] 0 2024-04-24T08:24:32.125 INFO:tasks.workunit.client.1.smithi149.stdout:1/60: creat d0/fd x:0 0 0 2024-04-24T08:24:32.143 INFO:tasks.workunit.client.1.smithi149.stdout:1/61: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:32.145 INFO:tasks.workunit.client.1.smithi149.stdout:1/62: write d0/d5/f8 [525130,129480] 0 2024-04-24T08:24:32.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/63: link d0/d5/f9 d0/fe 0 2024-04-24T08:24:32.154 INFO:tasks.workunit.client.1.smithi149.stdout:1/64: mkdir d0/d5/df 0 2024-04-24T08:24:32.155 INFO:tasks.workunit.client.1.smithi149.stdout:1/65: chown d0/d5/df 242765990 1 2024-04-24T08:24:32.186 INFO:tasks.workunit.client.1.smithi149.stdout:1/66: dwrite d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:32.317 INFO:tasks.workunit.client.1.smithi149.stdout:1/67: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:32.334 INFO:tasks.workunit.client.1.smithi149.stdout:1/68: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:32.336 INFO:tasks.workunit.client.1.smithi149.stdout:1/69: fdatasync d0/d5/f8 0 2024-04-24T08:24:32.337 INFO:tasks.workunit.client.1.smithi149.stdout:1/70: chown d0/fe 94300800 1 2024-04-24T08:24:32.339 INFO:tasks.workunit.client.1.smithi149.stdout:1/71: mknod d0/d5/c10 0 2024-04-24T08:24:33.049 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:32 smithi149 ceph-mon[31282]: pgmap v9: 65 pgs: 65 active+clean; 215 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-04-24T08:24:33.049 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:32 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:33.049 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:32 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:33.049 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:32 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:24:33.178 INFO:tasks.workunit.client.1.smithi149.stdout:6/4: sync 2024-04-24T08:24:33.204 INFO:tasks.workunit.client.1.smithi149.stdout:6/5: creat d0/f2 x:0 0 0 2024-04-24T08:24:33.240 INFO:tasks.workunit.client.1.smithi149.stdout:2/28: sync 2024-04-24T08:24:33.241 INFO:tasks.workunit.client.1.smithi149.stdout:4/10: sync 2024-04-24T08:24:33.242 INFO:tasks.workunit.client.1.smithi149.stdout:2/29: fdatasync d0/f1 0 2024-04-24T08:24:33.266 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:32 smithi090 ceph-mon[25016]: pgmap v9: 65 pgs: 65 active+clean; 215 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-04-24T08:24:33.266 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:32 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:33.266 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:32 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:33.266 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:32 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:24:33.285 INFO:tasks.workunit.client.1.smithi149.stdout:6/6: rename d0/f2 to d0/f3 0 2024-04-24T08:24:33.289 INFO:tasks.workunit.client.1.smithi149.stdout:6/7: truncate d0/f3 252394 0 2024-04-24T08:24:33.296 INFO:tasks.workunit.client.1.smithi149.stdout:4/11: getdents . 0 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:0/17: sync 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:8/13: sync 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/21: sync 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:7/16: sync 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:5/22: sync 2024-04-24T08:24:33.323 INFO:tasks.workunit.client.1.smithi149.stdout:9/54: sync 2024-04-24T08:24:33.324 INFO:tasks.workunit.client.1.smithi149.stdout:1/72: sync 2024-04-24T08:24:33.326 INFO:tasks.workunit.client.1.smithi149.stdout:7/17: fdatasync f1 0 2024-04-24T08:24:33.326 INFO:tasks.workunit.client.1.smithi149.stdout:1/73: fdatasync d0/da/fc 0 2024-04-24T08:24:33.327 INFO:tasks.workunit.client.1.smithi149.stdout:3/22: chown d0/f6 684113 1 2024-04-24T08:24:33.327 INFO:tasks.workunit.client.1.smithi149.stdout:5/23: creat f7 x:0 0 0 2024-04-24T08:24:33.331 INFO:tasks.workunit.client.1.smithi149.stdout:8/14: write d1/f3 [787645,114165] 0 2024-04-24T08:24:33.336 INFO:tasks.workunit.client.1.smithi149.stdout:7/18: chown c5 2496 1 2024-04-24T08:24:33.337 INFO:tasks.workunit.client.1.smithi149.stdout:0/18: dread d2/f4 [0,4194304] 0 2024-04-24T08:24:33.337 INFO:tasks.workunit.client.1.smithi149.stdout:7/19: stat c4 0 2024-04-24T08:24:33.339 INFO:tasks.workunit.client.1.smithi149.stdout:7/20: truncate f1 4735050 0 2024-04-24T08:24:33.339 INFO:tasks.workunit.client.1.smithi149.stdout:5/24: unlink c4 0 2024-04-24T08:24:33.339 INFO:tasks.workunit.client.1.smithi149.stdout:5/25: rmdir - no directory 2024-04-24T08:24:33.341 INFO:tasks.workunit.client.1.smithi149.stdout:7/21: write f1 [910015,59499] 0 2024-04-24T08:24:33.341 INFO:tasks.workunit.client.1.smithi149.stdout:0/19: dread d2/f4 [0,4194304] 0 2024-04-24T08:24:33.341 INFO:tasks.workunit.client.1.smithi149.stdout:5/26: chown f5 112 1 2024-04-24T08:24:33.343 INFO:tasks.workunit.client.1.smithi149.stdout:1/74: dread d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:33.343 INFO:tasks.workunit.client.1.smithi149.stdout:3/23: mknod d0/d5/c9 0 2024-04-24T08:24:33.344 INFO:tasks.workunit.client.1.smithi149.stdout:5/27: write f1 [3700265,35902] 0 2024-04-24T08:24:33.346 INFO:tasks.workunit.client.1.smithi149.stdout:8/15: mknod d1/c4 0 2024-04-24T08:24:33.347 INFO:tasks.workunit.client.1.smithi149.stdout:8/16: readlink l0 0 2024-04-24T08:24:33.352 INFO:tasks.workunit.client.1.smithi149.stdout:5/28: mknod c8 0 2024-04-24T08:24:33.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/55: getdents d1/da/dd 0 2024-04-24T08:24:33.357 INFO:tasks.workunit.client.1.smithi149.stdout:9/56: chown d1/f4 0 1 2024-04-24T08:24:33.365 INFO:tasks.workunit.client.1.smithi149.stdout:1/75: dread d0/fe [0,4194304] 0 2024-04-24T08:24:33.365 INFO:tasks.workunit.client.1.smithi149.stdout:5/29: rename f1 to f9 0 2024-04-24T08:24:33.367 INFO:tasks.workunit.client.1.smithi149.stdout:3/24: dread d0/f4 [4194304,4194304] 0 2024-04-24T08:24:33.370 INFO:tasks.workunit.client.1.smithi149.stdout:9/57: symlink d1/de/l14 0 2024-04-24T08:24:33.370 INFO:tasks.workunit.client.1.smithi149.stdout:1/76: dread d0/f4 [4194304,4194304] 0 2024-04-24T08:24:33.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/58: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:33.385 INFO:tasks.workunit.client.1.smithi149.stdout:1/77: dwrite d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:33.436 INFO:tasks.workunit.client.1.smithi149.stdout:5/30: symlink la 0 2024-04-24T08:24:33.440 INFO:tasks.workunit.client.1.smithi149.stdout:9/59: write d1/da/f8 [3942039,62635] 0 2024-04-24T08:24:33.444 INFO:tasks.workunit.client.1.smithi149.stdout:5/31: mkdir db 0 2024-04-24T08:24:33.444 INFO:tasks.workunit.client.1.smithi149.stdout:1/78: getdents d0/d5/df 0 2024-04-24T08:24:33.445 INFO:tasks.workunit.client.1.smithi149.stdout:5/32: truncate f5 521159 0 2024-04-24T08:24:33.448 INFO:tasks.workunit.client.1.smithi149.stdout:1/79: creat d0/da/f11 x:0 0 0 2024-04-24T08:24:33.450 INFO:tasks.workunit.client.1.smithi149.stdout:5/33: creat db/fc x:0 0 0 2024-04-24T08:24:33.455 INFO:tasks.workunit.client.1.smithi149.stdout:5/34: mkdir db/dd 0 2024-04-24T08:24:33.466 INFO:tasks.workunit.client.1.smithi149.stdout:3/25: dwrite d0/f4 [8388608,4194304] 0 2024-04-24T08:24:33.466 INFO:tasks.workunit.client.1.smithi149.stdout:5/35: rmdir db/dd 0 2024-04-24T08:24:33.471 INFO:tasks.workunit.client.1.smithi149.stdout:3/26: write d0/f3 [701329,100172] 0 2024-04-24T08:24:33.474 INFO:tasks.workunit.client.1.smithi149.stdout:9/60: dwrite d1/da/dd/f11 [0,4194304] 0 2024-04-24T08:24:33.478 INFO:tasks.workunit.client.1.smithi149.stdout:3/27: dread - d0/d5/f8 zero size 2024-04-24T08:24:33.483 INFO:tasks.workunit.client.1.smithi149.stdout:9/61: chown d1/da/f8 251161 1 2024-04-24T08:24:33.487 INFO:tasks.workunit.client.1.smithi149.stdout:9/62: chown d1/d2/f6 14228874 1 2024-04-24T08:24:33.489 INFO:tasks.workunit.client.1.smithi149.stdout:3/28: mkdir d0/d5/da 0 2024-04-24T08:24:33.494 INFO:tasks.workunit.client.1.smithi149.stdout:9/63: write d1/de/ff [104539,124200] 0 2024-04-24T08:24:33.496 INFO:tasks.workunit.client.1.smithi149.stdout:5/36: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:33.508 INFO:tasks.workunit.client.1.smithi149.stdout:9/64: mkdir d1/d2/d15 0 2024-04-24T08:24:33.512 INFO:tasks.workunit.client.1.smithi149.stdout:3/29: dread d0/f6 [0,4194304] 0 2024-04-24T08:24:33.512 INFO:tasks.workunit.client.1.smithi149.stdout:3/30: readlink - no filename 2024-04-24T08:24:33.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/65: symlink d1/da/dd/l16 0 2024-04-24T08:24:33.526 INFO:tasks.workunit.client.1.smithi149.stdout:5/37: dwrite f7 [0,4194304] 0 2024-04-24T08:24:33.530 INFO:tasks.workunit.client.1.smithi149.stdout:5/38: write f5 [1024813,78544] 0 2024-04-24T08:24:33.534 INFO:tasks.workunit.client.1.smithi149.stdout:9/66: creat d1/de/d12/f17 x:0 0 0 2024-04-24T08:24:33.536 INFO:tasks.workunit.client.1.smithi149.stdout:3/31: dread d0/f4 [8388608,4194304] 0 2024-04-24T08:24:33.538 INFO:tasks.workunit.client.1.smithi149.stdout:9/67: creat d1/de/f18 x:0 0 0 2024-04-24T08:24:33.539 INFO:tasks.workunit.client.1.smithi149.stdout:9/68: fsync d1/da/dd/f11 0 2024-04-24T08:24:33.545 INFO:tasks.workunit.client.1.smithi149.stdout:9/69: creat d1/d2/f19 x:0 0 0 2024-04-24T08:24:33.548 INFO:tasks.workunit.client.1.smithi149.stdout:3/32: dread d0/f4 [4194304,4194304] 0 2024-04-24T08:24:33.549 INFO:tasks.workunit.client.1.smithi149.stdout:9/70: unlink d1/d2/f6 0 2024-04-24T08:24:33.553 INFO:tasks.workunit.client.1.smithi149.stdout:3/33: creat d0/d5/da/fb x:0 0 0 2024-04-24T08:24:33.557 INFO:tasks.workunit.client.1.smithi149.stdout:3/34: creat d0/fc x:0 0 0 2024-04-24T08:24:33.560 INFO:tasks.workunit.client.1.smithi149.stdout:3/35: write d0/d5/da/fb [72006,83812] 0 2024-04-24T08:24:33.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/17: sync 2024-04-24T08:24:33.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/18: chown d1 449428471 1 2024-04-24T08:24:33.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/19: symlink d1/l5 0 2024-04-24T08:24:33.570 INFO:tasks.workunit.client.1.smithi149.stdout:8/20: symlink d1/l6 0 2024-04-24T08:24:33.586 INFO:tasks.workunit.client.1.smithi149.stdout:3/36: dwrite d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:33.591 INFO:tasks.workunit.client.1.smithi149.stdout:3/37: unlink d0/d5/c9 0 2024-04-24T08:24:33.593 INFO:tasks.workunit.client.1.smithi149.stdout:3/38: chown d0/f4 2716 1 2024-04-24T08:24:33.596 INFO:tasks.workunit.client.1.smithi149.stdout:3/39: creat d0/d7/fd x:0 0 0 2024-04-24T08:24:33.597 INFO:tasks.workunit.client.1.smithi149.stdout:3/40: stat d0/d5/f8 0 2024-04-24T08:24:33.597 INFO:tasks.workunit.client.1.smithi149.stdout:3/41: readlink - no filename 2024-04-24T08:24:33.600 INFO:tasks.workunit.client.1.smithi149.stdout:3/42: creat d0/fe x:0 0 0 2024-04-24T08:24:33.602 INFO:tasks.workunit.client.1.smithi149.stdout:3/43: read d0/d5/f8 [621819,122075] 0 2024-04-24T08:24:33.603 INFO:tasks.workunit.client.1.smithi149.stdout:3/44: dread - d0/fe zero size 2024-04-24T08:24:33.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/8: getdents d0 0 2024-04-24T08:24:33.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/9: readlink - no filename 2024-04-24T08:24:33.633 INFO:tasks.workunit.client.1.smithi149.stdout:2/30: truncate d0/f1 1066712 0 2024-04-24T08:24:33.633 INFO:tasks.workunit.client.1.smithi149.stdout:6/10: fdatasync d0/f1 0 2024-04-24T08:24:33.635 INFO:tasks.workunit.client.1.smithi149.stdout:2/31: write d0/f5 [908559,49384] 0 2024-04-24T08:24:33.636 INFO:tasks.workunit.client.1.smithi149.stdout:4/12: write f1 [1171546,505] 0 2024-04-24T08:24:33.637 INFO:tasks.workunit.client.1.smithi149.stdout:2/32: dread - d0/f4 zero size 2024-04-24T08:24:33.638 INFO:tasks.workunit.client.1.smithi149.stdout:2/33: stat d0/f5 0 2024-04-24T08:24:33.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/13: write f1 [871615,71861] 0 2024-04-24T08:24:33.645 INFO:tasks.workunit.client.1.smithi149.stdout:4/14: mkdir d2 0 2024-04-24T08:24:33.647 INFO:tasks.workunit.client.1.smithi149.stdout:4/15: chown f1 225 1 2024-04-24T08:24:33.651 INFO:tasks.workunit.client.1.smithi149.stdout:4/16: symlink d2/l3 0 2024-04-24T08:24:33.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/17: stat d2 0 2024-04-24T08:24:33.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/22: truncate f3 868762 0 2024-04-24T08:24:33.661 INFO:tasks.workunit.client.1.smithi149.stdout:2/34: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:33.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/11: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:33.667 INFO:tasks.workunit.client.1.smithi149.stdout:1/80: truncate d0/d5/f9 3428296 0 2024-04-24T08:24:33.674 INFO:tasks.workunit.client.1.smithi149.stdout:0/20: dwrite d2/f4 [0,4194304] 0 2024-04-24T08:24:33.676 INFO:tasks.workunit.client.1.smithi149.stdout:6/12: creat d0/f4 x:0 0 0 2024-04-24T08:24:33.678 INFO:tasks.workunit.client.1.smithi149.stdout:5/39: rmdir db 39 2024-04-24T08:24:33.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/23: fdatasync f1 0 2024-04-24T08:24:33.680 INFO:tasks.workunit.client.1.smithi149.stdout:5/40: readlink la 0 2024-04-24T08:24:33.680 INFO:tasks.workunit.client.1.smithi149.stdout:1/81: truncate d0/da/f11 927234 0 2024-04-24T08:24:33.681 INFO:tasks.workunit.client.1.smithi149.stdout:0/21: mknod d2/c6 0 2024-04-24T08:24:33.682 INFO:tasks.workunit.client.1.smithi149.stdout:7/24: chown f3 1 1 2024-04-24T08:24:33.684 INFO:tasks.workunit.client.1.smithi149.stdout:6/13: creat d0/f5 x:0 0 0 2024-04-24T08:24:33.689 INFO:tasks.workunit.client.1.smithi149.stdout:1/82: read d0/d5/f8 [362593,94642] 0 2024-04-24T08:24:33.694 INFO:tasks.workunit.client.1.smithi149.stdout:4/18: dwrite f1 [0,4194304] 0 2024-04-24T08:24:33.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/22: symlink d2/l7 0 2024-04-24T08:24:33.696 INFO:tasks.workunit.client.1.smithi149.stdout:0/23: fsync d2/f5 0 2024-04-24T08:24:33.698 INFO:tasks.workunit.client.1.smithi149.stdout:0/24: read - d2/f5 zero size 2024-04-24T08:24:33.698 INFO:tasks.workunit.client.1.smithi149.stdout:6/14: creat d0/f6 x:0 0 0 2024-04-24T08:24:33.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/15: symlink d0/l7 0 2024-04-24T08:24:33.714 INFO:tasks.workunit.client.1.smithi149.stdout:1/83: mkdir d0/d5/df/d12 0 2024-04-24T08:24:33.717 INFO:tasks.workunit.client.1.smithi149.stdout:6/16: mknod d0/c8 0 2024-04-24T08:24:33.719 INFO:tasks.workunit.client.1.smithi149.stdout:5/41: dwrite f9 [0,4194304] 0 2024-04-24T08:24:33.722 INFO:tasks.workunit.client.1.smithi149.stdout:4/19: dread f1 [0,4194304] 0 2024-04-24T08:24:33.726 INFO:tasks.workunit.client.1.smithi149.stdout:1/84: write d0/f4 [2810284,69299] 0 2024-04-24T08:24:33.728 INFO:tasks.workunit.client.1.smithi149.stdout:8/21: getdents d1 0 2024-04-24T08:24:33.745 INFO:tasks.workunit.client.1.smithi149.stdout:1/85: symlink d0/d5/db/l13 0 2024-04-24T08:24:33.749 INFO:tasks.workunit.client.1.smithi149.stdout:2/35: read d0/f1 [831396,103125] 0 2024-04-24T08:24:33.751 INFO:tasks.workunit.client.1.smithi149.stdout:6/17: truncate d0/f1 1378306 0 2024-04-24T08:24:33.752 INFO:tasks.workunit.client.1.smithi149.stdout:1/86: dread - d0/fd zero size 2024-04-24T08:24:33.756 INFO:tasks.workunit.client.1.smithi149.stdout:6/18: fdatasync d0/f4 0 2024-04-24T08:24:33.759 INFO:tasks.workunit.client.1.smithi149.stdout:1/87: dread - d0/fd zero size 2024-04-24T08:24:33.763 INFO:tasks.workunit.client.1.smithi149.stdout:8/22: dwrite d1/f3 [0,4194304] 0 2024-04-24T08:24:33.764 INFO:tasks.workunit.client.1.smithi149.stdout:9/71: sync 2024-04-24T08:24:33.768 INFO:tasks.workunit.client.1.smithi149.stdout:4/20: dwrite f1 [0,4194304] 0 2024-04-24T08:24:33.768 INFO:tasks.workunit.client.1.smithi149.stdout:6/19: write d0/f4 [355126,107316] 0 2024-04-24T08:24:33.774 INFO:tasks.workunit.client.1.smithi149.stdout:5/42: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:33.788 INFO:tasks.workunit.client.1.smithi149.stdout:0/25: dwrite d2/f4 [4194304,4194304] 0 2024-04-24T08:24:33.796 INFO:tasks.workunit.client.1.smithi149.stdout:1/88: creat d0/d5/df/d12/f14 x:0 0 0 2024-04-24T08:24:33.799 INFO:tasks.workunit.client.1.smithi149.stdout:0/26: creat d2/f8 x:0 0 0 2024-04-24T08:24:33.801 INFO:tasks.workunit.client.1.smithi149.stdout:9/72: link d1/da/dd/f11 d1/de/f1a 0 2024-04-24T08:24:33.805 INFO:tasks.workunit.client.1.smithi149.stdout:0/27: write d2/f5 [690713,87352] 0 2024-04-24T08:24:33.806 INFO:tasks.workunit.client.1.smithi149.stdout:3/45: sync 2024-04-24T08:24:33.807 INFO:tasks.workunit.client.1.smithi149.stdout:8/23: dread d1/f3 [0,4194304] 0 2024-04-24T08:24:33.808 INFO:tasks.workunit.client.1.smithi149.stdout:4/21: fsync f1 0 2024-04-24T08:24:33.810 INFO:tasks.workunit.client.1.smithi149.stdout:6/20: link d0/l7 d0/l9 0 2024-04-24T08:24:33.811 INFO:tasks.workunit.client.1.smithi149.stdout:8/24: read d1/f2 [1397980,57387] 0 2024-04-24T08:24:33.814 INFO:tasks.workunit.client.1.smithi149.stdout:0/28: write d2/f5 [1323128,106313] 0 2024-04-24T08:24:33.815 INFO:tasks.workunit.client.1.smithi149.stdout:8/25: read d1/f2 [1589160,10100] 0 2024-04-24T08:24:33.818 INFO:tasks.workunit.client.1.smithi149.stdout:4/22: creat d2/f4 x:0 0 0 2024-04-24T08:24:33.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/21: creat d0/fa x:0 0 0 2024-04-24T08:24:33.819 INFO:tasks.workunit.client.1.smithi149.stdout:0/29: dread - d2/f8 zero size 2024-04-24T08:24:33.821 INFO:tasks.workunit.client.1.smithi149.stdout:5/43: dwrite f7 [0,4194304] 0 2024-04-24T08:24:33.826 INFO:tasks.workunit.client.1.smithi149.stdout:0/30: read - d2/f8 zero size 2024-04-24T08:24:33.828 INFO:tasks.workunit.client.1.smithi149.stdout:6/22: write d0/f4 [1434612,48340] 0 2024-04-24T08:24:33.838 INFO:tasks.workunit.client.1.smithi149.stdout:8/26: write d1/f3 [3422404,15389] 0 2024-04-24T08:24:33.843 INFO:tasks.workunit.client.1.smithi149.stdout:2/36: truncate d0/f1 743752 0 2024-04-24T08:24:33.851 INFO:tasks.workunit.client.1.smithi149.stdout:1/89: dwrite d0/fd [0,4194304] 0 2024-04-24T08:24:33.854 INFO:tasks.workunit.client.1.smithi149.stdout:9/73: getdents d1 0 2024-04-24T08:24:33.857 INFO:tasks.workunit.client.1.smithi149.stdout:0/31: rmdir d2 39 2024-04-24T08:24:33.865 INFO:tasks.workunit.client.1.smithi149.stdout:7/25: dwrite f1 [0,4194304] 0 2024-04-24T08:24:33.869 INFO:tasks.workunit.client.1.smithi149.stdout:1/90: dread - d0/d5/df/d12/f14 zero size 2024-04-24T08:24:33.872 INFO:tasks.workunit.client.1.smithi149.stdout:4/23: dwrite d2/f4 [0,4194304] 0 2024-04-24T08:24:33.885 INFO:tasks.workunit.client.1.smithi149.stdout:2/37: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:33.887 INFO:tasks.workunit.client.1.smithi149.stdout:3/46: getdents d0/d5/da 0 2024-04-24T08:24:33.889 INFO:tasks.workunit.client.1.smithi149.stdout:9/74: rmdir d1/d2 39 2024-04-24T08:24:33.891 INFO:tasks.workunit.client.1.smithi149.stdout:0/32: fdatasync d2/f5 0 2024-04-24T08:24:33.894 INFO:tasks.workunit.client.1.smithi149.stdout:9/75: write d1/da/fc [606431,12490] 0 2024-04-24T08:24:33.896 INFO:tasks.workunit.client.1.smithi149.stdout:1/91: symlink d0/d5/df/l15 0 2024-04-24T08:24:33.899 INFO:tasks.workunit.client.1.smithi149.stdout:4/24: mknod d2/c5 0 2024-04-24T08:24:33.901 INFO:tasks.workunit.client.1.smithi149.stdout:4/25: chown d2/c5 144856254 1 2024-04-24T08:24:33.904 INFO:tasks.workunit.client.1.smithi149.stdout:2/38: rmdir d0 39 2024-04-24T08:24:33.913 INFO:tasks.workunit.client.1.smithi149.stdout:0/33: mkdir d2/d9 0 2024-04-24T08:24:33.913 INFO:tasks.workunit.client.1.smithi149.stdout:5/44: truncate f5 935329 0 2024-04-24T08:24:33.914 INFO:tasks.workunit.client.1.smithi149.stdout:0/34: readlink d2/l7 0 2024-04-24T08:24:33.915 INFO:tasks.workunit.client.1.smithi149.stdout:5/45: stat f7 0 2024-04-24T08:24:33.917 INFO:tasks.workunit.client.1.smithi149.stdout:1/92: creat d0/f16 x:0 0 0 2024-04-24T08:24:33.918 INFO:tasks.workunit.client.1.smithi149.stdout:9/76: truncate d1/d2/f19 334968 0 2024-04-24T08:24:33.919 INFO:tasks.workunit.client.1.smithi149.stdout:8/27: truncate d1/f3 1062039 0 2024-04-24T08:24:33.921 INFO:tasks.workunit.client.1.smithi149.stdout:5/46: write f7 [5071120,59882] 0 2024-04-24T08:24:33.921 INFO:tasks.workunit.client.1.smithi149.stdout:9/77: fsync d1/d2/f19 0 2024-04-24T08:24:33.921 INFO:tasks.workunit.client.1.smithi149.stdout:4/26: creat d2/f6 x:0 0 0 2024-04-24T08:24:33.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/39: write d0/f4 [469059,50010] 0 2024-04-24T08:24:33.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/40: chown d0 16215 1 2024-04-24T08:24:33.927 INFO:tasks.workunit.client.1.smithi149.stdout:7/26: truncate f3 2206186 0 2024-04-24T08:24:33.930 INFO:tasks.workunit.client.1.smithi149.stdout:0/35: creat d2/fa x:0 0 0 2024-04-24T08:24:33.931 INFO:tasks.workunit.client.1.smithi149.stdout:3/47: dwrite d0/d5/f8 [4194304,4194304] 0 2024-04-24T08:24:33.934 INFO:tasks.workunit.client.1.smithi149.stdout:3/48: dread - d0/fc zero size 2024-04-24T08:24:33.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/36: dread - d2/fa zero size 2024-04-24T08:24:33.937 INFO:tasks.workunit.client.1.smithi149.stdout:0/37: write d2/fa [41939,38653] 0 2024-04-24T08:24:33.941 INFO:tasks.workunit.client.1.smithi149.stdout:0/38: chown d2/f8 26009533 1 2024-04-24T08:24:33.942 INFO:tasks.workunit.client.1.smithi149.stdout:1/93: mknod d0/d5/c17 0 2024-04-24T08:24:33.946 INFO:tasks.workunit.client.1.smithi149.stdout:4/27: dread f1 [0,4194304] 0 2024-04-24T08:24:33.948 INFO:tasks.workunit.client.1.smithi149.stdout:5/47: mknod db/ce 0 2024-04-24T08:24:33.952 INFO:tasks.workunit.client.1.smithi149.stdout:3/49: rmdir d0/d5 39 2024-04-24T08:24:33.958 INFO:tasks.workunit.client.1.smithi149.stdout:7/27: write f1 [1174299,94757] 0 2024-04-24T08:24:33.960 INFO:tasks.workunit.client.1.smithi149.stdout:6/23: link d0/f1 d0/fb 0 2024-04-24T08:24:33.966 INFO:tasks.workunit.client.1.smithi149.stdout:7/28: fdatasync f3 0 2024-04-24T08:24:33.966 INFO:tasks.workunit.client.1.smithi149.stdout:1/94: mkdir d0/d5/db/d18 0 2024-04-24T08:24:33.968 INFO:tasks.workunit.client.1.smithi149.stdout:5/48: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:33.971 INFO:tasks.workunit.client.1.smithi149.stdout:1/95: readlink d0/d5/db/l13 0 2024-04-24T08:24:33.974 INFO:tasks.workunit.client.1.smithi149.stdout:4/28: rmdir d2 39 2024-04-24T08:24:33.974 INFO:tasks.workunit.client.1.smithi149.stdout:9/78: write d1/da/dd/f11 [1223477,57794] 0 2024-04-24T08:24:33.978 INFO:tasks.workunit.client.1.smithi149.stdout:9/79: chown d1/de/ff 4 1 2024-04-24T08:24:33.979 INFO:tasks.workunit.client.1.smithi149.stdout:2/41: rename d0/f1 to d0/d6/f7 0 2024-04-24T08:24:33.979 INFO:tasks.workunit.client.1.smithi149.stdout:6/24: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:33.979 INFO:tasks.workunit.client.1.smithi149.stdout:7/29: dread f1 [0,4194304] 0 2024-04-24T08:24:33.981 INFO:tasks.workunit.client.1.smithi149.stdout:2/42: chown d0/c3 243254461 1 2024-04-24T08:24:33.982 INFO:tasks.workunit.client.1.smithi149.stdout:7/30: write f3 [1498499,95641] 0 2024-04-24T08:24:33.988 INFO:tasks.workunit.client.1.smithi149.stdout:5/49: mkdir db/df 0 2024-04-24T08:24:33.991 INFO:tasks.workunit.client.1.smithi149.stdout:1/96: chown d0/d5/f9 19922946 1 2024-04-24T08:24:33.998 INFO:tasks.workunit.client.1.smithi149.stdout:3/50: rename d0/f6 to d0/d5/ff 0 2024-04-24T08:24:33.999 INFO:tasks.workunit.client.1.smithi149.stdout:6/25: creat d0/fc x:0 0 0 2024-04-24T08:24:33.999 INFO:tasks.workunit.client.1.smithi149.stdout:7/31: mknod c8 0 2024-04-24T08:24:34.002 INFO:tasks.workunit.client.1.smithi149.stdout:9/80: dwrite d1/de/f18 [0,4194304] 0 2024-04-24T08:24:34.006 INFO:tasks.workunit.client.1.smithi149.stdout:2/43: rename d0/f4 to d0/f8 0 2024-04-24T08:24:34.006 INFO:tasks.workunit.client.1.smithi149.stdout:2/44: rename d0 to d0/d9 22 2024-04-24T08:24:34.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/97: mkdir d0/d5/df/d12/d19 0 2024-04-24T08:24:34.010 INFO:tasks.workunit.client.1.smithi149.stdout:4/29: rename d2/f6 to d2/f7 0 2024-04-24T08:24:34.014 INFO:tasks.workunit.client.1.smithi149.stdout:5/50: dread f9 [0,4194304] 0 2024-04-24T08:24:34.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/32: mkdir d9 0 2024-04-24T08:24:34.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/39: getdents d2 0 2024-04-24T08:24:34.026 INFO:tasks.workunit.client.1.smithi149.stdout:2/45: mknod d0/d6/ca 0 2024-04-24T08:24:34.026 INFO:tasks.workunit.client.1.smithi149.stdout:8/28: dwrite d1/f3 [0,4194304] 0 2024-04-24T08:24:34.027 INFO:tasks.workunit.client.1.smithi149.stdout:3/51: mknod d0/c10 0 2024-04-24T08:24:34.029 INFO:tasks.workunit.client.1.smithi149.stdout:1/98: dread d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:34.032 INFO:tasks.workunit.client.1.smithi149.stdout:7/33: dread f3 [0,4194304] 0 2024-04-24T08:24:34.032 INFO:tasks.workunit.client.1.smithi149.stdout:7/34: readlink - no filename 2024-04-24T08:24:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:33 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:24:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:33 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:24:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:33 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:24:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:33 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:24:34.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:33 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:24:34.048 INFO:tasks.workunit.client.1.smithi149.stdout:2/46: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:34.051 INFO:tasks.workunit.client.1.smithi149.stdout:3/52: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:34.063 INFO:tasks.workunit.client.1.smithi149.stdout:0/40: chown c1 12262 1 2024-04-24T08:24:34.067 INFO:tasks.workunit.client.1.smithi149.stdout:8/29: fsync d1/f3 0 2024-04-24T08:24:34.073 INFO:tasks.workunit.client.1.smithi149.stdout:2/47: write d0/f8 [1541049,78917] 0 2024-04-24T08:24:34.076 INFO:tasks.workunit.client.1.smithi149.stdout:2/48: write d0/f8 [2620562,13803] 0 2024-04-24T08:24:34.081 INFO:tasks.workunit.client.1.smithi149.stdout:0/41: creat d2/fb x:0 0 0 2024-04-24T08:24:34.083 INFO:tasks.workunit.client.1.smithi149.stdout:6/26: getdents d0 0 2024-04-24T08:24:34.086 INFO:tasks.workunit.client.1.smithi149.stdout:1/99: symlink d0/d5/df/d12/d19/l1a 0 2024-04-24T08:24:34.086 INFO:tasks.workunit.client.1.smithi149.stdout:7/35: creat d9/fa x:0 0 0 2024-04-24T08:24:34.092 INFO:tasks.workunit.client.1.smithi149.stdout:8/30: dwrite d1/f3 [0,4194304] 0 2024-04-24T08:24:34.097 INFO:tasks.workunit.client.1.smithi149.stdout:6/27: read d0/fb [828971,76721] 0 2024-04-24T08:24:34.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/100: mkdir d0/d1b 0 2024-04-24T08:24:34.101 INFO:tasks.workunit.client.1.smithi149.stdout:1/101: read d0/da/fc [3120300,112131] 0 2024-04-24T08:24:34.102 INFO:tasks.workunit.client.1.smithi149.stdout:7/36: creat d9/fb x:0 0 0 2024-04-24T08:24:34.103 INFO:tasks.workunit.client.1.smithi149.stdout:7/37: dread - d9/fb zero size 2024-04-24T08:24:34.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/53: creat d0/f11 x:0 0 0 2024-04-24T08:24:34.108 INFO:tasks.workunit.client.1.smithi149.stdout:8/31: creat d1/f7 x:0 0 0 2024-04-24T08:24:34.109 INFO:tasks.workunit.client.1.smithi149.stdout:2/49: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:34.113 INFO:tasks.workunit.client.1.smithi149.stdout:6/28: readlink d0/l7 0 2024-04-24T08:24:34.115 INFO:tasks.workunit.client.1.smithi149.stdout:1/102: unlink d0/d5/c6 0 2024-04-24T08:24:34.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/38: fsync f1 0 2024-04-24T08:24:34.124 INFO:tasks.workunit.client.1.smithi149.stdout:3/54: mkdir d0/d7/d12 0 2024-04-24T08:24:34.125 INFO:tasks.workunit.client.1.smithi149.stdout:2/50: mkdir d0/d6/db 0 2024-04-24T08:24:34.126 INFO:tasks.workunit.client.1.smithi149.stdout:8/32: creat d1/f8 x:0 0 0 2024-04-24T08:24:34.127 INFO:tasks.workunit.client.1.smithi149.stdout:0/42: link d2/l7 d2/lc 0 2024-04-24T08:24:34.129 INFO:tasks.workunit.client.1.smithi149.stdout:6/29: symlink d0/ld 0 2024-04-24T08:24:34.130 INFO:tasks.workunit.client.1.smithi149.stdout:8/33: dread - d1/f7 zero size 2024-04-24T08:24:34.134 INFO:tasks.workunit.client.1.smithi149.stdout:5/51: dwrite f5 [0,4194304] 0 2024-04-24T08:24:34.138 INFO:tasks.workunit.client.1.smithi149.stdout:7/39: mkdir d9/dc 0 2024-04-24T08:24:34.140 INFO:tasks.workunit.client.1.smithi149.stdout:7/40: fdatasync f1 0 2024-04-24T08:24:34.140 INFO:tasks.workunit.client.1.smithi149.stdout:1/103: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:34.142 INFO:tasks.workunit.client.1.smithi149.stdout:3/55: mknod d0/d7/c13 0 2024-04-24T08:24:34.147 INFO:tasks.workunit.client.1.smithi149.stdout:2/51: symlink d0/d6/lc 0 2024-04-24T08:24:34.151 INFO:tasks.workunit.client.1.smithi149.stdout:6/30: creat d0/fe x:0 0 0 2024-04-24T08:24:34.157 INFO:tasks.workunit.client.1.smithi149.stdout:0/43: dwrite d2/fb [0,4194304] 0 2024-04-24T08:24:34.158 INFO:tasks.workunit.client.1.smithi149.stdout:0/44: chown d2 289046116 1 2024-04-24T08:24:34.160 INFO:tasks.workunit.client.1.smithi149.stdout:7/41: unlink c6 0 2024-04-24T08:24:34.162 INFO:tasks.workunit.client.1.smithi149.stdout:0/45: write d2/fb [4088564,40273] 0 2024-04-24T08:24:34.163 INFO:tasks.workunit.client.1.smithi149.stdout:1/104: creat d0/d5/db/f1c x:0 0 0 2024-04-24T08:24:34.166 INFO:tasks.workunit.client.1.smithi149.stdout:3/56: creat d0/d5/da/f14 x:0 0 0 2024-04-24T08:24:34.166 INFO:tasks.workunit.client.1.smithi149.stdout:1/105: fdatasync d0/d5/df/d12/f14 0 2024-04-24T08:24:34.187 INFO:tasks.workunit.client.1.smithi149.stdout:3/57: dread d0/d5/f8 [4194304,4194304] 0 2024-04-24T08:24:34.188 INFO:tasks.workunit.client.1.smithi149.stdout:0/46: dwrite d2/fa [0,4194304] 0 2024-04-24T08:24:34.191 INFO:tasks.workunit.client.1.smithi149.stdout:1/106: symlink d0/da/l1d 0 2024-04-24T08:24:34.203 INFO:tasks.workunit.client.1.smithi149.stdout:5/52: rmdir db/df 0 2024-04-24T08:24:34.213 INFO:tasks.workunit.client.1.smithi149.stdout:3/58: fdatasync d0/f3 0 2024-04-24T08:24:34.214 INFO:tasks.workunit.client.1.smithi149.stdout:1/107: dread d0/da/fc [0,4194304] 0 2024-04-24T08:24:34.216 INFO:tasks.workunit.client.1.smithi149.stdout:0/47: mkdir d2/dd 0 2024-04-24T08:24:34.216 INFO:tasks.workunit.client.1.smithi149.stdout:4/30: sync 2024-04-24T08:24:34.216 INFO:tasks.workunit.client.1.smithi149.stdout:9/81: sync 2024-04-24T08:24:34.217 INFO:tasks.workunit.client.1.smithi149.stdout:0/48: chown d2/fa 2874 1 2024-04-24T08:24:34.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/82: read d1/da/dd/f11 [796099,4196] 0 2024-04-24T08:24:34.219 INFO:tasks.workunit.client.1.smithi149.stdout:8/34: getdents d1 0 2024-04-24T08:24:34.221 INFO:tasks.workunit.client.1.smithi149.stdout:6/31: link d0/ld d0/lf 0 2024-04-24T08:24:34.223 INFO:tasks.workunit.client.1.smithi149.stdout:6/32: write d0/f3 [2021825,39448] 0 2024-04-24T08:24:34.224 INFO:tasks.workunit.client.1.smithi149.stdout:5/53: dread db/fc [0,4194304] 0 2024-04-24T08:24:34.226 INFO:tasks.workunit.client.1.smithi149.stdout:6/33: write d0/f5 [361710,105576] 0 2024-04-24T08:24:34.228 INFO:tasks.workunit.client.1.smithi149.stdout:6/34: truncate d0/fc 984287 0 2024-04-24T08:24:34.229 INFO:tasks.workunit.client.1.smithi149.stdout:6/35: dread - d0/fe zero size 2024-04-24T08:24:34.232 INFO:tasks.workunit.client.1.smithi149.stdout:6/36: read d0/f3 [3779890,47484] 0 2024-04-24T08:24:34.241 INFO:tasks.workunit.client.1.smithi149.stdout:5/54: dread f9 [0,4194304] 0 2024-04-24T08:24:34.244 INFO:tasks.workunit.client.1.smithi149.stdout:2/52: dwrite d0/d6/f7 [0,4194304] 0 2024-04-24T08:24:34.270 INFO:tasks.workunit.client.1.smithi149.stdout:5/55: dwrite f7 [0,4194304] 0 2024-04-24T08:24:34.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/53: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:33 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:24:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:33 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:24:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:33 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:24:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:33 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:24:34.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:33 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/2602858784' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:24:34.317 INFO:tasks.workunit.client.1.smithi149.stdout:5/56: fdatasync f7 0 2024-04-24T08:24:34.341 INFO:tasks.workunit.client.1.smithi149.stdout:7/42: link f1 d9/fd 0 2024-04-24T08:24:34.345 INFO:tasks.workunit.client.1.smithi149.stdout:3/59: symlink d0/d5/da/l15 0 2024-04-24T08:24:34.346 INFO:tasks.workunit.client.1.smithi149.stdout:1/108: rmdir d0/d5/df/d12 39 2024-04-24T08:24:34.348 INFO:tasks.workunit.client.1.smithi149.stdout:4/31: fdatasync d2/f7 0 2024-04-24T08:24:34.350 INFO:tasks.workunit.client.1.smithi149.stdout:4/32: dread - d2/f7 zero size 2024-04-24T08:24:34.351 INFO:tasks.workunit.client.1.smithi149.stdout:9/83: mknod d1/da/dd/c1b 0 2024-04-24T08:24:34.355 INFO:tasks.workunit.client.1.smithi149.stdout:6/37: creat d0/f10 x:0 0 0 2024-04-24T08:24:34.357 INFO:tasks.workunit.client.1.smithi149.stdout:6/38: dread - d0/fa zero size 2024-04-24T08:24:34.361 INFO:tasks.workunit.client.1.smithi149.stdout:2/54: mknod d0/cd 0 2024-04-24T08:24:34.365 INFO:tasks.workunit.client.1.smithi149.stdout:1/109: creat d0/d5/db/f1e x:0 0 0 2024-04-24T08:24:34.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/49: creat d2/d9/fe x:0 0 0 2024-04-24T08:24:34.368 INFO:tasks.workunit.client.1.smithi149.stdout:8/35: link d1/f7 d1/f9 0 2024-04-24T08:24:34.369 INFO:tasks.workunit.client.1.smithi149.stdout:8/36: truncate d1/f8 503941 0 2024-04-24T08:24:34.370 INFO:tasks.workunit.client.1.smithi149.stdout:8/37: readlink l0 0 2024-04-24T08:24:34.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/110: dwrite d0/da/f11 [0,4194304] 0 2024-04-24T08:24:34.482 INFO:tasks.workunit.client.1.smithi149.stdout:4/33: symlink d2/l8 0 2024-04-24T08:24:34.489 INFO:tasks.workunit.client.1.smithi149.stdout:6/39: rename d0/f6 to d0/f11 0 2024-04-24T08:24:34.496 INFO:tasks.workunit.client.1.smithi149.stdout:8/38: unlink d1/f7 0 2024-04-24T08:24:34.502 INFO:tasks.workunit.client.1.smithi149.stdout:4/34: creat d2/f9 x:0 0 0 2024-04-24T08:24:34.504 INFO:tasks.workunit.client.1.smithi149.stdout:4/35: write d2/f9 [845283,19624] 0 2024-04-24T08:24:34.508 INFO:tasks.workunit.client.1.smithi149.stdout:6/40: creat d0/f12 x:0 0 0 2024-04-24T08:24:34.515 INFO:tasks.workunit.client.1.smithi149.stdout:8/39: mknod d1/ca 0 2024-04-24T08:24:34.517 INFO:tasks.workunit.client.1.smithi149.stdout:1/111: creat d0/d1b/f1f x:0 0 0 2024-04-24T08:24:34.520 INFO:tasks.workunit.client.1.smithi149.stdout:0/50: truncate d2/f5 553139 0 2024-04-24T08:24:34.524 INFO:tasks.workunit.client.1.smithi149.stdout:8/40: rename d1/c4 to d1/cb 0 2024-04-24T08:24:34.528 INFO:tasks.workunit.client.1.smithi149.stdout:0/51: rename d2/dd to d2/d9/df 0 2024-04-24T08:24:34.534 INFO:tasks.workunit.client.1.smithi149.stdout:1/112: unlink d0/d5/df/d12/f14 0 2024-04-24T08:24:34.535 INFO:tasks.workunit.client.1.smithi149.stdout:6/41: dwrite d0/f10 [0,4194304] 0 2024-04-24T08:24:34.539 INFO:tasks.workunit.client.1.smithi149.stdout:1/113: write d0/da/f11 [1388115,79113] 0 2024-04-24T08:24:34.543 INFO:tasks.workunit.client.1.smithi149.stdout:6/42: rename d0/l9 to d0/l13 0 2024-04-24T08:24:34.546 INFO:tasks.workunit.client.1.smithi149.stdout:0/52: dread d2/f4 [4194304,4194304] 0 2024-04-24T08:24:34.550 INFO:tasks.workunit.client.1.smithi149.stdout:6/43: creat d0/f14 x:0 0 0 2024-04-24T08:24:34.551 INFO:tasks.workunit.client.1.smithi149.stdout:1/114: mknod d0/d5/df/c20 0 2024-04-24T08:24:34.557 INFO:tasks.workunit.client.1.smithi149.stdout:0/53: symlink d2/l10 0 2024-04-24T08:24:34.560 INFO:tasks.workunit.client.1.smithi149.stdout:0/54: write d2/f8 [190482,5696] 0 2024-04-24T08:24:34.564 INFO:tasks.workunit.client.1.smithi149.stdout:0/55: mknod d2/d9/c11 0 2024-04-24T08:24:34.567 INFO:tasks.workunit.client.1.smithi149.stdout:0/56: getdents d2/d9/df 0 2024-04-24T08:24:34.570 INFO:tasks.workunit.client.1.smithi149.stdout:0/57: symlink d2/d9/df/l12 0 2024-04-24T08:24:34.571 INFO:tasks.workunit.client.1.smithi149.stdout:1/115: dread d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:34.574 INFO:tasks.workunit.client.1.smithi149.stdout:1/116: write d0/d5/db/f1e [504352,130287] 0 2024-04-24T08:24:34.575 INFO:tasks.workunit.client.1.smithi149.stdout:1/117: stat d0/da/l1d 0 2024-04-24T08:24:34.577 INFO:tasks.workunit.client.1.smithi149.stdout:1/118: symlink d0/d5/l21 0 2024-04-24T08:24:34.581 INFO:tasks.workunit.client.1.smithi149.stdout:1/119: symlink d0/d5/df/l22 0 2024-04-24T08:24:34.584 INFO:tasks.workunit.client.1.smithi149.stdout:1/120: creat d0/d5/df/d12/d19/f23 x:0 0 0 2024-04-24T08:24:34.586 INFO:tasks.workunit.client.1.smithi149.stdout:1/121: fsync d0/da/fc 0 2024-04-24T08:24:34.589 INFO:tasks.workunit.client.1.smithi149.stdout:1/122: rmdir d0/d5/df 39 2024-04-24T08:24:34.590 INFO:tasks.workunit.client.1.smithi149.stdout:3/60: sync 2024-04-24T08:24:34.591 INFO:tasks.workunit.client.1.smithi149.stdout:3/61: fsync d0/fc 0 2024-04-24T08:24:34.593 INFO:tasks.workunit.client.1.smithi149.stdout:3/62: chown d0/fc 0 1 2024-04-24T08:24:34.602 INFO:tasks.workunit.client.1.smithi149.stdout:1/123: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:34.604 INFO:tasks.workunit.client.1.smithi149.stdout:1/124: chown d0/fe 42423074 1 2024-04-24T08:24:34.610 INFO:tasks.workunit.client.1.smithi149.stdout:1/125: link d0/da/f11 d0/da/f24 0 2024-04-24T08:24:34.613 INFO:tasks.workunit.client.1.smithi149.stdout:1/126: creat d0/f25 x:0 0 0 2024-04-24T08:24:34.616 INFO:tasks.workunit.client.1.smithi149.stdout:1/127: fsync d0/d5/db/f1c 0 2024-04-24T08:24:34.616 INFO:tasks.workunit.client.1.smithi149.stdout:3/63: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:34.633 INFO:tasks.workunit.client.1.smithi149.stdout:5/57: truncate db/fc 4569053 0 2024-04-24T08:24:34.638 INFO:tasks.workunit.client.1.smithi149.stdout:5/58: creat db/f10 x:0 0 0 2024-04-24T08:24:34.640 INFO:tasks.workunit.client.1.smithi149.stdout:2/55: sync 2024-04-24T08:24:34.640 INFO:tasks.workunit.client.1.smithi149.stdout:8/41: sync 2024-04-24T08:24:34.642 INFO:tasks.workunit.client.1.smithi149.stdout:9/84: truncate d1/da/dd/f11 948041 0 2024-04-24T08:24:34.648 INFO:tasks.workunit.client.1.smithi149.stdout:3/64: dwrite d0/d5/ff [0,4194304] 0 2024-04-24T08:24:34.649 INFO:tasks.workunit.client.1.smithi149.stdout:1/128: dwrite d0/d1b/f1f [0,4194304] 0 2024-04-24T08:24:34.650 INFO:tasks.workunit.client.1.smithi149.stdout:7/43: write d9/fd [2989371,19200] 0 2024-04-24T08:24:34.664 INFO:tasks.workunit.client.1.smithi149.stdout:4/36: rmdir d2 39 2024-04-24T08:24:34.667 INFO:tasks.workunit.client.1.smithi149.stdout:7/44: unlink d9/fa 0 2024-04-24T08:24:34.667 INFO:tasks.workunit.client.1.smithi149.stdout:7/45: readlink - no filename 2024-04-24T08:24:34.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/85: mkdir d1/d2/d15/d1c 0 2024-04-24T08:24:34.675 INFO:tasks.workunit.client.1.smithi149.stdout:0/58: unlink d2/f5 0 2024-04-24T08:24:34.678 INFO:tasks.workunit.client.1.smithi149.stdout:2/56: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:34.679 INFO:tasks.workunit.client.1.smithi149.stdout:8/42: dwrite d1/f3 [0,4194304] 0 2024-04-24T08:24:34.682 INFO:tasks.workunit.client.1.smithi149.stdout:9/86: readlink d1/da/dd/l16 0 2024-04-24T08:24:34.695 INFO:tasks.workunit.client.1.smithi149.stdout:1/129: creat d0/d5/db/d18/f26 x:0 0 0 2024-04-24T08:24:34.696 INFO:tasks.workunit.client.1.smithi149.stdout:6/44: rmdir d0 39 2024-04-24T08:24:34.698 INFO:tasks.workunit.client.1.smithi149.stdout:0/59: dwrite d2/fb [0,4194304] 0 2024-04-24T08:24:34.703 INFO:tasks.workunit.client.1.smithi149.stdout:2/57: unlink d0/d6/ca 0 2024-04-24T08:24:34.706 INFO:tasks.workunit.client.1.smithi149.stdout:9/87: creat d1/de/d12/f1d x:0 0 0 2024-04-24T08:24:34.706 INFO:tasks.workunit.client.1.smithi149.stdout:3/65: rmdir d0/d7/d12 0 2024-04-24T08:24:34.708 INFO:tasks.workunit.client.1.smithi149.stdout:9/88: fdatasync d1/d2/f19 0 2024-04-24T08:24:34.709 INFO:tasks.workunit.client.1.smithi149.stdout:4/37: rmdir d2 39 2024-04-24T08:24:34.711 INFO:tasks.workunit.client.1.smithi149.stdout:9/89: chown d1/da/dd/l16 19721437 1 2024-04-24T08:24:34.715 INFO:tasks.workunit.client.1.smithi149.stdout:9/90: read d1/da/f8 [2611082,5647] 0 2024-04-24T08:24:34.718 INFO:tasks.workunit.client.1.smithi149.stdout:6/45: read d0/f5 [141633,111598] 0 2024-04-24T08:24:34.718 INFO:tasks.workunit.client.1.smithi149.stdout:7/46: mkdir d9/dc/de 0 2024-04-24T08:24:34.722 INFO:tasks.workunit.client.1.smithi149.stdout:2/58: creat d0/fe x:0 0 0 2024-04-24T08:24:34.726 INFO:tasks.workunit.client.1.smithi149.stdout:0/60: dwrite d2/f8 [0,4194304] 0 2024-04-24T08:24:34.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/66: dread d0/d5/ff [0,4194304] 0 2024-04-24T08:24:34.729 INFO:tasks.workunit.client.1.smithi149.stdout:1/130: mkdir d0/d5/df/d12/d27 0 2024-04-24T08:24:34.732 INFO:tasks.workunit.client.1.smithi149.stdout:5/59: rmdir db 39 2024-04-24T08:24:34.736 INFO:tasks.workunit.client.1.smithi149.stdout:7/47: creat d9/ff x:0 0 0 2024-04-24T08:24:34.740 INFO:tasks.workunit.client.1.smithi149.stdout:4/38: rename f1 to d2/fa 0 2024-04-24T08:24:34.742 INFO:tasks.workunit.client.1.smithi149.stdout:6/46: dwrite d0/fe [0,4194304] 0 2024-04-24T08:24:34.745 INFO:tasks.workunit.client.1.smithi149.stdout:0/61: unlink d2/f4 0 2024-04-24T08:24:34.749 INFO:tasks.workunit.client.1.smithi149.stdout:3/67: creat d0/d5/f16 x:0 0 0 2024-04-24T08:24:34.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/62: write d2/fb [2536467,29129] 0 2024-04-24T08:24:34.756 INFO:tasks.workunit.client.1.smithi149.stdout:1/131: dwrite d0/da/fc [0,4194304] 0 2024-04-24T08:24:34.765 INFO:tasks.workunit.client.1.smithi149.stdout:7/48: mknod d9/c10 0 2024-04-24T08:24:34.767 INFO:tasks.workunit.client.1.smithi149.stdout:2/59: mknod d0/d6/db/cf 0 2024-04-24T08:24:34.769 INFO:tasks.workunit.client.1.smithi149.stdout:7/49: write d9/ff [150900,126671] 0 2024-04-24T08:24:34.769 INFO:tasks.workunit.client.1.smithi149.stdout:4/39: creat d2/fb x:0 0 0 2024-04-24T08:24:34.771 INFO:tasks.workunit.client.1.smithi149.stdout:6/47: fdatasync d0/f4 0 2024-04-24T08:24:34.773 INFO:tasks.workunit.client.1.smithi149.stdout:2/60: chown d0/d6/db/cf 101271 1 2024-04-24T08:24:34.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/68: dwrite d0/f4 [4194304,4194304] 0 2024-04-24T08:24:34.775 INFO:tasks.workunit.client.1.smithi149.stdout:6/48: write d0/f14 [860167,129944] 0 2024-04-24T08:24:34.776 INFO:tasks.workunit.client.1.smithi149.stdout:2/61: write d0/d6/f7 [1838574,108209] 0 2024-04-24T08:24:34.777 INFO:tasks.workunit.client.1.smithi149.stdout:3/69: chown d0/d5/da/fb 7 1 2024-04-24T08:24:34.778 INFO:tasks.workunit.client.1.smithi149.stdout:6/49: write d0/fa [351148,104131] 0 2024-04-24T08:24:34.780 INFO:tasks.workunit.client.1.smithi149.stdout:0/63: creat d2/d9/f13 x:0 0 0 2024-04-24T08:24:34.780 INFO:tasks.workunit.client.1.smithi149.stdout:3/70: chown d0/d5/da/f14 1216650308 1 2024-04-24T08:24:34.786 INFO:tasks.workunit.client.1.smithi149.stdout:0/64: write d2/fb [2817720,10459] 0 2024-04-24T08:24:34.788 INFO:tasks.workunit.client.1.smithi149.stdout:0/65: chown d2/d9 2 1 2024-04-24T08:24:34.789 INFO:tasks.workunit.client.1.smithi149.stdout:8/43: dread - d1/f9 zero size 2024-04-24T08:24:34.790 INFO:tasks.workunit.client.1.smithi149.stdout:6/50: dread d0/fc [0,4194304] 0 2024-04-24T08:24:34.798 INFO:tasks.workunit.client.1.smithi149.stdout:4/40: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:34.802 INFO:tasks.workunit.client.1.smithi149.stdout:4/41: read - d2/fb zero size 2024-04-24T08:24:34.804 INFO:tasks.workunit.client.1.smithi149.stdout:4/42: chown d2 849490 1 2024-04-24T08:24:34.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/60: dread db/fc [0,4194304] 0 2024-04-24T08:24:34.806 INFO:tasks.workunit.client.1.smithi149.stdout:2/62: creat d0/d6/db/f10 x:0 0 0 2024-04-24T08:24:34.810 INFO:tasks.workunit.client.1.smithi149.stdout:4/43: write d2/f9 [1556864,55264] 0 2024-04-24T08:24:34.816 INFO:tasks.workunit.client.1.smithi149.stdout:6/51: dread d0/f10 [0,4194304] 0 2024-04-24T08:24:34.820 INFO:tasks.workunit.client.1.smithi149.stdout:7/50: dwrite f3 [0,4194304] 0 2024-04-24T08:24:34.826 INFO:tasks.workunit.client.1.smithi149.stdout:7/51: write f3 [2119644,128859] 0 2024-04-24T08:24:34.826 INFO:tasks.workunit.client.1.smithi149.stdout:8/44: write d1/f3 [263642,43214] 0 2024-04-24T08:24:34.834 INFO:tasks.workunit.client.1.smithi149.stdout:4/44: dread d2/f7 [0,4194304] 0 2024-04-24T08:24:34.836 INFO:tasks.workunit.client.1.smithi149.stdout:5/61: mknod db/c11 0 2024-04-24T08:24:34.840 INFO:tasks.workunit.client.1.smithi149.stdout:2/63: symlink d0/d6/l11 0 2024-04-24T08:24:34.844 INFO:tasks.workunit.client.1.smithi149.stdout:3/71: mknod d0/c17 0 2024-04-24T08:24:34.850 INFO:tasks.workunit.client.1.smithi149.stdout:6/52: readlink d0/ld 0 2024-04-24T08:24:34.850 INFO:tasks.workunit.client.1.smithi149.stdout:3/72: write d0/d5/f16 [485454,95396] 0 2024-04-24T08:24:34.853 INFO:tasks.workunit.client.1.smithi149.stdout:7/52: dwrite f3 [0,4194304] 0 2024-04-24T08:24:34.858 INFO:tasks.workunit.client.1.smithi149.stdout:8/45: write d1/f2 [2256080,62222] 0 2024-04-24T08:24:34.863 INFO:tasks.workunit.client.1.smithi149.stdout:1/132: rename d0/da/f11 to d0/d5/df/f28 0 2024-04-24T08:24:34.865 INFO:tasks.workunit.client.1.smithi149.stdout:5/62: write f9 [3944821,92786] 0 2024-04-24T08:24:34.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/45: creat d2/fc x:0 0 0 2024-04-24T08:24:34.871 INFO:tasks.workunit.client.1.smithi149.stdout:2/64: creat d0/d6/f12 x:0 0 0 2024-04-24T08:24:34.872 INFO:tasks.workunit.client.1.smithi149.stdout:6/53: truncate d0/fc 660341 0 2024-04-24T08:24:34.874 INFO:tasks.workunit.client.1.smithi149.stdout:6/54: fdatasync d0/fe 0 2024-04-24T08:24:34.875 INFO:tasks.workunit.client.1.smithi149.stdout:9/91: truncate d1/de/f1a 1381477 0 2024-04-24T08:24:34.878 INFO:tasks.workunit.client.1.smithi149.stdout:1/133: creat d0/d5/db/d18/f29 x:0 0 0 2024-04-24T08:24:34.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/53: dwrite d9/ff [0,4194304] 0 2024-04-24T08:24:34.889 INFO:tasks.workunit.client.1.smithi149.stdout:1/134: mkdir d0/d1b/d2a 0 2024-04-24T08:24:34.910 INFO:tasks.workunit.client.1.smithi149.stdout:5/63: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:34.921 INFO:tasks.workunit.client.1.smithi149.stdout:1/135: write d0/d5/f9 [2461710,24170] 0 2024-04-24T08:24:34.924 INFO:tasks.workunit.client.1.smithi149.stdout:6/55: dread d0/f3 [0,4194304] 0 2024-04-24T08:24:34.929 INFO:tasks.workunit.client.1.smithi149.stdout:5/64: write f5 [4772144,89553] 0 2024-04-24T08:24:34.931 INFO:tasks.workunit.client.1.smithi149.stdout:2/65: dwrite d0/d6/f7 [0,4194304] 0 2024-04-24T08:24:34.931 INFO:tasks.workunit.client.1.smithi149.stdout:9/92: write d1/da/dd/f11 [58014,5487] 0 2024-04-24T08:24:34.933 INFO:tasks.workunit.client.1.smithi149.stdout:3/73: dwrite d0/d5/ff [0,4194304] 0 2024-04-24T08:24:34.938 INFO:tasks.workunit.client.1.smithi149.stdout:2/66: mknod d0/d6/c13 0 2024-04-24T08:24:34.942 INFO:tasks.workunit.client.1.smithi149.stdout:6/56: truncate d0/f12 1019689 0 2024-04-24T08:24:34.943 INFO:tasks.workunit.client.1.smithi149.stdout:4/46: dwrite d2/fa [0,4194304] 0 2024-04-24T08:24:34.948 INFO:tasks.workunit.client.1.smithi149.stdout:3/74: chown d0/d7/fd 1983763969 1 2024-04-24T08:24:34.948 INFO:tasks.workunit.client.1.smithi149.stdout:7/54: dwrite d9/fb [0,4194304] 0 2024-04-24T08:24:34.948 INFO:tasks.workunit.client.1.smithi149.stdout:5/65: write f7 [1252992,35282] 0 2024-04-24T08:24:34.950 INFO:tasks.workunit.client.1.smithi149.stdout:1/136: creat d0/d5/df/d12/d19/f2b x:0 0 0 2024-04-24T08:24:34.951 INFO:tasks.workunit.client.1.smithi149.stdout:9/93: creat d1/da/d13/f1e x:0 0 0 2024-04-24T08:24:34.952 INFO:tasks.workunit.client.1.smithi149.stdout:4/47: chown d2/f9 891045777 1 2024-04-24T08:24:34.955 INFO:tasks.workunit.client.1.smithi149.stdout:6/57: readlink d0/l13 0 2024-04-24T08:24:34.956 INFO:tasks.workunit.client.1.smithi149.stdout:3/75: write d0/d5/f16 [1471744,60312] 0 2024-04-24T08:24:34.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/55: rmdir d9 39 2024-04-24T08:24:34.960 INFO:tasks.workunit.client.1.smithi149.stdout:5/66: mkdir db/d12 0 2024-04-24T08:24:34.961 INFO:tasks.workunit.client.1.smithi149.stdout:5/67: rename db to db/d12/d13 22 2024-04-24T08:24:34.962 INFO:tasks.workunit.client.1.smithi149.stdout:4/48: rmdir d2 39 2024-04-24T08:24:34.965 INFO:tasks.workunit.client.1.smithi149.stdout:1/137: write d0/da/f24 [1173275,112528] 0 2024-04-24T08:24:34.966 INFO:tasks.workunit.client.1.smithi149.stdout:9/94: symlink d1/da/d13/l1f 0 2024-04-24T08:24:34.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/95: write d1/d2/f19 [967165,107810] 0 2024-04-24T08:24:34.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/96: fdatasync d1/f4 0 2024-04-24T08:24:34.971 INFO:tasks.workunit.client.1.smithi149.stdout:2/67: rmdir d0 39 2024-04-24T08:24:34.973 INFO:tasks.workunit.client.1.smithi149.stdout:4/49: chown d2/f9 3 1 2024-04-24T08:24:34.976 INFO:tasks.workunit.client.1.smithi149.stdout:5/68: unlink f5 0 2024-04-24T08:24:34.982 INFO:tasks.workunit.client.1.smithi149.stdout:2/68: truncate d0/d6/f12 1024322 0 2024-04-24T08:24:34.984 INFO:tasks.workunit.client.1.smithi149.stdout:3/76: getdents d0/d5/da 0 2024-04-24T08:24:34.984 INFO:tasks.workunit.client.1.smithi149.stdout:4/50: write d2/f7 [4571147,16811] 0 2024-04-24T08:24:34.990 INFO:tasks.workunit.client.1.smithi149.stdout:9/97: write d1/da/f8 [1264266,89844] 0 2024-04-24T08:24:34.996 INFO:tasks.workunit.client.1.smithi149.stdout:3/77: symlink d0/l18 0 2024-04-24T08:24:34.998 INFO:tasks.workunit.client.1.smithi149.stdout:9/98: dread - d1/de/d12/f17 zero size 2024-04-24T08:24:34.999 INFO:tasks.workunit.client.1.smithi149.stdout:6/58: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:35.005 INFO:tasks.workunit.client.1.smithi149.stdout:3/78: mkdir d0/d5/da/d19 0 2024-04-24T08:24:35.006 INFO:tasks.workunit.client.1.smithi149.stdout:6/59: unlink d0/l7 0 2024-04-24T08:24:35.007 INFO:tasks.workunit.client.1.smithi149.stdout:5/69: dread f9 [0,4194304] 0 2024-04-24T08:24:35.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/79: rename d0/d7 to d0/d5/d1a 0 2024-04-24T08:24:35.021 INFO:tasks.workunit.client.1.smithi149.stdout:2/69: dwrite d0/d6/f12 [0,4194304] 0 2024-04-24T08:24:35.022 INFO:tasks.workunit.client.1.smithi149.stdout:6/60: dread d0/f1 [0,4194304] 0 2024-04-24T08:24:35.023 INFO:tasks.workunit.client.1.smithi149.stdout:3/80: write d0/fe [113991,96291] 0 2024-04-24T08:24:35.027 INFO:tasks.workunit.client.1.smithi149.stdout:4/51: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:35.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:34 smithi149 ceph-mon[31282]: mgrmap e31: smithi090.jsrfxo(active, since 13s), standbys: smithi149.vavftl 2024-04-24T08:24:35.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:34 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:35.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:34 smithi149 ceph-mon[31282]: pgmap v10: 65 pgs: 65 active+clean; 215 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1002 KiB/s rd, 2.5 MiB/s wr, 196 op/s 2024-04-24T08:24:35.032 INFO:tasks.workunit.client.1.smithi149.stdout:2/70: mknod d0/c14 0 2024-04-24T08:24:35.032 INFO:tasks.workunit.client.1.smithi149.stdout:4/52: fdatasync d2/fc 0 2024-04-24T08:24:35.036 INFO:tasks.workunit.client.1.smithi149.stdout:4/53: write d2/f4 [2912800,40253] 0 2024-04-24T08:24:35.041 INFO:tasks.workunit.client.1.smithi149.stdout:5/70: dread f7 [0,4194304] 0 2024-04-24T08:24:35.043 INFO:tasks.workunit.client.1.smithi149.stdout:3/81: mkdir d0/d5/da/d19/d1b 0 2024-04-24T08:24:35.043 INFO:tasks.workunit.client.1.smithi149.stdout:5/71: chown f9 13 1 2024-04-24T08:24:35.043 INFO:tasks.workunit.client.1.smithi149.stdout:4/54: mknod d2/cd 0 2024-04-24T08:24:35.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/71: getdents d0 0 2024-04-24T08:24:35.058 INFO:tasks.workunit.client.1.smithi149.stdout:3/82: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:35.063 INFO:tasks.workunit.client.1.smithi149.stdout:6/61: getdents d0 0 2024-04-24T08:24:35.064 INFO:tasks.workunit.client.1.smithi149.stdout:6/62: readlink d0/ld 0 2024-04-24T08:24:35.072 INFO:tasks.workunit.client.1.smithi149.stdout:3/83: mkdir d0/d5/d1a/d1c 0 2024-04-24T08:24:35.073 INFO:tasks.workunit.client.1.smithi149.stdout:3/84: read d0/f3 [344807,117160] 0 2024-04-24T08:24:35.077 INFO:tasks.workunit.client.1.smithi149.stdout:2/72: dwrite d0/d6/f7 [0,4194304] 0 2024-04-24T08:24:35.077 INFO:tasks.workunit.client.1.smithi149.stdout:4/55: truncate d2/fa 1054360 0 2024-04-24T08:24:35.078 INFO:tasks.workunit.client.1.smithi149.stdout:5/72: getdents db 0 2024-04-24T08:24:35.080 INFO:tasks.workunit.client.1.smithi149.stdout:5/73: truncate db/f10 4704752 0 2024-04-24T08:24:35.082 INFO:tasks.workunit.client.1.smithi149.stdout:5/74: chown l6 2212 1 2024-04-24T08:24:35.082 INFO:tasks.workunit.client.1.smithi149.stdout:2/73: rmdir d0/d6 39 2024-04-24T08:24:35.085 INFO:tasks.workunit.client.1.smithi149.stdout:4/56: mknod d2/ce 0 2024-04-24T08:24:35.087 INFO:tasks.workunit.client.1.smithi149.stdout:5/75: creat db/f14 x:0 0 0 2024-04-24T08:24:35.102 INFO:tasks.workunit.client.1.smithi149.stdout:4/57: dwrite d2/f4 [0,4194304] 0 2024-04-24T08:24:35.102 INFO:tasks.workunit.client.1.smithi149.stdout:0/66: sync 2024-04-24T08:24:35.105 INFO:tasks.workunit.client.1.smithi149.stdout:6/63: creat d0/f15 x:0 0 0 2024-04-24T08:24:35.106 INFO:tasks.workunit.client.1.smithi149.stdout:2/74: rmdir d0/d6/db 39 2024-04-24T08:24:35.109 INFO:tasks.workunit.client.1.smithi149.stdout:5/76: dwrite db/f14 [0,4194304] 0 2024-04-24T08:24:35.115 INFO:tasks.workunit.client.1.smithi149.stdout:0/67: link d2/l10 d2/d9/l14 0 2024-04-24T08:24:35.117 INFO:tasks.workunit.client.1.smithi149.stdout:3/85: getdents d0/d5/d1a 0 2024-04-24T08:24:35.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/77: mknod db/c15 0 2024-04-24T08:24:35.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/78: chown db/f14 901815679 1 2024-04-24T08:24:35.120 INFO:tasks.workunit.client.1.smithi149.stdout:2/75: creat d0/d6/db/f15 x:0 0 0 2024-04-24T08:24:35.123 INFO:tasks.workunit.client.1.smithi149.stdout:2/76: write d0/d6/db/f15 [655898,125617] 0 2024-04-24T08:24:35.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/77: stat d0/f2 0 2024-04-24T08:24:35.125 INFO:tasks.workunit.client.1.smithi149.stdout:0/68: mknod d2/c15 0 2024-04-24T08:24:35.126 INFO:tasks.workunit.client.1.smithi149.stdout:0/69: chown c1 0 1 2024-04-24T08:24:35.127 INFO:tasks.workunit.client.1.smithi149.stdout:0/70: fsync d2/d9/fe 0 2024-04-24T08:24:35.129 INFO:tasks.workunit.client.1.smithi149.stdout:0/71: write d2/d9/f13 [648145,59545] 0 2024-04-24T08:24:35.131 INFO:tasks.workunit.client.1.smithi149.stdout:3/86: unlink d0/d5/d1a/fd 0 2024-04-24T08:24:35.135 INFO:tasks.workunit.client.1.smithi149.stdout:5/79: dread db/fc [0,4194304] 0 2024-04-24T08:24:35.141 INFO:tasks.workunit.client.1.smithi149.stdout:0/72: creat d2/d9/df/f16 x:0 0 0 2024-04-24T08:24:35.144 INFO:tasks.workunit.client.1.smithi149.stdout:3/87: dread d0/f3 [4194304,4194304] 0 2024-04-24T08:24:35.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/80: dread db/fc [0,4194304] 0 2024-04-24T08:24:35.148 INFO:tasks.workunit.client.1.smithi149.stdout:2/78: creat d0/d6/f16 x:0 0 0 2024-04-24T08:24:35.149 INFO:tasks.workunit.client.1.smithi149.stdout:5/81: write db/f14 [2424520,79697] 0 2024-04-24T08:24:35.150 INFO:tasks.workunit.client.1.smithi149.stdout:0/73: chown d2/l10 66226 1 2024-04-24T08:24:35.151 INFO:tasks.workunit.client.1.smithi149.stdout:5/82: read db/f10 [1179596,79188] 0 2024-04-24T08:24:35.151 INFO:tasks.workunit.client.1.smithi149.stdout:5/83: readlink l6 0 2024-04-24T08:24:35.153 INFO:tasks.workunit.client.1.smithi149.stdout:0/74: truncate d2/d9/fe 452300 0 2024-04-24T08:24:35.162 INFO:tasks.workunit.client.1.smithi149.stdout:5/84: write f9 [557166,4167] 0 2024-04-24T08:24:35.164 INFO:tasks.workunit.client.1.smithi149.stdout:5/85: chown la 11717 1 2024-04-24T08:24:35.167 INFO:tasks.workunit.client.1.smithi149.stdout:2/79: dwrite d0/d6/f7 [0,4194304] 0 2024-04-24T08:24:35.175 INFO:tasks.workunit.client.1.smithi149.stdout:6/64: rename d0/f14 to d0/f16 0 2024-04-24T08:24:35.179 INFO:tasks.workunit.client.1.smithi149.stdout:2/80: rename d0/fe to d0/d6/db/f17 0 2024-04-24T08:24:35.185 INFO:tasks.workunit.client.1.smithi149.stdout:0/75: fsync d2/d9/df/f16 0 2024-04-24T08:24:35.188 INFO:tasks.workunit.client.1.smithi149.stdout:2/81: symlink d0/l18 0 2024-04-24T08:24:35.188 INFO:tasks.workunit.client.1.smithi149.stdout:5/86: dwrite f7 [0,4194304] 0 2024-04-24T08:24:35.191 INFO:tasks.workunit.client.1.smithi149.stdout:5/87: read db/f14 [1466645,8890] 0 2024-04-24T08:24:35.199 INFO:tasks.workunit.client.1.smithi149.stdout:5/88: creat db/f16 x:0 0 0 2024-04-24T08:24:35.200 INFO:tasks.workunit.client.1.smithi149.stdout:2/82: symlink d0/d6/l19 0 2024-04-24T08:24:35.203 INFO:tasks.workunit.client.1.smithi149.stdout:3/88: truncate d0/d5/f16 1443605 0 2024-04-24T08:24:35.207 INFO:tasks.workunit.client.1.smithi149.stdout:2/83: creat d0/d6/db/f1a x:0 0 0 2024-04-24T08:24:35.211 INFO:tasks.workunit.client.1.smithi149.stdout:5/89: symlink db/d12/l17 0 2024-04-24T08:24:35.222 INFO:tasks.workunit.client.1.smithi149.stdout:3/89: dread d0/fc [0,4194304] 0 2024-04-24T08:24:35.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:34 smithi090 ceph-mon[25016]: mgrmap e31: smithi090.jsrfxo(active, since 13s), standbys: smithi149.vavftl 2024-04-24T08:24:35.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:34 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:24:35.247 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:34 smithi090 ceph-mon[25016]: pgmap v10: 65 pgs: 65 active+clean; 215 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1002 KiB/s rd, 2.5 MiB/s wr, 196 op/s 2024-04-24T08:24:35.287 INFO:tasks.workunit.client.1.smithi149.stdout:5/90: creat db/f18 x:0 0 0 2024-04-24T08:24:35.287 INFO:tasks.workunit.client.1.smithi149.stdout:2/84: symlink d0/l1b 0 2024-04-24T08:24:35.288 INFO:tasks.workunit.client.1.smithi149.stdout:3/90: getdents d0/d5/d1a/d1c 0 2024-04-24T08:24:35.291 INFO:tasks.workunit.client.1.smithi149.stdout:6/65: creat d0/f17 x:0 0 0 2024-04-24T08:24:35.295 INFO:tasks.workunit.client.1.smithi149.stdout:2/85: rename d0/f5 to d0/d6/db/f1c 0 2024-04-24T08:24:35.295 INFO:tasks.workunit.client.1.smithi149.stdout:3/91: mknod d0/d5/d1a/d1c/c1d 0 2024-04-24T08:24:35.296 INFO:tasks.workunit.client.1.smithi149.stdout:6/66: mknod d0/c18 0 2024-04-24T08:24:35.298 INFO:tasks.workunit.client.1.smithi149.stdout:2/86: dread - d0/d6/db/f10 zero size 2024-04-24T08:24:35.300 INFO:tasks.workunit.client.1.smithi149.stdout:3/92: creat d0/d5/d1a/f1e x:0 0 0 2024-04-24T08:24:35.301 INFO:tasks.workunit.client.1.smithi149.stdout:6/67: symlink d0/l19 0 2024-04-24T08:24:35.301 INFO:tasks.workunit.client.1.smithi149.stdout:3/93: fsync d0/d5/ff 0 2024-04-24T08:24:35.304 INFO:tasks.workunit.client.1.smithi149.stdout:2/87: mknod d0/d6/c1d 0 2024-04-24T08:24:35.308 INFO:tasks.workunit.client.1.smithi149.stdout:2/88: symlink d0/l1e 0 2024-04-24T08:24:35.311 INFO:tasks.workunit.client.1.smithi149.stdout:5/91: dwrite db/f14 [0,4194304] 0 2024-04-24T08:24:35.311 INFO:tasks.workunit.client.1.smithi149.stdout:8/46: sync 2024-04-24T08:24:35.311 INFO:tasks.workunit.client.1.smithi149.stdout:2/89: write d0/d6/f16 [284008,100580] 0 2024-04-24T08:24:35.313 INFO:tasks.workunit.client.1.smithi149.stdout:5/92: chown c8 12432474 1 2024-04-24T08:24:35.313 INFO:tasks.workunit.client.1.smithi149.stdout:3/94: rename d0/d5/da/l15 to d0/l1f 0 2024-04-24T08:24:35.317 INFO:tasks.workunit.client.1.smithi149.stdout:5/93: symlink db/l19 0 2024-04-24T08:24:35.317 INFO:tasks.workunit.client.1.smithi149.stdout:8/47: write d1/f8 [1109882,36147] 0 2024-04-24T08:24:35.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/95: rename d0/d5/f8 to d0/d5/da/d19/f20 0 2024-04-24T08:24:35.325 INFO:tasks.workunit.client.1.smithi149.stdout:2/90: link d0/f2 d0/d6/db/f1f 0 2024-04-24T08:24:35.328 INFO:tasks.workunit.client.1.smithi149.stdout:3/96: creat d0/d5/d1a/d1c/f21 x:0 0 0 2024-04-24T08:24:35.329 INFO:tasks.workunit.client.1.smithi149.stdout:5/94: dread f9 [0,4194304] 0 2024-04-24T08:24:35.330 INFO:tasks.workunit.client.1.smithi149.stdout:3/97: stat d0/d5/ff 0 2024-04-24T08:24:35.334 INFO:tasks.workunit.client.1.smithi149.stdout:2/91: mkdir d0/d20 0 2024-04-24T08:24:35.338 INFO:tasks.workunit.client.1.smithi149.stdout:8/48: dwrite d1/f8 [0,4194304] 0 2024-04-24T08:24:35.342 INFO:tasks.workunit.client.1.smithi149.stdout:3/98: mkdir d0/d5/da/d19/d1b/d22 0 2024-04-24T08:24:35.346 INFO:tasks.workunit.client.1.smithi149.stdout:8/49: creat d1/fc x:0 0 0 2024-04-24T08:24:35.347 INFO:tasks.workunit.client.1.smithi149.stdout:3/99: symlink d0/l23 0 2024-04-24T08:24:35.349 INFO:tasks.workunit.client.1.smithi149.stdout:2/92: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:35.350 INFO:tasks.workunit.client.1.smithi149.stdout:8/50: write d1/f3 [3041988,124608] 0 2024-04-24T08:24:35.356 INFO:tasks.workunit.client.1.smithi149.stdout:2/93: creat d0/f21 x:0 0 0 2024-04-24T08:24:35.357 INFO:tasks.workunit.client.1.smithi149.stdout:8/51: creat d1/fd x:0 0 0 2024-04-24T08:24:35.358 INFO:tasks.workunit.client.1.smithi149.stdout:2/94: fdatasync d0/d6/f7 0 2024-04-24T08:24:35.358 INFO:tasks.workunit.client.1.smithi149.stdout:3/100: dread d0/f4 [8388608,4194304] 0 2024-04-24T08:24:35.359 INFO:tasks.workunit.client.1.smithi149.stdout:8/52: read d1/f3 [2668881,9375] 0 2024-04-24T08:24:35.360 INFO:tasks.workunit.client.1.smithi149.stdout:3/101: chown d0/f3 1452685 1 2024-04-24T08:24:35.361 INFO:tasks.workunit.client.1.smithi149.stdout:3/102: chown d0/d5 3619784 1 2024-04-24T08:24:35.362 INFO:tasks.workunit.client.1.smithi149.stdout:3/103: fsync d0/d5/ff 0 2024-04-24T08:24:35.366 INFO:tasks.workunit.client.1.smithi149.stdout:3/104: creat d0/d5/da/f24 x:0 0 0 2024-04-24T08:24:35.368 INFO:tasks.workunit.client.1.smithi149.stdout:3/105: write d0/d5/d1a/f1e [6179,119475] 0 2024-04-24T08:24:35.368 INFO:tasks.workunit.client.1.smithi149.stdout:2/95: link d0/d6/f16 d0/f22 0 2024-04-24T08:24:35.370 INFO:tasks.workunit.client.1.smithi149.stdout:2/96: dread - d0/d6/db/f10 zero size 2024-04-24T08:24:35.372 INFO:tasks.workunit.client.1.smithi149.stdout:3/106: mkdir d0/d5/da/d19/d1b/d25 0 2024-04-24T08:24:35.372 INFO:tasks.workunit.client.1.smithi149.stdout:3/107: readlink d0/l18 0 2024-04-24T08:24:35.373 INFO:tasks.workunit.client.1.smithi149.stdout:2/97: mknod d0/d6/c23 0 2024-04-24T08:24:35.376 INFO:tasks.workunit.client.1.smithi149.stdout:3/108: mkdir d0/d26 0 2024-04-24T08:24:35.378 INFO:tasks.workunit.client.1.smithi149.stdout:2/98: symlink d0/d20/l24 0 2024-04-24T08:24:35.381 INFO:tasks.workunit.client.1.smithi149.stdout:2/99: symlink d0/d6/db/l25 0 2024-04-24T08:24:35.400 INFO:tasks.workunit.client.1.smithi149.stdout:2/100: dwrite d0/d6/db/f17 [0,4194304] 0 2024-04-24T08:24:35.423 INFO:tasks.workunit.client.1.smithi149.stdout:2/101: dwrite d0/d6/db/f17 [0,4194304] 0 2024-04-24T08:24:35.437 INFO:tasks.workunit.client.1.smithi149.stdout:6/68: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:35.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/95: write db/fc [4446915,47115] 0 2024-04-24T08:24:35.441 INFO:tasks.workunit.client.1.smithi149.stdout:6/69: mkdir d0/d1a 0 2024-04-24T08:24:35.442 INFO:tasks.workunit.client.1.smithi149.stdout:5/96: creat db/f1a x:0 0 0 2024-04-24T08:24:35.445 INFO:tasks.workunit.client.1.smithi149.stdout:6/70: fdatasync d0/fa 0 2024-04-24T08:24:35.447 INFO:tasks.workunit.client.1.smithi149.stdout:8/53: write d1/f8 [4244754,76547] 0 2024-04-24T08:24:35.450 INFO:tasks.workunit.client.1.smithi149.stdout:6/71: chown d0/f5 3591 1 2024-04-24T08:24:35.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/102: write d0/f2 [652787,99164] 0 2024-04-24T08:24:35.468 INFO:tasks.workunit.client.1.smithi149.stdout:8/54: dread d1/f8 [0,4194304] 0 2024-04-24T08:24:35.472 INFO:tasks.workunit.client.1.smithi149.stdout:8/55: chown d1/f9 1961 1 2024-04-24T08:24:35.476 INFO:tasks.workunit.client.1.smithi149.stdout:3/109: dwrite d0/d5/da/d19/f20 [8388608,4194304] 0 2024-04-24T08:24:35.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/72: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:35.480 INFO:tasks.workunit.client.1.smithi149.stdout:1/138: sync 2024-04-24T08:24:35.480 INFO:tasks.workunit.client.1.smithi149.stdout:7/56: sync 2024-04-24T08:24:35.480 INFO:tasks.workunit.client.1.smithi149.stdout:9/99: sync 2024-04-24T08:24:35.496 INFO:tasks.workunit.client.1.smithi149.stdout:3/110: write d0/d5/d1a/f1e [848975,62000] 0 2024-04-24T08:24:35.496 INFO:tasks.workunit.client.1.smithi149.stdout:5/97: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:35.501 INFO:tasks.workunit.client.1.smithi149.stdout:1/139: creat d0/d5/db/d18/f2c x:0 0 0 2024-04-24T08:24:35.501 INFO:tasks.workunit.client.1.smithi149.stdout:5/98: dread - db/f16 zero size 2024-04-24T08:24:35.503 INFO:tasks.workunit.client.1.smithi149.stdout:2/103: dwrite d0/d6/db/f10 [0,4194304] 0 2024-04-24T08:24:35.503 INFO:tasks.workunit.client.1.smithi149.stdout:3/111: symlink d0/d5/da/d19/l27 0 2024-04-24T08:24:35.507 INFO:tasks.workunit.client.1.smithi149.stdout:8/56: getdents d1 0 2024-04-24T08:24:35.507 INFO:tasks.workunit.client.1.smithi149.stdout:5/99: fsync db/fc 0 2024-04-24T08:24:35.509 INFO:tasks.workunit.client.1.smithi149.stdout:1/140: rename d0 to d0/d5/db/d18/d2d 22 2024-04-24T08:24:35.518 INFO:tasks.workunit.client.1.smithi149.stdout:8/57: getdents d1 0 2024-04-24T08:24:35.526 INFO:tasks.workunit.client.1.smithi149.stdout:2/104: getdents d0 0 2024-04-24T08:24:35.527 INFO:tasks.workunit.client.1.smithi149.stdout:8/58: dread - d1/fd zero size 2024-04-24T08:24:35.529 INFO:tasks.workunit.client.1.smithi149.stdout:1/141: rmdir d0 39 2024-04-24T08:24:35.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/59: dread - d1/fc zero size 2024-04-24T08:24:35.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/100: dwrite d1/de/f18 [0,4194304] 0 2024-04-24T08:24:35.541 INFO:tasks.workunit.client.1.smithi149.stdout:1/142: readlink d0/l2 0 2024-04-24T08:24:35.546 INFO:tasks.workunit.client.1.smithi149.stdout:7/57: dwrite f1 [0,4194304] 0 2024-04-24T08:24:35.547 INFO:tasks.workunit.client.1.smithi149.stdout:6/73: dwrite d0/fa [0,4194304] 0 2024-04-24T08:24:35.549 INFO:tasks.workunit.client.1.smithi149.stdout:9/101: truncate d1/da/d13/f1e 567121 0 2024-04-24T08:24:35.549 INFO:tasks.workunit.client.1.smithi149.stdout:2/105: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:35.552 INFO:tasks.workunit.client.1.smithi149.stdout:5/100: dread db/f10 [0,4194304] 0 2024-04-24T08:24:35.554 INFO:tasks.workunit.client.1.smithi149.stdout:1/143: stat d0/d5/db/d18/f29 0 2024-04-24T08:24:35.555 INFO:tasks.workunit.client.1.smithi149.stdout:8/60: write d1/f9 [17208,53393] 0 2024-04-24T08:24:35.556 INFO:tasks.workunit.client.1.smithi149.stdout:7/58: rmdir d9 39 2024-04-24T08:24:35.561 INFO:tasks.workunit.client.1.smithi149.stdout:5/101: creat db/d12/f1b x:0 0 0 2024-04-24T08:24:35.563 INFO:tasks.workunit.client.1.smithi149.stdout:5/102: dread - db/f18 zero size 2024-04-24T08:24:35.565 INFO:tasks.workunit.client.1.smithi149.stdout:9/102: truncate d1/de/d12/f1d 423671 0 2024-04-24T08:24:35.566 INFO:tasks.workunit.client.1.smithi149.stdout:6/74: read d0/fe [3608705,42128] 0 2024-04-24T08:24:35.567 INFO:tasks.workunit.client.1.smithi149.stdout:2/106: creat d0/d6/db/f26 x:0 0 0 2024-04-24T08:24:35.570 INFO:tasks.workunit.client.1.smithi149.stdout:7/59: read d9/ff [1883730,1743] 0 2024-04-24T08:24:35.576 INFO:tasks.workunit.client.1.smithi149.stdout:6/75: truncate d0/f4 1087888 0 2024-04-24T08:24:35.581 INFO:tasks.workunit.client.1.smithi149.stdout:2/107: rename d0/d20 to d0/d6/db/d27 0 2024-04-24T08:24:35.586 INFO:tasks.workunit.client.1.smithi149.stdout:6/76: readlink d0/l13 0 2024-04-24T08:24:35.588 INFO:tasks.workunit.client.1.smithi149.stdout:5/103: dread db/f10 [0,4194304] 0 2024-04-24T08:24:35.592 INFO:tasks.workunit.client.1.smithi149.stdout:5/104: chown db/f16 1399 1 2024-04-24T08:24:35.592 INFO:tasks.workunit.client.1.smithi149.stdout:7/60: getdents d9/dc 0 2024-04-24T08:24:35.593 INFO:tasks.workunit.client.1.smithi149.stdout:5/105: stat db/fc 0 2024-04-24T08:24:35.593 INFO:tasks.workunit.client.1.smithi149.stdout:1/144: dread d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:35.594 INFO:tasks.workunit.client.1.smithi149.stdout:9/103: dread d1/f4 [4194304,4194304] 0 2024-04-24T08:24:35.596 INFO:tasks.workunit.client.1.smithi149.stdout:6/77: creat d0/d1a/f1b x:0 0 0 2024-04-24T08:24:35.598 INFO:tasks.workunit.client.1.smithi149.stdout:5/106: truncate db/f18 819828 0 2024-04-24T08:24:35.600 INFO:tasks.workunit.client.1.smithi149.stdout:8/61: dwrite d1/f2 [0,4194304] 0 2024-04-24T08:24:35.600 INFO:tasks.workunit.client.1.smithi149.stdout:5/107: chown db/c11 1222182 1 2024-04-24T08:24:35.601 INFO:tasks.workunit.client.1.smithi149.stdout:7/61: rmdir d9 39 2024-04-24T08:24:35.602 INFO:tasks.workunit.client.1.smithi149.stdout:5/108: write db/f16 [269788,92695] 0 2024-04-24T08:24:35.602 INFO:tasks.workunit.client.1.smithi149.stdout:2/108: dread - d0/d6/db/f26 zero size 2024-04-24T08:24:35.604 INFO:tasks.workunit.client.1.smithi149.stdout:1/145: mknod d0/d1b/c2e 0 2024-04-24T08:24:35.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/109: mkdir db/d1c 0 2024-04-24T08:24:35.609 INFO:tasks.workunit.client.1.smithi149.stdout:5/110: chown f9 101 1 2024-04-24T08:24:35.611 INFO:tasks.workunit.client.1.smithi149.stdout:9/104: rename d1/da/dd/f11 to d1/d2/d15/d1c/f20 0 2024-04-24T08:24:35.613 INFO:tasks.workunit.client.1.smithi149.stdout:2/109: mkdir d0/d6/db/d28 0 2024-04-24T08:24:35.615 INFO:tasks.workunit.client.1.smithi149.stdout:7/62: mkdir d9/dc/d11 0 2024-04-24T08:24:35.620 INFO:tasks.workunit.client.1.smithi149.stdout:5/111: mknod db/d1c/c1d 0 2024-04-24T08:24:35.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/78: getdents d0/d1a 0 2024-04-24T08:24:35.622 INFO:tasks.workunit.client.1.smithi149.stdout:2/110: dread d0/d6/f16 [0,4194304] 0 2024-04-24T08:24:35.626 INFO:tasks.workunit.client.1.smithi149.stdout:9/105: creat d1/f21 x:0 0 0 2024-04-24T08:24:35.628 INFO:tasks.workunit.client.1.smithi149.stdout:8/62: dwrite d1/f2 [0,4194304] 0 2024-04-24T08:24:35.629 INFO:tasks.workunit.client.1.smithi149.stdout:1/146: dwrite d0/da/fc [0,4194304] 0 2024-04-24T08:24:35.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/111: dread - d0/f21 zero size 2024-04-24T08:24:35.632 INFO:tasks.workunit.client.1.smithi149.stdout:5/112: dread db/f18 [0,4194304] 0 2024-04-24T08:24:35.633 INFO:tasks.workunit.client.1.smithi149.stdout:8/63: fsync d1/fc 0 2024-04-24T08:24:35.635 INFO:tasks.workunit.client.1.smithi149.stdout:5/113: chown db/d12/l17 46661 1 2024-04-24T08:24:35.639 INFO:tasks.workunit.client.1.smithi149.stdout:5/114: truncate db/d12/f1b 256609 0 2024-04-24T08:24:35.641 INFO:tasks.workunit.client.1.smithi149.stdout:9/106: fsync d1/de/f1a 0 2024-04-24T08:24:35.642 INFO:tasks.workunit.client.1.smithi149.stdout:6/79: chown d0/f10 118 1 2024-04-24T08:24:35.643 INFO:tasks.workunit.client.1.smithi149.stdout:9/107: fdatasync d1/d2/d15/d1c/f20 0 2024-04-24T08:24:35.643 INFO:tasks.workunit.client.1.smithi149.stdout:1/147: mkdir d0/d5/db/d18/d2f 0 2024-04-24T08:24:35.645 INFO:tasks.workunit.client.1.smithi149.stdout:2/112: rmdir d0/d6/db 39 2024-04-24T08:24:35.649 INFO:tasks.workunit.client.1.smithi149.stdout:8/64: dread d1/f8 [0,4194304] 0 2024-04-24T08:24:35.649 INFO:tasks.workunit.client.1.smithi149.stdout:8/65: readlink l0 0 2024-04-24T08:24:35.650 INFO:tasks.workunit.client.1.smithi149.stdout:5/115: mkdir db/d12/d1e 0 2024-04-24T08:24:35.651 INFO:tasks.workunit.client.1.smithi149.stdout:5/116: chown db 179290 1 2024-04-24T08:24:35.652 INFO:tasks.workunit.client.1.smithi149.stdout:6/80: mknod d0/d1a/c1c 0 2024-04-24T08:24:35.654 INFO:tasks.workunit.client.1.smithi149.stdout:9/108: creat d1/de/f22 x:0 0 0 2024-04-24T08:24:35.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/66: mkdir d1/de 0 2024-04-24T08:24:35.656 INFO:tasks.workunit.client.1.smithi149.stdout:5/117: read db/f10 [4349551,129102] 0 2024-04-24T08:24:35.659 INFO:tasks.workunit.client.1.smithi149.stdout:2/113: mknod d0/d6/c29 0 2024-04-24T08:24:35.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/118: mkdir db/d1f 0 2024-04-24T08:24:35.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/109: chown d1/de/d12/f17 802 1 2024-04-24T08:24:35.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/67: creat d1/de/ff x:0 0 0 2024-04-24T08:24:35.665 INFO:tasks.workunit.client.1.smithi149.stdout:2/114: dread - d0/f21 zero size 2024-04-24T08:24:35.670 INFO:tasks.workunit.client.1.smithi149.stdout:9/110: read d1/de/f18 [521501,86897] 0 2024-04-24T08:24:35.671 INFO:tasks.workunit.client.1.smithi149.stdout:7/63: dwrite d9/ff [0,4194304] 0 2024-04-24T08:24:35.671 INFO:tasks.workunit.client.1.smithi149.stdout:5/119: unlink db/ce 0 2024-04-24T08:24:35.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/81: getdents d0 0 2024-04-24T08:24:35.673 INFO:tasks.workunit.client.1.smithi149.stdout:1/148: dwrite d0/f16 [0,4194304] 0 2024-04-24T08:24:35.677 INFO:tasks.workunit.client.1.smithi149.stdout:9/111: write d1/de/d12/f17 [795620,25714] 0 2024-04-24T08:24:35.677 INFO:tasks.workunit.client.1.smithi149.stdout:8/68: truncate d1/fd 181058 0 2024-04-24T08:24:35.677 INFO:tasks.workunit.client.1.smithi149.stdout:5/120: read db/f18 [29723,54252] 0 2024-04-24T08:24:35.682 INFO:tasks.workunit.client.1.smithi149.stdout:6/82: unlink d0/f5 0 2024-04-24T08:24:35.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/69: write d1/f3 [2472358,124907] 0 2024-04-24T08:24:35.685 INFO:tasks.workunit.client.1.smithi149.stdout:8/70: truncate d1/fc 344950 0 2024-04-24T08:24:35.689 INFO:tasks.workunit.client.1.smithi149.stdout:5/121: creat db/f20 x:0 0 0 2024-04-24T08:24:35.691 INFO:tasks.workunit.client.1.smithi149.stdout:1/149: mknod d0/d1b/d2a/c30 0 2024-04-24T08:24:35.692 INFO:tasks.workunit.client.1.smithi149.stdout:5/122: chown db/d12 1028 1 2024-04-24T08:24:35.692 INFO:tasks.workunit.client.1.smithi149.stdout:7/64: creat d9/dc/de/f12 x:0 0 0 2024-04-24T08:24:35.693 INFO:tasks.workunit.client.1.smithi149.stdout:8/71: symlink d1/l10 0 2024-04-24T08:24:35.699 INFO:tasks.workunit.client.1.smithi149.stdout:8/72: readlink d1/l5 0 2024-04-24T08:24:35.709 INFO:tasks.workunit.client.1.smithi149.stdout:5/123: rename db/f18 to db/d1c/f21 0 2024-04-24T08:24:35.711 INFO:tasks.workunit.client.1.smithi149.stdout:7/65: write d9/dc/de/f12 [47495,92167] 0 2024-04-24T08:24:35.713 INFO:tasks.workunit.client.1.smithi149.stdout:8/73: symlink d1/l11 0 2024-04-24T08:24:35.715 INFO:tasks.workunit.client.1.smithi149.stdout:1/150: dread d0/d5/db/f1e [0,4194304] 0 2024-04-24T08:24:35.716 INFO:tasks.workunit.client.1.smithi149.stdout:5/124: chown c8 1536587233 1 2024-04-24T08:24:35.716 INFO:tasks.workunit.client.1.smithi149.stdout:4/58: sync 2024-04-24T08:24:35.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/76: sync 2024-04-24T08:24:35.717 INFO:tasks.workunit.client.1.smithi149.stdout:2/115: dwrite d0/d6/db/f26 [0,4194304] 0 2024-04-24T08:24:35.718 INFO:tasks.workunit.client.1.smithi149.stdout:9/112: dwrite d1/d2/f19 [0,4194304] 0 2024-04-24T08:24:35.722 INFO:tasks.workunit.client.1.smithi149.stdout:0/77: chown d2/d9 18122871 1 2024-04-24T08:24:35.724 INFO:tasks.workunit.client.1.smithi149.stdout:4/59: write d2/f9 [1419033,92228] 0 2024-04-24T08:24:35.724 INFO:tasks.workunit.client.1.smithi149.stdout:5/125: truncate db/f16 813522 0 2024-04-24T08:24:35.725 INFO:tasks.workunit.client.1.smithi149.stdout:9/113: fsync d1/de/f1a 0 2024-04-24T08:24:35.727 INFO:tasks.workunit.client.1.smithi149.stdout:4/60: write d2/f7 [686267,53375] 0 2024-04-24T08:24:35.731 INFO:tasks.workunit.client.1.smithi149.stdout:7/66: symlink d9/dc/l13 0 2024-04-24T08:24:35.733 INFO:tasks.workunit.client.1.smithi149.stdout:6/83: unlink d0/l13 0 2024-04-24T08:24:35.735 INFO:tasks.workunit.client.1.smithi149.stdout:0/78: mkdir d2/d17 0 2024-04-24T08:24:35.735 INFO:tasks.workunit.client.1.smithi149.stdout:4/61: fsync d2/f9 0 2024-04-24T08:24:35.739 INFO:tasks.workunit.client.1.smithi149.stdout:2/116: unlink d0/l1e 0 2024-04-24T08:24:35.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/79: dread - d2/d9/df/f16 zero size 2024-04-24T08:24:35.739 INFO:tasks.workunit.client.1.smithi149.stdout:4/62: chown d2/cd 478400676 1 2024-04-24T08:24:35.742 INFO:tasks.workunit.client.1.smithi149.stdout:1/151: creat d0/d5/df/d12/d19/f31 x:0 0 0 2024-04-24T08:24:35.743 INFO:tasks.workunit.client.1.smithi149.stdout:4/63: fsync d2/f4 0 2024-04-24T08:24:35.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/126: mkdir db/d12/d22 0 2024-04-24T08:24:35.750 INFO:tasks.workunit.client.1.smithi149.stdout:7/67: getdents d9/dc/d11 0 2024-04-24T08:24:35.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/80: rename d2 to d2/d9/df/d18 22 2024-04-24T08:24:35.751 INFO:tasks.workunit.client.1.smithi149.stdout:6/84: mknod d0/d1a/c1d 0 2024-04-24T08:24:35.753 INFO:tasks.workunit.client.1.smithi149.stdout:2/117: stat d0/d6/c1d 0 2024-04-24T08:24:35.754 INFO:tasks.workunit.client.1.smithi149.stdout:7/68: write d9/fd [474277,89845] 0 2024-04-24T08:24:35.755 INFO:tasks.workunit.client.1.smithi149.stdout:5/127: write db/fc [3532303,58723] 0 2024-04-24T08:24:35.763 INFO:tasks.workunit.client.1.smithi149.stdout:2/118: write d0/f8 [363594,130198] 0 2024-04-24T08:24:35.763 INFO:tasks.workunit.client.1.smithi149.stdout:8/74: dwrite d1/f9 [0,4194304] 0 2024-04-24T08:24:35.764 INFO:tasks.workunit.client.1.smithi149.stdout:0/81: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:35.770 INFO:tasks.workunit.client.1.smithi149.stdout:8/75: chown d1/f2 1391459 1 2024-04-24T08:24:35.778 INFO:tasks.workunit.client.1.smithi149.stdout:7/69: mknod d9/c14 0 2024-04-24T08:24:35.778 INFO:tasks.workunit.client.1.smithi149.stdout:2/119: write d0/d6/db/f1a [703113,88513] 0 2024-04-24T08:24:35.781 INFO:tasks.workunit.client.1.smithi149.stdout:8/76: write d1/de/ff [712198,56820] 0 2024-04-24T08:24:35.783 INFO:tasks.workunit.client.1.smithi149.stdout:2/120: truncate d0/f8 2874342 0 2024-04-24T08:24:35.785 INFO:tasks.workunit.client.1.smithi149.stdout:1/152: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:35.787 INFO:tasks.workunit.client.1.smithi149.stdout:6/85: dread d0/fb [0,4194304] 0 2024-04-24T08:24:35.801 INFO:tasks.workunit.client.1.smithi149.stdout:0/82: dwrite d2/f8 [0,4194304] 0 2024-04-24T08:24:35.816 INFO:tasks.workunit.client.1.smithi149.stdout:5/128: mkdir db/d23 0 2024-04-24T08:24:35.819 INFO:tasks.workunit.client.1.smithi149.stdout:0/83: dread d2/d9/f13 [0,4194304] 0 2024-04-24T08:24:35.822 INFO:tasks.workunit.client.1.smithi149.stdout:7/70: mknod d9/dc/de/c15 0 2024-04-24T08:24:35.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/77: symlink d1/l12 0 2024-04-24T08:24:35.826 INFO:tasks.workunit.client.1.smithi149.stdout:8/78: readlink d1/l11 0 2024-04-24T08:24:35.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/121: mknod d0/d6/db/d27/c2a 0 2024-04-24T08:24:35.837 INFO:tasks.workunit.client.1.smithi149.stdout:7/71: rename d9/fb to d9/dc/de/f16 0 2024-04-24T08:24:35.839 INFO:tasks.workunit.client.1.smithi149.stdout:8/79: rmdir d1 39 2024-04-24T08:24:35.844 INFO:tasks.workunit.client.1.smithi149.stdout:0/84: dread d2/fb [0,4194304] 0 2024-04-24T08:24:35.845 INFO:tasks.workunit.client.1.smithi149.stdout:6/86: symlink d0/l1e 0 2024-04-24T08:24:35.845 INFO:tasks.workunit.client.1.smithi149.stdout:5/129: dwrite db/d12/f1b [0,4194304] 0 2024-04-24T08:24:35.862 INFO:tasks.workunit.client.1.smithi149.stdout:8/80: write d1/f3 [3221062,29667] 0 2024-04-24T08:24:35.862 INFO:tasks.workunit.client.1.smithi149.stdout:7/72: creat d9/dc/f17 x:0 0 0 2024-04-24T08:24:35.864 INFO:tasks.workunit.client.1.smithi149.stdout:6/87: chown d0/f16 0 1 2024-04-24T08:24:35.867 INFO:tasks.workunit.client.1.smithi149.stdout:0/85: mknod d2/d9/c19 0 2024-04-24T08:24:35.872 INFO:tasks.workunit.client.1.smithi149.stdout:3/112: sync 2024-04-24T08:24:35.877 INFO:tasks.workunit.client.1.smithi149.stdout:5/130: mknod db/d12/d22/c24 0 2024-04-24T08:24:35.878 INFO:tasks.workunit.client.1.smithi149.stdout:0/86: readlink d2/d9/l14 0 2024-04-24T08:24:35.880 INFO:tasks.workunit.client.1.smithi149.stdout:7/73: dread d9/ff [0,4194304] 0 2024-04-24T08:24:35.883 INFO:tasks.workunit.client.1.smithi149.stdout:3/113: creat d0/d5/da/f28 x:0 0 0 2024-04-24T08:24:35.885 INFO:tasks.workunit.client.1.smithi149.stdout:0/87: mkdir d2/d9/d1a 0 2024-04-24T08:24:35.886 INFO:tasks.workunit.client.1.smithi149.stdout:6/88: creat d0/f1f x:0 0 0 2024-04-24T08:24:35.888 INFO:tasks.workunit.client.1.smithi149.stdout:7/74: creat d9/dc/f18 x:0 0 0 2024-04-24T08:24:35.888 INFO:tasks.workunit.client.1.smithi149.stdout:5/131: dread db/f16 [0,4194304] 0 2024-04-24T08:24:35.890 INFO:tasks.workunit.client.1.smithi149.stdout:0/88: write d2/d9/df/f16 [531760,22046] 0 2024-04-24T08:24:35.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/114: creat d0/d5/f29 x:0 0 0 2024-04-24T08:24:35.893 INFO:tasks.workunit.client.1.smithi149.stdout:3/115: chown d0/d5/d1a/d1c/c1d 38896290 1 2024-04-24T08:24:35.894 INFO:tasks.workunit.client.1.smithi149.stdout:7/75: creat d9/f19 x:0 0 0 2024-04-24T08:24:35.895 INFO:tasks.workunit.client.1.smithi149.stdout:3/116: read d0/d5/da/d19/f20 [8805045,87094] 0 2024-04-24T08:24:35.896 INFO:tasks.workunit.client.1.smithi149.stdout:5/132: rename la to db/d1f/l25 0 2024-04-24T08:24:35.897 INFO:tasks.workunit.client.1.smithi149.stdout:3/117: dread - d0/d5/da/f28 zero size 2024-04-24T08:24:35.904 INFO:tasks.workunit.client.1.smithi149.stdout:7/76: creat d9/dc/de/f1a x:0 0 0 2024-04-24T08:24:35.912 INFO:tasks.workunit.client.1.smithi149.stdout:7/77: chown d9/dc/d11 0 1 2024-04-24T08:24:35.912 INFO:tasks.workunit.client.1.smithi149.stdout:5/133: fsync db/d1c/f21 0 2024-04-24T08:24:35.914 INFO:tasks.workunit.client.1.smithi149.stdout:3/118: link d0/d5/ff d0/d26/f2a 0 2024-04-24T08:24:35.921 INFO:tasks.workunit.client.1.smithi149.stdout:1/153: dwrite d0/d5/f8 [4194304,4194304] 0 2024-04-24T08:24:35.924 INFO:tasks.workunit.client.1.smithi149.stdout:7/78: chown d9/dc/de/f1a 367 1 2024-04-24T08:24:35.928 INFO:tasks.workunit.client.1.smithi149.stdout:3/119: dread - d0/d5/da/f14 zero size 2024-04-24T08:24:35.929 INFO:tasks.workunit.client.1.smithi149.stdout:2/122: truncate d0/d6/db/f1a 776413 0 2024-04-24T08:24:35.932 INFO:tasks.workunit.client.1.smithi149.stdout:0/89: dwrite d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:35.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/81: truncate d1/f3 2447858 0 2024-04-24T08:24:35.936 INFO:tasks.workunit.client.1.smithi149.stdout:6/89: dwrite d0/fa [4194304,4194304] 0 2024-04-24T08:24:35.938 INFO:tasks.workunit.client.1.smithi149.stdout:3/120: write d0/d5/f29 [252509,350] 0 2024-04-24T08:24:35.939 INFO:tasks.workunit.client.1.smithi149.stdout:2/123: fsync d0/d6/db/f10 0 2024-04-24T08:24:35.944 INFO:tasks.workunit.client.1.smithi149.stdout:7/79: creat d9/dc/f1b x:0 0 0 2024-04-24T08:24:35.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/82: dread d1/fc [0,4194304] 0 2024-04-24T08:24:35.948 INFO:tasks.workunit.client.1.smithi149.stdout:6/90: symlink d0/d1a/l20 0 2024-04-24T08:24:35.951 INFO:tasks.workunit.client.1.smithi149.stdout:5/134: dread f7 [0,4194304] 0 2024-04-24T08:24:35.951 INFO:tasks.workunit.client.1.smithi149.stdout:0/90: creat d2/d9/df/f1b x:0 0 0 2024-04-24T08:24:35.952 INFO:tasks.workunit.client.1.smithi149.stdout:9/114: sync 2024-04-24T08:24:35.956 INFO:tasks.workunit.client.1.smithi149.stdout:5/135: rename db/d12 to db/d12/d1e/d26 22 2024-04-24T08:24:35.956 INFO:tasks.workunit.client.1.smithi149.stdout:5/136: stat db/fc 0 2024-04-24T08:24:35.956 INFO:tasks.workunit.client.1.smithi149.stdout:4/64: sync 2024-04-24T08:24:35.957 INFO:tasks.workunit.client.1.smithi149.stdout:5/137: read db/fc [4432083,119894] 0 2024-04-24T08:24:35.958 INFO:tasks.workunit.client.1.smithi149.stdout:2/124: write d0/f2 [2863639,67124] 0 2024-04-24T08:24:35.960 INFO:tasks.workunit.client.1.smithi149.stdout:5/138: chown db/f14 91915163 1 2024-04-24T08:24:35.964 INFO:tasks.workunit.client.1.smithi149.stdout:1/154: symlink d0/d5/l32 0 2024-04-24T08:24:35.964 INFO:tasks.workunit.client.1.smithi149.stdout:7/80: symlink d9/dc/de/l1c 0 2024-04-24T08:24:35.965 INFO:tasks.workunit.client.1.smithi149.stdout:5/139: dread db/fc [4194304,4194304] 0 2024-04-24T08:24:35.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/83: symlink d1/l13 0 2024-04-24T08:24:35.969 INFO:tasks.workunit.client.1.smithi149.stdout:6/91: mknod d0/d1a/c21 0 2024-04-24T08:24:35.969 INFO:tasks.workunit.client.1.smithi149.stdout:0/91: mknod d2/d9/c1c 0 2024-04-24T08:24:35.970 INFO:tasks.workunit.client.1.smithi149.stdout:8/84: chown d1/f8 5449 1 2024-04-24T08:24:35.971 INFO:tasks.workunit.client.1.smithi149.stdout:0/92: chown d2/d9/c1c 1003 1 2024-04-24T08:24:35.973 INFO:tasks.workunit.client.1.smithi149.stdout:9/115: creat d1/d2/d15/f23 x:0 0 0 2024-04-24T08:24:35.973 INFO:tasks.workunit.client.1.smithi149.stdout:0/93: write d2/fa [2417632,53831] 0 2024-04-24T08:24:35.975 INFO:tasks.workunit.client.1.smithi149.stdout:0/94: readlink d2/d9/l14 0 2024-04-24T08:24:35.977 INFO:tasks.workunit.client.1.smithi149.stdout:9/116: truncate d1/d2/d15/f23 354100 0 2024-04-24T08:24:35.978 INFO:tasks.workunit.client.1.smithi149.stdout:4/65: dread d2/f4 [0,4194304] 0 2024-04-24T08:24:35.983 INFO:tasks.workunit.client.1.smithi149.stdout:1/155: creat d0/d5/df/d12/d19/f33 x:0 0 0 2024-04-24T08:24:35.988 INFO:tasks.workunit.client.1.smithi149.stdout:8/85: unlink d1/cb 0 2024-04-24T08:24:35.990 INFO:tasks.workunit.client.1.smithi149.stdout:3/121: dwrite d0/d26/f2a [0,4194304] 0 2024-04-24T08:24:35.992 INFO:tasks.workunit.client.1.smithi149.stdout:9/117: rmdir d1/da/dd 39 2024-04-24T08:24:35.996 INFO:tasks.workunit.client.1.smithi149.stdout:6/92: symlink d0/l22 0 2024-04-24T08:24:36.000 INFO:tasks.workunit.client.1.smithi149.stdout:0/95: dread d2/fa [0,4194304] 0 2024-04-24T08:24:36.000 INFO:tasks.workunit.client.1.smithi149.stdout:1/156: mkdir d0/d5/db/d18/d34 0 2024-04-24T08:24:36.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/157: chown d0/d5/db/d18/d2f 45817 1 2024-04-24T08:24:36.007 INFO:tasks.workunit.client.1.smithi149.stdout:5/140: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:36.008 INFO:tasks.workunit.client.1.smithi149.stdout:3/122: mkdir d0/d5/da/d19/d1b/d2b 0 2024-04-24T08:24:36.011 INFO:tasks.workunit.client.1.smithi149.stdout:1/158: fsync d0/da/fc 0 2024-04-24T08:24:36.012 INFO:tasks.workunit.client.1.smithi149.stdout:0/96: fdatasync d2/d9/fe 0 2024-04-24T08:24:36.015 INFO:tasks.workunit.client.1.smithi149.stdout:1/159: chown d0/f25 95 1 2024-04-24T08:24:36.015 INFO:tasks.workunit.client.1.smithi149.stdout:9/118: creat d1/da/dd/f24 x:0 0 0 2024-04-24T08:24:36.015 INFO:tasks.workunit.client.1.smithi149.stdout:7/81: dwrite f1 [0,4194304] 0 2024-04-24T08:24:36.031 INFO:tasks.workunit.client.1.smithi149.stdout:3/123: creat d0/d5/da/d19/d1b/f2c x:0 0 0 2024-04-24T08:24:36.040 INFO:tasks.workunit.client.1.smithi149.stdout:2/125: truncate d0/d6/db/f1c 1128189 0 2024-04-24T08:24:36.043 INFO:tasks.workunit.client.1.smithi149.stdout:8/86: truncate d1/f2 1892920 0 2024-04-24T08:24:36.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/66: truncate d2/f7 4237375 0 2024-04-24T08:24:36.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/124: creat d0/d5/d1a/f2d x:0 0 0 2024-04-24T08:24:36.047 INFO:tasks.workunit.client.1.smithi149.stdout:6/93: dwrite d0/f12 [0,4194304] 0 2024-04-24T08:24:36.049 INFO:tasks.workunit.client.1.smithi149.stdout:4/67: fdatasync d2/fc 0 2024-04-24T08:24:36.051 INFO:tasks.workunit.client.1.smithi149.stdout:0/97: dread d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:36.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/68: dread - d2/fb zero size 2024-04-24T08:24:36.052 INFO:tasks.workunit.client.1.smithi149.stdout:2/126: fdatasync d0/d6/f16 0 2024-04-24T08:24:36.055 INFO:tasks.workunit.client.1.smithi149.stdout:1/160: rmdir d0/d5/df/d12/d27 0 2024-04-24T08:24:36.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/94: dread - d0/d1a/f1b zero size 2024-04-24T08:24:36.056 INFO:tasks.workunit.client.1.smithi149.stdout:9/119: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:36.056 INFO:tasks.workunit.client.1.smithi149.stdout:5/141: write db/f16 [288552,30890] 0 2024-04-24T08:24:36.060 INFO:tasks.workunit.client.1.smithi149.stdout:7/82: dwrite d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:24:36.063 INFO:tasks.workunit.client.1.smithi149.stdout:0/98: fdatasync d2/d9/f13 0 2024-04-24T08:24:36.070 INFO:tasks.workunit.client.1.smithi149.stdout:0/99: dread d2/d9/f13 [0,4194304] 0 2024-04-24T08:24:36.073 INFO:tasks.workunit.client.1.smithi149.stdout:0/100: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:36.074 INFO:tasks.workunit.client.1.smithi149.stdout:8/87: rmdir d1 39 2024-04-24T08:24:36.077 INFO:tasks.workunit.client.1.smithi149.stdout:3/125: dwrite d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:36.082 INFO:tasks.workunit.client.1.smithi149.stdout:0/101: write d2/f8 [4334743,4516] 0 2024-04-24T08:24:36.086 INFO:tasks.workunit.client.1.smithi149.stdout:5/142: unlink f7 0 2024-04-24T08:24:36.089 INFO:tasks.workunit.client.1.smithi149.stdout:7/83: chown c8 35 1 2024-04-24T08:24:36.094 INFO:tasks.workunit.client.1.smithi149.stdout:6/95: dwrite d0/f17 [0,4194304] 0 2024-04-24T08:24:36.094 INFO:tasks.workunit.client.1.smithi149.stdout:1/161: symlink d0/d5/df/d12/l35 0 2024-04-24T08:24:36.096 INFO:tasks.workunit.client.1.smithi149.stdout:6/96: dread - d0/f15 zero size 2024-04-24T08:24:36.104 INFO:tasks.workunit.client.1.smithi149.stdout:4/69: write d2/fa [1267314,108220] 0 2024-04-24T08:24:36.106 INFO:tasks.workunit.client.1.smithi149.stdout:8/88: fdatasync d1/de/ff 0 2024-04-24T08:24:36.107 INFO:tasks.workunit.client.1.smithi149.stdout:8/89: stat d1/de/ff 0 2024-04-24T08:24:36.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/90: chown d1/l13 7 1 2024-04-24T08:24:36.110 INFO:tasks.workunit.client.1.smithi149.stdout:9/120: mknod d1/da/c25 0 2024-04-24T08:24:36.113 INFO:tasks.workunit.client.1.smithi149.stdout:3/126: symlink d0/d5/da/d19/l2e 0 2024-04-24T08:24:36.115 INFO:tasks.workunit.client.1.smithi149.stdout:0/102: creat d2/d9/df/f1d x:0 0 0 2024-04-24T08:24:36.115 INFO:tasks.workunit.client.1.smithi149.stdout:3/127: chown d0/d5/da/d19/d1b/d22 298047 1 2024-04-24T08:24:36.122 INFO:tasks.workunit.client.1.smithi149.stdout:4/70: dwrite d2/fc [0,4194304] 0 2024-04-24T08:24:36.123 INFO:tasks.workunit.client.1.smithi149.stdout:1/162: creat d0/da/f36 x:0 0 0 2024-04-24T08:24:36.125 INFO:tasks.workunit.client.1.smithi149.stdout:6/97: mknod d0/d1a/c23 0 2024-04-24T08:24:36.138 INFO:tasks.workunit.client.1.smithi149.stdout:5/143: write db/d1c/f21 [1618899,114297] 0 2024-04-24T08:24:36.140 INFO:tasks.workunit.client.1.smithi149.stdout:0/103: fdatasync d2/fa 0 2024-04-24T08:24:36.142 INFO:tasks.workunit.client.1.smithi149.stdout:3/128: fsync d0/d5/da/d19/f20 0 2024-04-24T08:24:36.144 INFO:tasks.workunit.client.1.smithi149.stdout:7/84: getdents d9/dc/d11 0 2024-04-24T08:24:36.144 INFO:tasks.workunit.client.1.smithi149.stdout:3/129: dread - d0/d5/d1a/d1c/f21 zero size 2024-04-24T08:24:36.147 INFO:tasks.workunit.client.1.smithi149.stdout:4/71: creat d2/ff x:0 0 0 2024-04-24T08:24:36.148 INFO:tasks.workunit.client.1.smithi149.stdout:4/72: fsync d2/f9 0 2024-04-24T08:24:36.153 INFO:tasks.workunit.client.1.smithi149.stdout:2/127: link d0/d6/db/d27/l24 d0/l2b 0 2024-04-24T08:24:36.156 INFO:tasks.workunit.client.1.smithi149.stdout:3/130: dread d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:36.159 INFO:tasks.workunit.client.1.smithi149.stdout:3/131: dread d0/fe [0,4194304] 0 2024-04-24T08:24:36.160 INFO:tasks.workunit.client.1.smithi149.stdout:3/132: chown d0/fe 4 1 2024-04-24T08:24:36.162 INFO:tasks.workunit.client.1.smithi149.stdout:3/133: dread - d0/d5/d1a/d1c/f21 zero size 2024-04-24T08:24:36.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/85: symlink d9/l1d 0 2024-04-24T08:24:36.165 INFO:tasks.workunit.client.1.smithi149.stdout:7/86: chown d9/c10 1396450 1 2024-04-24T08:24:36.167 INFO:tasks.workunit.client.1.smithi149.stdout:5/144: dread db/fc [0,4194304] 0 2024-04-24T08:24:36.168 INFO:tasks.workunit.client.1.smithi149.stdout:4/73: mkdir d2/d10 0 2024-04-24T08:24:36.170 INFO:tasks.workunit.client.1.smithi149.stdout:1/163: mknod d0/d5/db/d18/d34/c37 0 2024-04-24T08:24:36.170 INFO:tasks.workunit.client.1.smithi149.stdout:6/98: fdatasync d0/f1 0 2024-04-24T08:24:36.172 INFO:tasks.workunit.client.1.smithi149.stdout:5/145: dread db/f16 [0,4194304] 0 2024-04-24T08:24:36.172 INFO:tasks.workunit.client.1.smithi149.stdout:6/99: dread - d0/d1a/f1b zero size 2024-04-24T08:24:36.174 INFO:tasks.workunit.client.1.smithi149.stdout:6/100: chown d0/d1a/c1d 57359 1 2024-04-24T08:24:36.176 INFO:tasks.workunit.client.1.smithi149.stdout:9/121: rename d1/de/d12/f17 to d1/f26 0 2024-04-24T08:24:36.178 INFO:tasks.workunit.client.1.smithi149.stdout:7/87: dread d9/ff [0,4194304] 0 2024-04-24T08:24:36.179 INFO:tasks.workunit.client.1.smithi149.stdout:0/104: mknod d2/d17/c1e 0 2024-04-24T08:24:36.182 INFO:tasks.workunit.client.1.smithi149.stdout:3/134: mknod d0/d5/c2f 0 2024-04-24T08:24:36.183 INFO:tasks.workunit.client.1.smithi149.stdout:3/135: stat d0/f4 0 2024-04-24T08:24:36.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/146: dread f9 [0,4194304] 0 2024-04-24T08:24:36.190 INFO:tasks.workunit.client.1.smithi149.stdout:1/164: symlink d0/d1b/l38 0 2024-04-24T08:24:36.191 INFO:tasks.workunit.client.1.smithi149.stdout:2/128: creat d0/d6/db/d28/f2c x:0 0 0 2024-04-24T08:24:36.193 INFO:tasks.workunit.client.1.smithi149.stdout:2/129: chown d0/d6/db/d28 69845021 1 2024-04-24T08:24:36.195 INFO:tasks.workunit.client.1.smithi149.stdout:6/101: write d0/f11 [949949,129154] 0 2024-04-24T08:24:36.196 INFO:tasks.workunit.client.1.smithi149.stdout:9/122: unlink d1/f21 0 2024-04-24T08:24:36.199 INFO:tasks.workunit.client.1.smithi149.stdout:3/136: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:36.201 INFO:tasks.workunit.client.1.smithi149.stdout:7/88: rename d9/ff to d9/dc/d11/f1e 0 2024-04-24T08:24:36.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/91: link d1/f3 d1/f14 0 2024-04-24T08:24:36.204 INFO:tasks.workunit.client.1.smithi149.stdout:8/92: stat d1/l12 0 2024-04-24T08:24:36.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/74: mknod d2/d10/c11 0 2024-04-24T08:24:36.219 INFO:tasks.workunit.client.1.smithi149.stdout:6/102: symlink d0/d1a/l24 0 2024-04-24T08:24:36.227 INFO:tasks.workunit.client.1.smithi149.stdout:5/147: link db/f1a db/d12/d1e/f27 0 2024-04-24T08:24:36.227 INFO:tasks.workunit.client.1.smithi149.stdout:4/75: symlink d2/l12 0 2024-04-24T08:24:36.230 INFO:tasks.workunit.client.1.smithi149.stdout:0/105: dwrite d2/fb [0,4194304] 0 2024-04-24T08:24:36.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/106: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:36.236 INFO:tasks.workunit.client.1.smithi149.stdout:6/103: creat d0/d1a/f25 x:0 0 0 2024-04-24T08:24:36.237 INFO:tasks.workunit.client.1.smithi149.stdout:1/165: dwrite d0/d5/db/f1e [0,4194304] 0 2024-04-24T08:24:36.238 INFO:tasks.workunit.client.1.smithi149.stdout:6/104: write d0/f11 [225191,102163] 0 2024-04-24T08:24:36.239 INFO:tasks.workunit.client.1.smithi149.stdout:3/137: creat d0/d5/da/d19/d1b/d25/f30 x:0 0 0 2024-04-24T08:24:36.243 INFO:tasks.workunit.client.1.smithi149.stdout:3/138: chown d0/c17 3836 1 2024-04-24T08:24:36.244 INFO:tasks.workunit.client.1.smithi149.stdout:2/130: creat d0/f2d x:0 0 0 2024-04-24T08:24:36.244 INFO:tasks.workunit.client.1.smithi149.stdout:2/131: stat d0/d6/f16 0 2024-04-24T08:24:36.246 INFO:tasks.workunit.client.1.smithi149.stdout:4/76: mkdir d2/d13 0 2024-04-24T08:24:36.248 INFO:tasks.workunit.client.1.smithi149.stdout:5/148: mknod db/d12/c28 0 2024-04-24T08:24:36.251 INFO:tasks.workunit.client.1.smithi149.stdout:9/123: rename d1/f4 to d1/da/f27 0 2024-04-24T08:24:36.252 INFO:tasks.workunit.client.1.smithi149.stdout:9/124: write d1/d2/d15/f23 [873088,30964] 0 2024-04-24T08:24:36.262 INFO:tasks.workunit.client.1.smithi149.stdout:0/107: chown d2/lc 22971766 1 2024-04-24T08:24:36.263 INFO:tasks.workunit.client.1.smithi149.stdout:9/125: dread d1/da/fc [0,4194304] 0 2024-04-24T08:24:36.266 INFO:tasks.workunit.client.1.smithi149.stdout:9/126: write d1/d2/d15/d1c/f20 [1730613,12885] 0 2024-04-24T08:24:36.269 INFO:tasks.workunit.client.1.smithi149.stdout:1/166: unlink d0/d5/db/d18/f26 0 2024-04-24T08:24:36.272 INFO:tasks.workunit.client.1.smithi149.stdout:3/139: fsync d0/d5/da/f24 0 2024-04-24T08:24:36.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/89: rmdir d9/dc 39 2024-04-24T08:24:36.274 INFO:tasks.workunit.client.1.smithi149.stdout:7/90: read - d9/f19 zero size 2024-04-24T08:24:36.275 INFO:tasks.workunit.client.1.smithi149.stdout:3/140: write d0/d5/d1a/f2d [475189,86777] 0 2024-04-24T08:24:36.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:35 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:36.280 INFO:tasks.workunit.client.1.smithi149.stdout:5/149: symlink db/d12/d22/l29 0 2024-04-24T08:24:36.287 INFO:tasks.workunit.client.1.smithi149.stdout:0/108: creat d2/d17/f1f x:0 0 0 2024-04-24T08:24:36.289 INFO:tasks.workunit.client.1.smithi149.stdout:3/141: dread d0/fc [0,4194304] 0 2024-04-24T08:24:36.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:35 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:36.293 INFO:tasks.workunit.client.1.smithi149.stdout:9/127: symlink d1/d2/d15/l28 0 2024-04-24T08:24:36.299 INFO:tasks.workunit.client.1.smithi149.stdout:4/77: link d2/ff d2/f14 0 2024-04-24T08:24:36.300 INFO:tasks.workunit.client.1.smithi149.stdout:4/78: chown d2/ff 69 1 2024-04-24T08:24:36.302 INFO:tasks.workunit.client.1.smithi149.stdout:4/79: write d2/fa [2354169,19086] 0 2024-04-24T08:24:36.304 INFO:tasks.workunit.client.1.smithi149.stdout:5/150: creat db/d1f/f2a x:0 0 0 2024-04-24T08:24:36.311 INFO:tasks.workunit.client.1.smithi149.stdout:6/105: creat d0/f26 x:0 0 0 2024-04-24T08:24:36.323 INFO:tasks.workunit.client.1.smithi149.stdout:5/151: dwrite db/d1c/f21 [0,4194304] 0 2024-04-24T08:24:36.335 INFO:tasks.workunit.client.1.smithi149.stdout:7/91: rename c5 to d9/dc/c1f 0 2024-04-24T08:24:36.335 INFO:tasks.workunit.client.1.smithi149.stdout:2/132: link d0/d6/db/d27/l24 d0/d6/db/d27/l2e 0 2024-04-24T08:24:36.337 INFO:tasks.workunit.client.1.smithi149.stdout:0/109: rmdir d2/d17 39 2024-04-24T08:24:36.339 INFO:tasks.workunit.client.1.smithi149.stdout:0/110: write d2/fa [2176348,25335] 0 2024-04-24T08:24:36.341 INFO:tasks.workunit.client.1.smithi149.stdout:9/128: creat d1/da/f29 x:0 0 0 2024-04-24T08:24:36.342 INFO:tasks.workunit.client.1.smithi149.stdout:0/111: read d2/d9/df/f16 [3363937,70113] 0 2024-04-24T08:24:36.343 INFO:tasks.workunit.client.1.smithi149.stdout:9/129: stat d1/de/d12/f1d 0 2024-04-24T08:24:36.345 INFO:tasks.workunit.client.1.smithi149.stdout:9/130: stat d1/de 0 2024-04-24T08:24:36.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/131: fsync d1/de/f22 0 2024-04-24T08:24:36.349 INFO:tasks.workunit.client.1.smithi149.stdout:4/80: rename d2/fa to d2/d13/f15 0 2024-04-24T08:24:36.349 INFO:tasks.workunit.client.1.smithi149.stdout:4/81: rename d2 to d2/d16 22 2024-04-24T08:24:36.351 INFO:tasks.workunit.client.1.smithi149.stdout:7/92: rmdir d9/dc/d11 39 2024-04-24T08:24:36.354 INFO:tasks.workunit.client.1.smithi149.stdout:2/133: symlink d0/d6/l2f 0 2024-04-24T08:24:36.362 INFO:tasks.workunit.client.1.smithi149.stdout:6/106: dread d0/f10 [0,4194304] 0 2024-04-24T08:24:36.365 INFO:tasks.workunit.client.1.smithi149.stdout:2/134: mknod d0/d6/db/c30 0 2024-04-24T08:24:36.368 INFO:tasks.workunit.client.1.smithi149.stdout:5/152: getdents db/d12/d22 0 2024-04-24T08:24:36.369 INFO:tasks.workunit.client.1.smithi149.stdout:1/167: truncate d0/da/f24 329366 0 2024-04-24T08:24:36.371 INFO:tasks.workunit.client.1.smithi149.stdout:1/168: write d0/fe [3774660,95698] 0 2024-04-24T08:24:36.371 INFO:tasks.workunit.client.1.smithi149.stdout:4/82: link d2/f4 d2/d10/f17 0 2024-04-24T08:24:36.372 INFO:tasks.workunit.client.1.smithi149.stdout:6/107: creat d0/d1a/f27 x:0 0 0 2024-04-24T08:24:36.372 INFO:tasks.workunit.client.1.smithi149.stdout:1/169: stat d0/d5/df/l15 0 2024-04-24T08:24:36.373 INFO:tasks.workunit.client.1.smithi149.stdout:6/108: stat d0/d1a/c21 0 2024-04-24T08:24:36.374 INFO:tasks.workunit.client.1.smithi149.stdout:1/170: chown d0/d5/db/f1c 151819850 1 2024-04-24T08:24:36.375 INFO:tasks.workunit.client.1.smithi149.stdout:7/93: mknod d9/dc/d11/c20 0 2024-04-24T08:24:36.376 INFO:tasks.workunit.client.1.smithi149.stdout:2/135: chown d0/f21 3957 1 2024-04-24T08:24:36.377 INFO:tasks.workunit.client.1.smithi149.stdout:0/112: link d2/d9/l14 d2/d9/df/l20 0 2024-04-24T08:24:36.378 INFO:tasks.workunit.client.1.smithi149.stdout:2/136: chown d0/d6/f16 6569 1 2024-04-24T08:24:36.380 INFO:tasks.workunit.client.1.smithi149.stdout:5/153: truncate db/fc 3736891 0 2024-04-24T08:24:36.382 INFO:tasks.workunit.client.1.smithi149.stdout:3/142: truncate d0/f3 9096082 0 2024-04-24T08:24:36.383 INFO:tasks.workunit.client.1.smithi149.stdout:3/143: read d0/fc [2120172,34931] 0 2024-04-24T08:24:36.384 INFO:tasks.workunit.client.1.smithi149.stdout:1/171: mkdir d0/d39 0 2024-04-24T08:24:36.385 INFO:tasks.workunit.client.1.smithi149.stdout:1/172: readlink d0/d5/df/d12/l35 0 2024-04-24T08:24:36.391 INFO:tasks.workunit.client.1.smithi149.stdout:2/137: creat d0/d6/db/d27/f31 x:0 0 0 2024-04-24T08:24:36.400 INFO:tasks.workunit.client.1.smithi149.stdout:9/132: write d1/de/f18 [412968,90814] 0 2024-04-24T08:24:36.401 INFO:tasks.workunit.client.1.smithi149.stdout:1/173: creat d0/d5/db/d18/f3a x:0 0 0 2024-04-24T08:24:36.404 INFO:tasks.workunit.client.1.smithi149.stdout:0/113: link d2/f8 d2/d9/df/f21 0 2024-04-24T08:24:36.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/138: symlink d0/d6/db/l32 0 2024-04-24T08:24:36.407 INFO:tasks.workunit.client.1.smithi149.stdout:2/139: readlink d0/d6/lc 0 2024-04-24T08:24:36.408 INFO:tasks.workunit.client.1.smithi149.stdout:3/144: symlink d0/d5/da/d19/d1b/d2b/l31 0 2024-04-24T08:24:36.411 INFO:tasks.workunit.client.1.smithi149.stdout:3/145: fsync d0/d5/d1a/f2d 0 2024-04-24T08:24:36.412 INFO:tasks.workunit.client.1.smithi149.stdout:9/133: creat d1/d2/d15/d1c/f2a x:0 0 0 2024-04-24T08:24:36.413 INFO:tasks.workunit.client.1.smithi149.stdout:4/83: dread d2/f7 [0,4194304] 0 2024-04-24T08:24:36.415 INFO:tasks.workunit.client.1.smithi149.stdout:7/94: rmdir d9 39 2024-04-24T08:24:36.425 INFO:tasks.workunit.client.1.smithi149.stdout:3/146: mknod d0/d5/c32 0 2024-04-24T08:24:36.426 INFO:tasks.workunit.client.1.smithi149.stdout:9/134: creat d1/d2/f2b x:0 0 0 2024-04-24T08:24:36.426 INFO:tasks.workunit.client.1.smithi149.stdout:2/140: dread d0/d6/db/f1f [0,4194304] 0 2024-04-24T08:24:36.430 INFO:tasks.workunit.client.1.smithi149.stdout:0/114: rmdir d2/d17 39 2024-04-24T08:24:36.431 INFO:tasks.workunit.client.1.smithi149.stdout:5/154: write db/f16 [364643,130883] 0 2024-04-24T08:24:36.435 INFO:tasks.workunit.client.1.smithi149.stdout:0/115: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:36.435 INFO:tasks.workunit.client.1.smithi149.stdout:3/147: dread d0/d5/d1a/f1e [0,4194304] 0 2024-04-24T08:24:36.440 INFO:tasks.workunit.client.1.smithi149.stdout:3/148: dread - d0/d5/da/d19/d1b/f2c zero size 2024-04-24T08:24:36.443 INFO:tasks.workunit.client.1.smithi149.stdout:6/109: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:36.445 INFO:tasks.workunit.client.1.smithi149.stdout:5/155: creat db/f2b x:0 0 0 2024-04-24T08:24:36.447 INFO:tasks.workunit.client.1.smithi149.stdout:5/156: truncate db/f20 560339 0 2024-04-24T08:24:36.449 INFO:tasks.workunit.client.1.smithi149.stdout:1/174: link d0/d5/df/d12/l35 d0/d5/df/l3b 0 2024-04-24T08:24:36.453 INFO:tasks.workunit.client.1.smithi149.stdout:0/116: dread d2/fb [0,4194304] 0 2024-04-24T08:24:36.463 INFO:tasks.workunit.client.1.smithi149.stdout:3/149: rename d0/d5/d1a/d1c/f21 to d0/d5/da/d19/d1b/d2b/f33 0 2024-04-24T08:24:36.465 INFO:tasks.workunit.client.1.smithi149.stdout:4/84: write d2/d10/f17 [244851,40240] 0 2024-04-24T08:24:36.469 INFO:tasks.workunit.client.1.smithi149.stdout:5/157: mkdir db/d12/d22/d2c 0 2024-04-24T08:24:36.469 INFO:tasks.workunit.client.1.smithi149.stdout:6/110: dwrite d0/d1a/f27 [0,4194304] 0 2024-04-24T08:24:36.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/111: truncate d0/f26 647946 0 2024-04-24T08:24:36.481 INFO:tasks.workunit.client.1.smithi149.stdout:9/135: creat d1/da/f2c x:0 0 0 2024-04-24T08:24:36.482 INFO:tasks.workunit.client.1.smithi149.stdout:1/175: symlink d0/d5/db/d18/l3c 0 2024-04-24T08:24:36.484 INFO:tasks.workunit.client.1.smithi149.stdout:7/95: creat d9/f21 x:0 0 0 2024-04-24T08:24:36.484 INFO:tasks.workunit.client.1.smithi149.stdout:9/136: read - d1/d2/f2b zero size 2024-04-24T08:24:36.487 INFO:tasks.workunit.client.1.smithi149.stdout:1/176: chown d0/d5/f8 7449 1 2024-04-24T08:24:36.488 INFO:tasks.workunit.client.1.smithi149.stdout:0/117: dwrite d2/d9/df/f1d [0,4194304] 0 2024-04-24T08:24:36.490 INFO:tasks.workunit.client.1.smithi149.stdout:4/85: mkdir d2/d18 0 2024-04-24T08:24:36.494 INFO:tasks.workunit.client.1.smithi149.stdout:4/86: write d2/fb [41045,101914] 0 2024-04-24T08:24:36.500 INFO:tasks.workunit.client.1.smithi149.stdout:5/158: mknod db/d1f/c2d 0 2024-04-24T08:24:36.501 INFO:tasks.workunit.client.1.smithi149.stdout:6/112: unlink d0/f15 0 2024-04-24T08:24:36.506 INFO:tasks.workunit.client.1.smithi149.stdout:9/137: symlink d1/da/d13/l2d 0 2024-04-24T08:24:36.506 INFO:tasks.workunit.client.1.smithi149.stdout:1/177: rmdir d0/da 39 2024-04-24T08:24:36.507 INFO:tasks.workunit.client.1.smithi149.stdout:7/96: chown c4 2065 1 2024-04-24T08:24:36.508 INFO:tasks.workunit.client.1.smithi149.stdout:2/141: dwrite d0/d6/db/f1a [0,4194304] 0 2024-04-24T08:24:36.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/87: creat d2/f19 x:0 0 0 2024-04-24T08:24:36.511 INFO:tasks.workunit.client.1.smithi149.stdout:8/93: sync 2024-04-24T08:24:36.511 INFO:tasks.workunit.client.1.smithi149.stdout:4/88: chown d2/l3 4622957 1 2024-04-24T08:24:36.513 INFO:tasks.workunit.client.1.smithi149.stdout:2/142: write d0/f21 [999532,5692] 0 2024-04-24T08:24:36.520 INFO:tasks.workunit.client.1.smithi149.stdout:6/113: fdatasync d0/f16 0 2024-04-24T08:24:36.522 INFO:tasks.workunit.client.1.smithi149.stdout:7/97: write f3 [2381807,29442] 0 2024-04-24T08:24:36.523 INFO:tasks.workunit.client.1.smithi149.stdout:6/114: write d0/f17 [4066679,49751] 0 2024-04-24T08:24:36.525 INFO:tasks.workunit.client.1.smithi149.stdout:0/118: symlink d2/d17/l22 0 2024-04-24T08:24:36.526 INFO:tasks.workunit.client.1.smithi149.stdout:7/98: dread - d9/dc/f18 zero size 2024-04-24T08:24:36.528 INFO:tasks.workunit.client.1.smithi149.stdout:3/150: unlink d0/f3 0 2024-04-24T08:24:36.529 INFO:tasks.workunit.client.1.smithi149.stdout:4/89: mkdir d2/d10/d1a 0 2024-04-24T08:24:36.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/178: chown d0/d5/df/f28 1006045651 1 2024-04-24T08:24:36.531 INFO:tasks.workunit.client.1.smithi149.stdout:3/151: stat d0/d5/da/f24 0 2024-04-24T08:24:36.531 INFO:tasks.workunit.client.1.smithi149.stdout:4/90: chown d2/d18 1 1 2024-04-24T08:24:36.533 INFO:tasks.workunit.client.1.smithi149.stdout:1/179: chown d0/d5/f9 0 1 2024-04-24T08:24:36.533 INFO:tasks.workunit.client.1.smithi149.stdout:4/91: dread - d2/f19 zero size 2024-04-24T08:24:36.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/180: write d0/d5/f9 [1055263,20077] 0 2024-04-24T08:24:36.538 INFO:tasks.workunit.client.1.smithi149.stdout:2/143: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:36.543 INFO:tasks.workunit.client.1.smithi149.stdout:6/115: rmdir d0/d1a 39 2024-04-24T08:24:36.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/159: truncate db/f20 459455 0 2024-04-24T08:24:36.551 INFO:tasks.workunit.client.1.smithi149.stdout:5/160: readlink db/d12/d22/l29 0 2024-04-24T08:24:36.551 INFO:tasks.workunit.client.1.smithi149.stdout:3/152: read d0/f4 [2335015,121261] 0 2024-04-24T08:24:36.554 INFO:tasks.workunit.client.1.smithi149.stdout:7/99: link d9/dc/f17 d9/dc/de/f22 0 2024-04-24T08:24:36.556 INFO:tasks.workunit.client.1.smithi149.stdout:7/100: chown d9/dc/de 177711 1 2024-04-24T08:24:36.564 INFO:tasks.workunit.client.1.smithi149.stdout:7/101: write d9/dc/f1b [516982,52552] 0 2024-04-24T08:24:36.564 INFO:tasks.workunit.client.1.smithi149.stdout:9/138: dwrite d1/da/fc [0,4194304] 0 2024-04-24T08:24:36.564 INFO:tasks.workunit.client.1.smithi149.stdout:4/92: creat d2/d18/f1b x:0 0 0 2024-04-24T08:24:36.564 INFO:tasks.workunit.client.1.smithi149.stdout:0/119: dwrite d2/fb [0,4194304] 0 2024-04-24T08:24:36.569 INFO:tasks.workunit.client.1.smithi149.stdout:4/93: read d2/f7 [347504,115068] 0 2024-04-24T08:24:36.570 INFO:tasks.workunit.client.1.smithi149.stdout:8/94: getdents d1 0 2024-04-24T08:24:36.571 INFO:tasks.workunit.client.1.smithi149.stdout:1/181: creat d0/d5/db/d18/d2f/f3d x:0 0 0 2024-04-24T08:24:36.574 INFO:tasks.workunit.client.1.smithi149.stdout:4/94: dread - d2/d18/f1b zero size 2024-04-24T08:24:36.574 INFO:tasks.workunit.client.1.smithi149.stdout:1/182: write d0/fe [4730872,72999] 0 2024-04-24T08:24:36.575 INFO:tasks.workunit.client.1.smithi149.stdout:9/139: truncate d1/d2/f2b 1026757 0 2024-04-24T08:24:36.577 INFO:tasks.workunit.client.1.smithi149.stdout:7/102: chown c8 0 1 2024-04-24T08:24:36.578 INFO:tasks.workunit.client.1.smithi149.stdout:0/120: creat d2/d17/f23 x:0 0 0 2024-04-24T08:24:36.584 INFO:tasks.workunit.client.1.smithi149.stdout:7/103: readlink d9/dc/l13 0 2024-04-24T08:24:36.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/95: unlink d2/f4 0 2024-04-24T08:24:36.585 INFO:tasks.workunit.client.1.smithi149.stdout:9/140: creat d1/de/f2e x:0 0 0 2024-04-24T08:24:36.585 INFO:tasks.workunit.client.1.smithi149.stdout:1/183: read - d0/d5/df/d12/d19/f33 zero size 2024-04-24T08:24:36.587 INFO:tasks.workunit.client.1.smithi149.stdout:9/141: chown d1/da/f8 4882522 1 2024-04-24T08:24:36.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/121: write d2/d9/df/f16 [1052137,50408] 0 2024-04-24T08:24:36.589 INFO:tasks.workunit.client.1.smithi149.stdout:6/116: dread d0/d1a/f27 [0,4194304] 0 2024-04-24T08:24:36.597 INFO:tasks.workunit.client.1.smithi149.stdout:5/161: dread db/f20 [0,4194304] 0 2024-04-24T08:24:36.597 INFO:tasks.workunit.client.1.smithi149.stdout:0/122: readlink d2/d9/df/l12 0 2024-04-24T08:24:36.597 INFO:tasks.workunit.client.1.smithi149.stdout:8/95: dread d1/f3 [0,4194304] 0 2024-04-24T08:24:36.597 INFO:tasks.workunit.client.1.smithi149.stdout:7/104: rename d9/c14 to d9/dc/c23 0 2024-04-24T08:24:36.600 INFO:tasks.workunit.client.1.smithi149.stdout:4/96: symlink d2/d10/d1a/l1c 0 2024-04-24T08:24:36.601 INFO:tasks.workunit.client.1.smithi149.stdout:9/142: dread d1/f26 [0,4194304] 0 2024-04-24T08:24:36.603 INFO:tasks.workunit.client.1.smithi149.stdout:4/97: chown d2/d13/f15 316931 1 2024-04-24T08:24:36.608 INFO:tasks.workunit.client.1.smithi149.stdout:0/123: creat d2/d17/f24 x:0 0 0 2024-04-24T08:24:36.609 INFO:tasks.workunit.client.1.smithi149.stdout:0/124: dread - d2/d17/f1f zero size 2024-04-24T08:24:36.614 INFO:tasks.workunit.client.1.smithi149.stdout:8/96: mknod d1/c15 0 2024-04-24T08:24:36.614 INFO:tasks.workunit.client.1.smithi149.stdout:8/97: stat d1/l6 0 2024-04-24T08:24:36.621 INFO:tasks.workunit.client.1.smithi149.stdout:4/98: creat d2/d10/d1a/f1d x:0 0 0 2024-04-24T08:24:36.622 INFO:tasks.workunit.client.1.smithi149.stdout:4/99: read - d2/f14 zero size 2024-04-24T08:24:36.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/162: write db/fc [1975718,35832] 0 2024-04-24T08:24:36.629 INFO:tasks.workunit.client.1.smithi149.stdout:6/117: truncate d0/f17 352555 0 2024-04-24T08:24:36.633 INFO:tasks.workunit.client.1.smithi149.stdout:7/105: dwrite d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:36.634 INFO:tasks.workunit.client.1.smithi149.stdout:7/106: fdatasync d9/fd 0 2024-04-24T08:24:36.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/143: link d1/d2/d15/d1c/f20 d1/d2/d15/f2f 0 2024-04-24T08:24:36.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/153: rmdir d0/d5 39 2024-04-24T08:24:36.649 INFO:tasks.workunit.client.1.smithi149.stdout:5/163: unlink db/d12/d22/l29 0 2024-04-24T08:24:36.649 INFO:tasks.workunit.client.1.smithi149.stdout:5/164: write db/f10 [5257011,9839] 0 2024-04-24T08:24:36.652 INFO:tasks.workunit.client.1.smithi149.stdout:5/165: read db/f20 [278731,90362] 0 2024-04-24T08:24:36.653 INFO:tasks.workunit.client.1.smithi149.stdout:8/98: write d1/f14 [1179887,31638] 0 2024-04-24T08:24:36.654 INFO:tasks.workunit.client.1.smithi149.stdout:9/144: creat d1/d2/d15/f30 x:0 0 0 2024-04-24T08:24:36.654 INFO:tasks.workunit.client.1.smithi149.stdout:6/118: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:24:36.658 INFO:tasks.workunit.client.1.smithi149.stdout:3/154: write d0/d5/da/fb [783209,32426] 0 2024-04-24T08:24:36.668 INFO:tasks.workunit.client.1.smithi149.stdout:8/99: unlink d1/f2 0 2024-04-24T08:24:36.669 INFO:tasks.workunit.client.1.smithi149.stdout:6/119: chown d0/f4 361 1 2024-04-24T08:24:36.677 INFO:tasks.workunit.client.1.smithi149.stdout:3/155: mkdir d0/d5/d1a/d1c/d34 0 2024-04-24T08:24:36.678 INFO:tasks.workunit.client.1.smithi149.stdout:5/166: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:36.679 INFO:tasks.workunit.client.1.smithi149.stdout:7/107: creat d9/dc/f24 x:0 0 0 2024-04-24T08:24:36.679 INFO:tasks.workunit.client.1.smithi149.stdout:8/100: creat d1/de/f16 x:0 0 0 2024-04-24T08:24:36.680 INFO:tasks.workunit.client.1.smithi149.stdout:5/167: readlink db/d12/l17 0 2024-04-24T08:24:36.681 INFO:tasks.workunit.client.1.smithi149.stdout:6/120: mkdir d0/d1a/d28 0 2024-04-24T08:24:36.685 INFO:tasks.workunit.client.1.smithi149.stdout:5/168: write db/f2b [165251,115675] 0 2024-04-24T08:24:36.687 INFO:tasks.workunit.client.1.smithi149.stdout:6/121: fsync d0/f4 0 2024-04-24T08:24:36.688 INFO:tasks.workunit.client.1.smithi149.stdout:7/108: symlink d9/dc/l25 0 2024-04-24T08:24:36.690 INFO:tasks.workunit.client.1.smithi149.stdout:5/169: fsync db/d12/d1e/f27 0 2024-04-24T08:24:36.692 INFO:tasks.workunit.client.1.smithi149.stdout:6/122: creat d0/d1a/f29 x:0 0 0 2024-04-24T08:24:36.693 INFO:tasks.workunit.client.1.smithi149.stdout:7/109: mkdir d9/dc/d11/d26 0 2024-04-24T08:24:36.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/110: stat d9/dc/d11/f1e 0 2024-04-24T08:24:36.697 INFO:tasks.workunit.client.1.smithi149.stdout:5/170: rmdir db/d12/d1e 39 2024-04-24T08:24:36.716 INFO:tasks.workunit.client.1.smithi149.stdout:9/145: truncate d1/d2/f19 2295094 0 2024-04-24T08:24:36.718 INFO:tasks.workunit.client.1.smithi149.stdout:3/156: write d0/d5/da/f24 [1105719,119194] 0 2024-04-24T08:24:36.720 INFO:tasks.workunit.client.1.smithi149.stdout:9/146: chown d1/da/c25 2 1 2024-04-24T08:24:36.721 INFO:tasks.workunit.client.1.smithi149.stdout:7/111: dwrite d9/dc/f18 [0,4194304] 0 2024-04-24T08:24:36.723 INFO:tasks.workunit.client.1.smithi149.stdout:7/112: chown d9/fd 552 1 2024-04-24T08:24:36.728 INFO:tasks.workunit.client.1.smithi149.stdout:6/123: getdents d0/d1a 0 2024-04-24T08:24:36.733 INFO:tasks.workunit.client.1.smithi149.stdout:5/171: truncate db/f16 560349 0 2024-04-24T08:24:36.738 INFO:tasks.workunit.client.1.smithi149.stdout:9/147: mknod d1/da/c31 0 2024-04-24T08:24:36.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/157: creat d0/d5/d1a/d1c/d34/f35 x:0 0 0 2024-04-24T08:24:36.745 INFO:tasks.workunit.client.1.smithi149.stdout:9/148: read d1/d2/d15/f2f [1187028,120307] 0 2024-04-24T08:24:36.747 INFO:tasks.workunit.client.1.smithi149.stdout:5/172: symlink db/d12/d22/l2e 0 2024-04-24T08:24:36.750 INFO:tasks.workunit.client.1.smithi149.stdout:7/113: dwrite d9/f19 [0,4194304] 0 2024-04-24T08:24:36.755 INFO:tasks.workunit.client.1.smithi149.stdout:3/158: creat d0/d5/da/d19/d1b/d2b/f36 x:0 0 0 2024-04-24T08:24:36.758 INFO:tasks.workunit.client.1.smithi149.stdout:7/114: truncate d9/dc/de/f22 170763 0 2024-04-24T08:24:36.762 INFO:tasks.workunit.client.1.smithi149.stdout:5/173: mkdir db/d12/d2f 0 2024-04-24T08:24:36.775 INFO:tasks.workunit.client.1.smithi149.stdout:7/115: dwrite d9/dc/de/f1a [0,4194304] 0 2024-04-24T08:24:36.777 INFO:tasks.workunit.client.1.smithi149.stdout:5/174: dread f9 [0,4194304] 0 2024-04-24T08:24:36.780 INFO:tasks.workunit.client.1.smithi149.stdout:5/175: creat db/d1f/f30 x:0 0 0 2024-04-24T08:24:36.783 INFO:tasks.workunit.client.1.smithi149.stdout:7/116: fdatasync d9/dc/de/f22 0 2024-04-24T08:24:36.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/176: write db/f14 [2223846,83999] 0 2024-04-24T08:24:36.790 INFO:tasks.workunit.client.1.smithi149.stdout:7/117: dread d9/dc/f1b [0,4194304] 0 2024-04-24T08:24:36.795 INFO:tasks.workunit.client.1.smithi149.stdout:7/118: symlink d9/dc/d11/l27 0 2024-04-24T08:24:36.798 INFO:tasks.workunit.client.1.smithi149.stdout:7/119: unlink d9/dc/f24 0 2024-04-24T08:24:36.799 INFO:tasks.workunit.client.1.smithi149.stdout:5/177: dread f9 [0,4194304] 0 2024-04-24T08:24:36.803 INFO:tasks.workunit.client.1.smithi149.stdout:7/120: creat d9/dc/d11/d26/f28 x:0 0 0 2024-04-24T08:24:36.806 INFO:tasks.workunit.client.1.smithi149.stdout:7/121: dread d9/dc/f17 [0,4194304] 0 2024-04-24T08:24:36.818 INFO:tasks.workunit.client.1.smithi149.stdout:5/178: dwrite db/d1f/f2a [0,4194304] 0 2024-04-24T08:24:36.830 INFO:tasks.workunit.client.1.smithi149.stdout:7/122: dwrite d9/dc/de/f22 [0,4194304] 0 2024-04-24T08:24:36.833 INFO:tasks.workunit.client.1.smithi149.stdout:7/123: write d9/f19 [2918963,93857] 0 2024-04-24T08:24:36.838 INFO:tasks.workunit.client.1.smithi149.stdout:7/124: write d9/dc/f18 [3694134,42449] 0 2024-04-24T08:24:36.847 INFO:tasks.workunit.client.1.smithi149.stdout:7/125: creat d9/dc/f29 x:0 0 0 2024-04-24T08:24:36.848 INFO:tasks.workunit.client.1.smithi149.stdout:5/179: fdatasync db/d1f/f2a 0 2024-04-24T08:24:36.851 INFO:tasks.workunit.client.1.smithi149.stdout:5/180: chown db/d1f/l25 2 1 2024-04-24T08:24:36.853 INFO:tasks.workunit.client.1.smithi149.stdout:7/126: creat d9/dc/d11/f2a x:0 0 0 2024-04-24T08:24:36.854 INFO:tasks.workunit.client.1.smithi149.stdout:7/127: dread - d9/f21 zero size 2024-04-24T08:24:36.856 INFO:tasks.workunit.client.1.smithi149.stdout:5/181: rename db/fc to db/d12/d22/f31 0 2024-04-24T08:24:36.869 INFO:tasks.workunit.client.1.smithi149.stdout:7/128: getdents d9/dc 0 2024-04-24T08:24:36.871 INFO:tasks.workunit.client.1.smithi149.stdout:5/182: write db/f16 [729573,70605] 0 2024-04-24T08:24:36.884 INFO:tasks.workunit.client.1.smithi149.stdout:7/129: dread f1 [0,4194304] 0 2024-04-24T08:24:36.887 INFO:tasks.workunit.client.1.smithi149.stdout:7/130: fsync d9/dc/d11/d26/f28 0 2024-04-24T08:24:36.889 INFO:tasks.workunit.client.1.smithi149.stdout:5/183: dwrite db/d1c/f21 [4194304,4194304] 0 2024-04-24T08:24:36.890 INFO:tasks.workunit.client.1.smithi149.stdout:5/184: chown db/l19 551557 1 2024-04-24T08:24:36.897 INFO:tasks.workunit.client.1.smithi149.stdout:2/144: sync 2024-04-24T08:24:36.899 INFO:tasks.workunit.client.1.smithi149.stdout:2/145: write d0/f8 [3100513,64005] 0 2024-04-24T08:24:36.899 INFO:tasks.workunit.client.1.smithi149.stdout:5/185: rmdir db/d12/d22/d2c 0 2024-04-24T08:24:36.901 INFO:tasks.workunit.client.1.smithi149.stdout:5/186: stat db/c11 0 2024-04-24T08:24:36.903 INFO:tasks.workunit.client.1.smithi149.stdout:2/146: symlink d0/d6/db/d28/l33 0 2024-04-24T08:24:36.908 INFO:tasks.workunit.client.1.smithi149.stdout:2/147: symlink d0/d6/l34 0 2024-04-24T08:24:36.910 INFO:tasks.workunit.client.1.smithi149.stdout:7/131: dread d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:36.913 INFO:tasks.workunit.client.1.smithi149.stdout:7/132: creat d9/dc/d11/f2b x:0 0 0 2024-04-24T08:24:36.914 INFO:tasks.workunit.client.1.smithi149.stdout:5/187: write db/d12/d1e/f27 [809947,47668] 0 2024-04-24T08:24:36.917 INFO:tasks.workunit.client.1.smithi149.stdout:5/188: chown db/d12/d1e 5 1 2024-04-24T08:24:36.919 INFO:tasks.workunit.client.1.smithi149.stdout:5/189: truncate db/f1a 1808759 0 2024-04-24T08:24:36.928 INFO:tasks.workunit.client.1.smithi149.stdout:5/190: link db/d1f/c2d db/d12/d2f/c32 0 2024-04-24T08:24:36.932 INFO:tasks.workunit.client.1.smithi149.stdout:2/148: dwrite d0/f2d [0,4194304] 0 2024-04-24T08:24:36.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/133: dwrite d9/dc/de/f1a [0,4194304] 0 2024-04-24T08:24:36.942 INFO:tasks.workunit.client.1.smithi149.stdout:7/134: chown d9/dc/f18 5 1 2024-04-24T08:24:36.946 INFO:tasks.workunit.client.1.smithi149.stdout:7/135: write d9/fd [1845908,127390] 0 2024-04-24T08:24:36.949 INFO:tasks.workunit.client.1.smithi149.stdout:5/191: dread db/d1f/f2a [0,4194304] 0 2024-04-24T08:24:36.953 INFO:tasks.workunit.client.1.smithi149.stdout:5/192: unlink db/f2b 0 2024-04-24T08:24:36.954 INFO:tasks.workunit.client.1.smithi149.stdout:7/136: creat d9/f2c x:0 0 0 2024-04-24T08:24:36.960 INFO:tasks.workunit.client.1.smithi149.stdout:7/137: mknod d9/c2d 0 2024-04-24T08:24:36.964 INFO:tasks.workunit.client.1.smithi149.stdout:2/149: dread d0/d6/db/f1c [0,4194304] 0 2024-04-24T08:24:36.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/150: chown d0 381408586 1 2024-04-24T08:24:36.967 INFO:tasks.workunit.client.1.smithi149.stdout:7/138: write d9/dc/f17 [1140300,129341] 0 2024-04-24T08:24:36.967 INFO:tasks.workunit.client.1.smithi149.stdout:5/193: dread db/f1a [0,4194304] 0 2024-04-24T08:24:36.971 INFO:tasks.workunit.client.1.smithi149.stdout:5/194: truncate db/d12/d22/f31 4254873 0 2024-04-24T08:24:36.971 INFO:tasks.workunit.client.1.smithi149.stdout:7/139: mkdir d9/dc/d2e 0 2024-04-24T08:24:36.972 INFO:tasks.workunit.client.1.smithi149.stdout:2/151: creat d0/f35 x:0 0 0 2024-04-24T08:24:36.975 INFO:tasks.workunit.client.1.smithi149.stdout:7/140: readlink d9/dc/d11/l27 0 2024-04-24T08:24:36.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/141: truncate d9/dc/d11/f2b 332105 0 2024-04-24T08:24:36.986 INFO:tasks.workunit.client.1.smithi149.stdout:5/195: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:36.988 INFO:tasks.workunit.client.1.smithi149.stdout:5/196: chown db/d12/c28 405 1 2024-04-24T08:24:36.991 INFO:tasks.workunit.client.1.smithi149.stdout:5/197: fdatasync db/d12/d22/f31 0 2024-04-24T08:24:36.997 INFO:tasks.workunit.client.1.smithi149.stdout:7/142: dwrite d9/dc/de/f22 [0,4194304] 0 2024-04-24T08:24:37.000 INFO:tasks.workunit.client.1.smithi149.stdout:5/198: truncate db/d1f/f2a 3072094 0 2024-04-24T08:24:37.004 INFO:tasks.workunit.client.1.smithi149.stdout:7/143: mknod d9/dc/d2e/c2f 0 2024-04-24T08:24:37.009 INFO:tasks.workunit.client.1.smithi149.stdout:7/144: symlink d9/dc/d2e/l30 0 2024-04-24T08:24:37.011 INFO:tasks.workunit.client.1.smithi149.stdout:5/199: truncate db/d12/d1e/f27 1019045 0 2024-04-24T08:24:37.016 INFO:tasks.workunit.client.1.smithi149.stdout:7/145: creat d9/dc/de/f31 x:0 0 0 2024-04-24T08:24:37.017 INFO:tasks.workunit.client.1.smithi149.stdout:7/146: readlink d9/dc/d11/l27 0 2024-04-24T08:24:37.021 INFO:tasks.workunit.client.1.smithi149.stdout:7/147: dread d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:37.025 INFO:tasks.workunit.client.1.smithi149.stdout:7/148: mknod d9/dc/d11/c32 0 2024-04-24T08:24:37.039 INFO:tasks.workunit.client.1.smithi149.stdout:5/200: write db/f1a [963719,11619] 0 2024-04-24T08:24:37.045 INFO:tasks.workunit.client.1.smithi149.stdout:5/201: rename f9 to db/d12/d22/f33 0 2024-04-24T08:24:37.046 INFO:tasks.workunit.client.1.smithi149.stdout:7/149: dwrite d9/dc/de/f22 [0,4194304] 0 2024-04-24T08:24:37.063 INFO:tasks.workunit.client.1.smithi149.stdout:5/202: dread db/d12/d22/f33 [0,4194304] 0 2024-04-24T08:24:37.065 INFO:tasks.workunit.client.1.smithi149.stdout:7/150: dwrite d9/dc/de/f12 [0,4194304] 0 2024-04-24T08:24:37.069 INFO:tasks.workunit.client.1.smithi149.stdout:5/203: rename db/d1c/c1d to db/d1f/c34 0 2024-04-24T08:24:37.072 INFO:tasks.workunit.client.1.smithi149.stdout:5/204: fdatasync db/d12/d22/f31 0 2024-04-24T08:24:37.082 INFO:tasks.workunit.client.1.smithi149.stdout:5/205: mkdir db/d35 0 2024-04-24T08:24:37.083 INFO:tasks.workunit.client.1.smithi149.stdout:7/151: getdents d9/dc/de 0 2024-04-24T08:24:37.090 INFO:tasks.workunit.client.1.smithi149.stdout:5/206: creat db/d12/d2f/f36 x:0 0 0 2024-04-24T08:24:37.091 INFO:tasks.workunit.client.1.smithi149.stdout:5/207: fsync db/d12/d22/f31 0 2024-04-24T08:24:37.092 INFO:tasks.workunit.client.1.smithi149.stdout:7/152: link d9/fd d9/dc/de/f33 0 2024-04-24T08:24:37.096 INFO:tasks.workunit.client.1.smithi149.stdout:7/153: chown c7 3 1 2024-04-24T08:24:37.097 INFO:tasks.workunit.client.1.smithi149.stdout:5/208: getdents db/d23 0 2024-04-24T08:24:37.099 INFO:tasks.workunit.client.1.smithi149.stdout:7/154: dread - d9/dc/f29 zero size 2024-04-24T08:24:37.103 INFO:tasks.workunit.client.1.smithi149.stdout:7/155: unlink d9/f21 0 2024-04-24T08:24:37.104 INFO:tasks.workunit.client.1.smithi149.stdout:7/156: chown d9/f2c 53311133 1 2024-04-24T08:24:37.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/209: symlink db/d35/l37 0 2024-04-24T08:24:37.109 INFO:tasks.workunit.client.1.smithi149.stdout:5/210: mknod db/d12/c38 0 2024-04-24T08:24:37.110 INFO:tasks.workunit.client.1.smithi149.stdout:5/211: read - db/d12/d2f/f36 zero size 2024-04-24T08:24:37.111 INFO:tasks.workunit.client.1.smithi149.stdout:5/212: chown c8 16 1 2024-04-24T08:24:37.113 INFO:tasks.workunit.client.1.smithi149.stdout:1/184: sync 2024-04-24T08:24:37.115 INFO:tasks.workunit.client.1.smithi149.stdout:7/157: dread d9/dc/de/f1a [0,4194304] 0 2024-04-24T08:24:37.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/213: symlink db/d12/d2f/l39 0 2024-04-24T08:24:37.117 INFO:tasks.workunit.client.1.smithi149.stdout:1/185: symlink d0/d5/df/d12/d19/l3e 0 2024-04-24T08:24:37.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/214: symlink db/d1f/l3a 0 2024-04-24T08:24:37.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/215: chown c8 182463 1 2024-04-24T08:24:37.120 INFO:tasks.workunit.client.1.smithi149.stdout:7/158: mknod d9/dc/d11/c34 0 2024-04-24T08:24:37.124 INFO:tasks.workunit.client.1.smithi149.stdout:1/186: unlink d0/d5/df/d12/l35 0 2024-04-24T08:24:37.127 INFO:tasks.workunit.client.1.smithi149.stdout:1/187: rmdir d0/d1b/d2a 39 2024-04-24T08:24:37.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/159: rename d9/dc/de/f1a to d9/dc/f35 0 2024-04-24T08:24:37.220 INFO:tasks.workunit.client.1.smithi149.stdout:5/216: link db/d1f/c34 db/d12/d1e/c3b 0 2024-04-24T08:24:37.222 INFO:tasks.workunit.client.1.smithi149.stdout:1/188: creat d0/da/f3f x:0 0 0 2024-04-24T08:24:37.223 INFO:tasks.workunit.client.1.smithi149.stdout:1/189: readlink d0/d5/df/d12/d19/l3e 0 2024-04-24T08:24:37.227 INFO:tasks.workunit.client.1.smithi149.stdout:5/217: creat db/d12/d1e/f3c x:0 0 0 2024-04-24T08:24:37.229 INFO:tasks.workunit.client.1.smithi149.stdout:1/190: symlink d0/d1b/l40 0 2024-04-24T08:24:37.245 INFO:tasks.workunit.client.1.smithi149.stdout:1/191: dwrite d0/d1b/f1f [0,4194304] 0 2024-04-24T08:24:37.248 INFO:tasks.workunit.client.1.smithi149.stdout:1/192: write d0/d5/db/d18/f3a [663950,106154] 0 2024-04-24T08:24:37.250 INFO:tasks.workunit.client.1.smithi149.stdout:1/193: dread - d0/f25 zero size 2024-04-24T08:24:37.250 INFO:tasks.workunit.client.1.smithi149.stdout:1/194: readlink d0/l3 0 2024-04-24T08:24:37.252 INFO:tasks.workunit.client.1.smithi149.stdout:1/195: chown d0/fd 2832 1 2024-04-24T08:24:37.268 INFO:tasks.workunit.client.1.smithi149.stdout:1/196: rename d0/d5/df/d12/d19/f31 to d0/d5/f41 0 2024-04-24T08:24:37.296 INFO:tasks.workunit.client.1.smithi149.stdout:7/160: write d9/dc/d11/f2b [713312,68435] 0 2024-04-24T08:24:37.297 INFO:tasks.workunit.client.1.smithi149.stdout:7/161: chown d9/dc/d2e 19586161 1 2024-04-24T08:24:37.301 INFO:tasks.workunit.client.1.smithi149.stdout:7/162: symlink d9/dc/d11/d26/l36 0 2024-04-24T08:24:37.302 INFO:tasks.workunit.client.1.smithi149.stdout:7/163: dread - d9/f2c zero size 2024-04-24T08:24:37.309 INFO:tasks.workunit.client.1.smithi149.stdout:7/164: creat d9/dc/d11/f37 x:0 0 0 2024-04-24T08:24:37.311 INFO:tasks.workunit.client.1.smithi149.stdout:1/197: symlink d0/d5/df/d12/l42 0 2024-04-24T08:24:37.315 INFO:tasks.workunit.client.1.smithi149.stdout:7/165: getdents d9/dc 0 2024-04-24T08:24:37.316 INFO:tasks.workunit.client.1.smithi149.stdout:1/198: write d0/d5/df/d12/d19/f2b [481780,99987] 0 2024-04-24T08:24:37.316 INFO:tasks.workunit.client.1.smithi149.stdout:7/166: readlink d9/dc/d2e/l30 0 2024-04-24T08:24:37.328 INFO:tasks.workunit.client.1.smithi149.stdout:1/199: dread d0/d5/db/f1e [0,4194304] 0 2024-04-24T08:24:37.344 INFO:tasks.workunit.client.1.smithi149.stdout:1/200: dwrite d0/da/f3f [0,4194304] 0 2024-04-24T08:24:37.363 INFO:tasks.workunit.client.1.smithi149.stdout:1/201: dwrite d0/d5/db/d18/d2f/f3d [0,4194304] 0 2024-04-24T08:24:37.367 INFO:tasks.workunit.client.1.smithi149.stdout:1/202: mknod d0/d5/db/d18/c43 0 2024-04-24T08:24:37.369 INFO:tasks.workunit.client.1.smithi149.stdout:0/125: sync 2024-04-24T08:24:37.369 INFO:tasks.workunit.client.1.smithi149.stdout:4/100: sync 2024-04-24T08:24:37.372 INFO:tasks.workunit.client.1.smithi149.stdout:4/101: write d2/d18/f1b [245406,116294] 0 2024-04-24T08:24:37.378 INFO:tasks.workunit.client.1.smithi149.stdout:1/203: creat d0/f44 x:0 0 0 2024-04-24T08:24:37.380 INFO:tasks.workunit.client.1.smithi149.stdout:1/204: chown d0/d5/l32 4521238 1 2024-04-24T08:24:37.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/205: chown d0/d1b/c2e 1381 1 2024-04-24T08:24:37.385 INFO:tasks.workunit.client.1.smithi149.stdout:1/206: creat d0/d5/db/d18/f45 x:0 0 0 2024-04-24T08:24:37.388 INFO:tasks.workunit.client.1.smithi149.stdout:0/126: dread d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:37.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/207: readlink d0/d5/df/l3b 0 2024-04-24T08:24:37.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/208: fdatasync d0/f25 0 2024-04-24T08:24:37.391 INFO:tasks.workunit.client.1.smithi149.stdout:0/127: dread - d2/d9/df/f1b zero size 2024-04-24T08:24:37.392 INFO:tasks.workunit.client.1.smithi149.stdout:4/102: dwrite d2/d18/f1b [0,4194304] 0 2024-04-24T08:24:37.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/209: unlink d0/l3 0 2024-04-24T08:24:37.396 INFO:tasks.workunit.client.1.smithi149.stdout:0/128: unlink d2/d9/df/l12 0 2024-04-24T08:24:37.400 INFO:tasks.workunit.client.1.smithi149.stdout:4/103: rename d2/d13 to d2/d18/d1e 0 2024-04-24T08:24:37.403 INFO:tasks.workunit.client.1.smithi149.stdout:4/104: write d2/fc [4280869,65028] 0 2024-04-24T08:24:37.408 INFO:tasks.workunit.client.1.smithi149.stdout:4/105: write d2/fc [4659538,26606] 0 2024-04-24T08:24:37.417 INFO:tasks.workunit.client.1.smithi149.stdout:0/129: dread d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:37.418 INFO:tasks.workunit.client.1.smithi149.stdout:1/210: dwrite d0/f16 [4194304,4194304] 0 2024-04-24T08:24:37.421 INFO:tasks.workunit.client.1.smithi149.stdout:1/211: fdatasync d0/fd 0 2024-04-24T08:24:37.421 INFO:tasks.workunit.client.1.smithi149.stdout:0/130: read d2/f8 [769614,79175] 0 2024-04-24T08:24:37.423 INFO:tasks.workunit.client.1.smithi149.stdout:1/212: chown d0/da/f3f 34 1 2024-04-24T08:24:37.431 INFO:tasks.workunit.client.1.smithi149.stdout:1/213: readlink d0/d5/df/l15 0 2024-04-24T08:24:37.434 INFO:tasks.workunit.client.1.smithi149.stdout:1/214: write d0/d5/db/d18/f3a [1298438,16742] 0 2024-04-24T08:24:37.439 INFO:tasks.workunit.client.1.smithi149.stdout:4/106: dwrite d2/f19 [0,4194304] 0 2024-04-24T08:24:37.439 INFO:tasks.workunit.client.1.smithi149.stdout:0/131: write d2/d9/f13 [315281,76039] 0 2024-04-24T08:24:37.445 INFO:tasks.workunit.client.1.smithi149.stdout:0/132: truncate d2/d17/f23 216837 0 2024-04-24T08:24:37.451 INFO:tasks.workunit.client.1.smithi149.stdout:4/107: getdents d2 0 2024-04-24T08:24:37.455 INFO:tasks.workunit.client.1.smithi149.stdout:4/108: write d2/d10/d1a/f1d [108645,37563] 0 2024-04-24T08:24:37.462 INFO:tasks.workunit.client.1.smithi149.stdout:4/109: symlink d2/d18/d1e/l1f 0 2024-04-24T08:24:37.466 INFO:tasks.workunit.client.1.smithi149.stdout:4/110: creat d2/d18/f20 x:0 0 0 2024-04-24T08:24:37.468 INFO:tasks.workunit.client.1.smithi149.stdout:1/215: dwrite d0/d5/db/d18/d2f/f3d [4194304,4194304] 0 2024-04-24T08:24:37.470 INFO:tasks.workunit.client.1.smithi149.stdout:0/133: dwrite d2/d9/df/f1b [0,4194304] 0 2024-04-24T08:24:37.473 INFO:tasks.workunit.client.1.smithi149.stdout:4/111: creat d2/d18/d1e/f21 x:0 0 0 2024-04-24T08:24:37.474 INFO:tasks.workunit.client.1.smithi149.stdout:0/134: write d2/d17/f24 [707197,56138] 0 2024-04-24T08:24:37.477 INFO:tasks.workunit.client.1.smithi149.stdout:1/216: mknod d0/d5/db/c46 0 2024-04-24T08:24:37.480 INFO:tasks.workunit.client.1.smithi149.stdout:4/112: symlink d2/d18/d1e/l22 0 2024-04-24T08:24:37.482 INFO:tasks.workunit.client.1.smithi149.stdout:4/113: chown d2/d10/d1a 15 1 2024-04-24T08:24:37.482 INFO:tasks.workunit.client.1.smithi149.stdout:0/135: creat d2/d9/d1a/f25 x:0 0 0 2024-04-24T08:24:37.485 INFO:tasks.workunit.client.1.smithi149.stdout:1/217: mknod d0/da/c47 0 2024-04-24T08:24:37.485 INFO:tasks.workunit.client.1.smithi149.stdout:4/114: fsync d2/d18/d1e/f15 0 2024-04-24T08:24:37.492 INFO:tasks.workunit.client.1.smithi149.stdout:4/115: getdents d2/d18 0 2024-04-24T08:24:37.495 INFO:tasks.workunit.client.1.smithi149.stdout:4/116: write d2/d18/d1e/f15 [1106494,34785] 0 2024-04-24T08:24:37.498 INFO:tasks.workunit.client.1.smithi149.stdout:1/218: dread d0/d5/db/d18/f3a [0,4194304] 0 2024-04-24T08:24:37.503 INFO:tasks.workunit.client.1.smithi149.stdout:4/117: link d2/d18/d1e/f21 d2/d10/f23 0 2024-04-24T08:24:37.506 INFO:tasks.workunit.client.1.smithi149.stdout:4/118: getdents d2 0 2024-04-24T08:24:37.507 INFO:tasks.workunit.client.1.smithi149.stdout:1/219: dread d0/da/f24 [0,4194304] 0 2024-04-24T08:24:37.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/119: write d2/f14 [744757,8667] 0 2024-04-24T08:24:37.521 INFO:tasks.workunit.client.1.smithi149.stdout:4/120: mkdir d2/d24 0 2024-04-24T08:24:37.523 INFO:tasks.workunit.client.1.smithi149.stdout:4/121: rename d2 to d2/d18/d25 22 2024-04-24T08:24:37.524 INFO:tasks.workunit.client.1.smithi149.stdout:1/220: dwrite d0/f16 [4194304,4194304] 0 2024-04-24T08:24:37.525 INFO:tasks.workunit.client.1.smithi149.stdout:4/122: getdents d2/d24 0 2024-04-24T08:24:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:24:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:37 smithi149 ceph-mon[31282]: pgmap v11: 65 pgs: 65 active+clean; 288 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 14 MiB/s wr, 207 op/s 2024-04-24T08:24:37.531 INFO:tasks.workunit.client.1.smithi149.stdout:4/123: unlink d2/fb 0 2024-04-24T08:24:37.532 INFO:tasks.workunit.client.1.smithi149.stdout:1/221: truncate d0/d5/df/d12/d19/f23 573055 0 2024-04-24T08:24:37.534 INFO:tasks.workunit.client.1.smithi149.stdout:4/124: symlink d2/l26 0 2024-04-24T08:24:37.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/222: creat d0/da/f48 x:0 0 0 2024-04-24T08:24:37.540 INFO:tasks.workunit.client.1.smithi149.stdout:1/223: write d0/d5/df/d12/d19/f33 [636030,15606] 0 2024-04-24T08:24:37.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/224: rename d0 to d0/d5/db/d18/d34/d49 22 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:24:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:37 smithi090 ceph-mon[25016]: pgmap v11: 65 pgs: 65 active+clean; 288 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 14 MiB/s wr, 207 op/s 2024-04-24T08:24:37.545 INFO:tasks.workunit.client.1.smithi149.stdout:1/225: write d0/d5/df/d12/d19/f23 [78421,38561] 0 2024-04-24T08:24:37.552 INFO:tasks.workunit.client.1.smithi149.stdout:1/226: creat d0/d1b/d2a/f4a x:0 0 0 2024-04-24T08:24:37.552 INFO:tasks.workunit.client.1.smithi149.stdout:4/125: write d2/d18/d1e/f21 [879247,107797] 0 2024-04-24T08:24:37.556 INFO:tasks.workunit.client.1.smithi149.stdout:4/126: fdatasync d2/d18/d1e/f21 0 2024-04-24T08:24:37.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/127: rename d2/d10/d1a/f1d to d2/d10/f27 0 2024-04-24T08:24:37.564 INFO:tasks.workunit.client.1.smithi149.stdout:4/128: stat d2/d18/f20 0 2024-04-24T08:24:37.565 INFO:tasks.workunit.client.1.smithi149.stdout:4/129: rename d2 to d2/d10/d1a/d28 22 2024-04-24T08:24:37.566 INFO:tasks.workunit.client.1.smithi149.stdout:4/130: dread - d2/d18/f20 zero size 2024-04-24T08:24:37.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/227: dwrite d0/da/fc [0,4194304] 0 2024-04-24T08:24:37.568 INFO:tasks.workunit.client.1.smithi149.stdout:4/131: write d2/d10/f23 [1127841,9179] 0 2024-04-24T08:24:37.571 INFO:tasks.workunit.client.1.smithi149.stdout:4/132: write d2/f14 [1277790,41162] 0 2024-04-24T08:24:37.575 INFO:tasks.workunit.client.1.smithi149.stdout:1/228: mkdir d0/d1b/d4b 0 2024-04-24T08:24:37.577 INFO:tasks.workunit.client.1.smithi149.stdout:4/133: truncate d2/d18/d1e/f15 2965089 0 2024-04-24T08:24:37.580 INFO:tasks.workunit.client.1.smithi149.stdout:1/229: symlink d0/d5/l4c 0 2024-04-24T08:24:37.581 INFO:tasks.workunit.client.1.smithi149.stdout:4/134: creat d2/d24/f29 x:0 0 0 2024-04-24T08:24:37.584 INFO:tasks.workunit.client.1.smithi149.stdout:4/135: creat d2/d24/f2a x:0 0 0 2024-04-24T08:24:37.585 INFO:tasks.workunit.client.1.smithi149.stdout:1/230: read d0/fe [169271,15321] 0 2024-04-24T08:24:37.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/231: write d0/d1b/f1f [383868,105886] 0 2024-04-24T08:24:37.587 INFO:tasks.workunit.client.1.smithi149.stdout:4/136: truncate d2/d18/f20 1000557 0 2024-04-24T08:24:37.595 INFO:tasks.workunit.client.1.smithi149.stdout:1/232: mkdir d0/d5/db/d18/d4d 0 2024-04-24T08:24:37.599 INFO:tasks.workunit.client.1.smithi149.stdout:1/233: chown d0/d5/df/d12/d19/f2b 338106431 1 2024-04-24T08:24:37.601 INFO:tasks.workunit.client.1.smithi149.stdout:1/234: read - d0/d5/f41 zero size 2024-04-24T08:24:37.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/137: dwrite d2/d18/f1b [0,4194304] 0 2024-04-24T08:24:37.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/138: mknod d2/c2b 0 2024-04-24T08:24:37.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/139: readlink d2/l8 0 2024-04-24T08:24:37.615 INFO:tasks.workunit.client.1.smithi149.stdout:4/140: creat d2/d18/d1e/f2c x:0 0 0 2024-04-24T08:24:37.615 INFO:tasks.workunit.client.1.smithi149.stdout:4/141: chown d2/f14 0 1 2024-04-24T08:24:37.618 INFO:tasks.workunit.client.1.smithi149.stdout:4/142: mknod d2/d18/d1e/c2d 0 2024-04-24T08:24:37.619 INFO:tasks.workunit.client.1.smithi149.stdout:4/143: chown d2/l3 794 1 2024-04-24T08:24:37.619 INFO:tasks.workunit.client.1.smithi149.stdout:1/235: dwrite d0/f4 [4194304,4194304] 0 2024-04-24T08:24:37.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/101: sync 2024-04-24T08:24:37.621 INFO:tasks.workunit.client.1.smithi149.stdout:4/144: chown d2/l3 13542 1 2024-04-24T08:24:37.625 INFO:tasks.workunit.client.1.smithi149.stdout:4/145: mkdir d2/d24/d2e 0 2024-04-24T08:24:37.625 INFO:tasks.workunit.client.1.smithi149.stdout:1/236: fdatasync d0/d5/db/f1e 0 2024-04-24T08:24:37.626 INFO:tasks.workunit.client.1.smithi149.stdout:8/102: mkdir d1/d17 0 2024-04-24T08:24:37.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/146: rmdir d2/d10/d1a 39 2024-04-24T08:24:37.631 INFO:tasks.workunit.client.1.smithi149.stdout:1/237: creat d0/d5/db/d18/d34/f4e x:0 0 0 2024-04-24T08:24:37.631 INFO:tasks.workunit.client.1.smithi149.stdout:4/147: truncate d2/d24/f2a 246467 0 2024-04-24T08:24:37.637 INFO:tasks.workunit.client.1.smithi149.stdout:4/148: mknod d2/d18/c2f 0 2024-04-24T08:24:37.640 INFO:tasks.workunit.client.1.smithi149.stdout:1/238: mkdir d0/d5/d4f 0 2024-04-24T08:24:37.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/103: rmdir d1/d17 0 2024-04-24T08:24:37.657 INFO:tasks.workunit.client.1.smithi149.stdout:4/149: dwrite d2/ff [0,4194304] 0 2024-04-24T08:24:37.660 INFO:tasks.workunit.client.1.smithi149.stdout:1/239: truncate d0/d5/df/d12/d19/f33 432515 0 2024-04-24T08:24:37.660 INFO:tasks.workunit.client.1.smithi149.stdout:8/104: mkdir d1/de/d18 0 2024-04-24T08:24:37.664 INFO:tasks.workunit.client.1.smithi149.stdout:4/150: unlink d2/d24/f29 0 2024-04-24T08:24:37.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/105: symlink d1/de/d18/l19 0 2024-04-24T08:24:37.667 INFO:tasks.workunit.client.1.smithi149.stdout:1/240: mkdir d0/d5/db/d18/d4d/d50 0 2024-04-24T08:24:37.670 INFO:tasks.workunit.client.1.smithi149.stdout:4/151: creat d2/d10/f30 x:0 0 0 2024-04-24T08:24:37.670 INFO:tasks.workunit.client.1.smithi149.stdout:1/241: chown d0/f16 269 1 2024-04-24T08:24:37.671 INFO:tasks.workunit.client.1.smithi149.stdout:8/106: dread d1/fc [0,4194304] 0 2024-04-24T08:24:37.672 INFO:tasks.workunit.client.1.smithi149.stdout:1/242: read d0/da/f3f [1542246,5793] 0 2024-04-24T08:24:37.686 INFO:tasks.workunit.client.1.smithi149.stdout:8/107: rename d1/f3 to d1/de/f1a 0 2024-04-24T08:24:37.687 INFO:tasks.workunit.client.1.smithi149.stdout:8/108: dread - d1/de/f16 zero size 2024-04-24T08:24:37.691 INFO:tasks.workunit.client.1.smithi149.stdout:1/243: getdents d0/d5 0 2024-04-24T08:24:37.693 INFO:tasks.workunit.client.1.smithi149.stdout:4/152: dread d2/d18/d1e/f15 [0,4194304] 0 2024-04-24T08:24:37.697 INFO:tasks.workunit.client.1.smithi149.stdout:6/124: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:5/218: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:0/136: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:9/149: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/167: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:3/159: sync 2024-04-24T08:24:37.698 INFO:tasks.workunit.client.1.smithi149.stdout:2/152: sync 2024-04-24T08:24:37.707 INFO:tasks.workunit.client.1.smithi149.stdout:7/168: fdatasync d9/dc/d11/d26/f28 0 2024-04-24T08:24:37.707 INFO:tasks.workunit.client.1.smithi149.stdout:2/153: read d0/d6/db/f17 [1924409,111562] 0 2024-04-24T08:24:37.707 INFO:tasks.workunit.client.1.smithi149.stdout:4/153: creat d2/d10/f31 x:0 0 0 2024-04-24T08:24:37.714 INFO:tasks.workunit.client.1.smithi149.stdout:2/154: read d0/d6/db/f26 [3234873,75776] 0 2024-04-24T08:24:37.716 INFO:tasks.workunit.client.1.smithi149.stdout:5/219: creat db/d12/f3d x:0 0 0 2024-04-24T08:24:37.716 INFO:tasks.workunit.client.1.smithi149.stdout:7/169: mkdir d9/dc/d11/d38 0 2024-04-24T08:24:37.721 INFO:tasks.workunit.client.1.smithi149.stdout:8/109: dread d1/de/f1a [0,4194304] 0 2024-04-24T08:24:37.722 INFO:tasks.workunit.client.1.smithi149.stdout:0/137: truncate d2/f8 1133382 0 2024-04-24T08:24:37.730 INFO:tasks.workunit.client.1.smithi149.stdout:9/150: dread d1/da/f8 [0,4194304] 0 2024-04-24T08:24:37.731 INFO:tasks.workunit.client.1.smithi149.stdout:0/138: dread d2/d17/f23 [0,4194304] 0 2024-04-24T08:24:37.731 INFO:tasks.workunit.client.1.smithi149.stdout:5/220: mknod db/d1f/c3e 0 2024-04-24T08:24:37.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/110: rmdir d1 39 2024-04-24T08:24:37.735 INFO:tasks.workunit.client.1.smithi149.stdout:1/244: dwrite d0/da/f3f [0,4194304] 0 2024-04-24T08:24:37.737 INFO:tasks.workunit.client.1.smithi149.stdout:0/139: stat d2/d17/l22 0 2024-04-24T08:24:37.739 INFO:tasks.workunit.client.1.smithi149.stdout:9/151: read - d1/de/f22 zero size 2024-04-24T08:24:37.745 INFO:tasks.workunit.client.1.smithi149.stdout:3/160: dwrite d0/d5/da/d19/d1b/d25/f30 [0,4194304] 0 2024-04-24T08:24:37.746 INFO:tasks.workunit.client.1.smithi149.stdout:6/125: dwrite d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:37.748 INFO:tasks.workunit.client.1.smithi149.stdout:2/155: rename d0/l2b to d0/d6/db/l36 0 2024-04-24T08:24:37.756 INFO:tasks.workunit.client.1.smithi149.stdout:4/154: getdents d2 0 2024-04-24T08:24:37.756 INFO:tasks.workunit.client.1.smithi149.stdout:7/170: dread f1 [0,4194304] 0 2024-04-24T08:24:37.760 INFO:tasks.workunit.client.1.smithi149.stdout:2/156: write d0/d6/f12 [4623582,59100] 0 2024-04-24T08:24:37.760 INFO:tasks.workunit.client.1.smithi149.stdout:0/140: mkdir d2/d17/d26 0 2024-04-24T08:24:37.760 INFO:tasks.workunit.client.1.smithi149.stdout:4/155: chown d2/f19 118444549 1 2024-04-24T08:24:37.760 INFO:tasks.workunit.client.1.smithi149.stdout:6/126: write d0/f1 [4490083,764] 0 2024-04-24T08:24:37.765 INFO:tasks.workunit.client.1.smithi149.stdout:7/171: fsync d9/dc/de/f31 0 2024-04-24T08:24:37.765 INFO:tasks.workunit.client.1.smithi149.stdout:5/221: rename db/d12/c28 to db/c3f 0 2024-04-24T08:24:37.765 INFO:tasks.workunit.client.1.smithi149.stdout:6/127: readlink d0/l22 0 2024-04-24T08:24:37.765 INFO:tasks.workunit.client.1.smithi149.stdout:1/245: mkdir d0/d5/db/d18/d34/d51 0 2024-04-24T08:24:37.768 INFO:tasks.workunit.client.1.smithi149.stdout:8/111: mknod d1/de/c1b 0 2024-04-24T08:24:37.770 INFO:tasks.workunit.client.1.smithi149.stdout:2/157: write d0/d6/db/d27/f31 [673410,84387] 0 2024-04-24T08:24:37.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/161: unlink d0/c10 0 2024-04-24T08:24:37.775 INFO:tasks.workunit.client.1.smithi149.stdout:2/158: truncate d0/f8 4196973 0 2024-04-24T08:24:37.776 INFO:tasks.workunit.client.1.smithi149.stdout:4/156: dread d2/d18/f20 [0,4194304] 0 2024-04-24T08:24:37.777 INFO:tasks.workunit.client.1.smithi149.stdout:5/222: mknod db/d12/d1e/c40 0 2024-04-24T08:24:37.777 INFO:tasks.workunit.client.1.smithi149.stdout:2/159: chown d0 14 1 2024-04-24T08:24:37.778 INFO:tasks.workunit.client.1.smithi149.stdout:7/172: creat d9/dc/d2e/f39 x:0 0 0 2024-04-24T08:24:37.783 INFO:tasks.workunit.client.1.smithi149.stdout:7/173: chown d9/dc/d11/d38 28174 1 2024-04-24T08:24:37.788 INFO:tasks.workunit.client.1.smithi149.stdout:6/128: readlink d0/ld 0 2024-04-24T08:24:37.789 INFO:tasks.workunit.client.1.smithi149.stdout:5/223: creat db/d12/d1e/f41 x:0 0 0 2024-04-24T08:24:37.790 INFO:tasks.workunit.client.1.smithi149.stdout:0/141: rmdir d2 39 2024-04-24T08:24:37.791 INFO:tasks.workunit.client.1.smithi149.stdout:8/112: unlink d1/ca 0 2024-04-24T08:24:37.794 INFO:tasks.workunit.client.1.smithi149.stdout:6/129: dread d0/f26 [0,4194304] 0 2024-04-24T08:24:37.795 INFO:tasks.workunit.client.1.smithi149.stdout:9/152: dwrite d1/d2/d15/d1c/f2a [0,4194304] 0 2024-04-24T08:24:37.797 INFO:tasks.workunit.client.1.smithi149.stdout:4/157: symlink d2/d18/l32 0 2024-04-24T08:24:37.798 INFO:tasks.workunit.client.1.smithi149.stdout:6/130: readlink d0/d1a/l24 0 2024-04-24T08:24:37.802 INFO:tasks.workunit.client.1.smithi149.stdout:2/160: rmdir d0/d6/db/d28 39 2024-04-24T08:24:37.805 INFO:tasks.workunit.client.1.smithi149.stdout:7/174: creat d9/dc/de/f3a x:0 0 0 2024-04-24T08:24:37.812 INFO:tasks.workunit.client.1.smithi149.stdout:0/142: readlink d2/l3 0 2024-04-24T08:24:37.814 INFO:tasks.workunit.client.1.smithi149.stdout:1/246: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:37.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/143: read d2/d9/df/f1d [2950502,76069] 0 2024-04-24T08:24:37.818 INFO:tasks.workunit.client.1.smithi149.stdout:5/224: dwrite db/f16 [0,4194304] 0 2024-04-24T08:24:37.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/153: creat d1/da/dd/f32 x:0 0 0 2024-04-24T08:24:37.826 INFO:tasks.workunit.client.1.smithi149.stdout:4/158: creat d2/d18/d1e/f33 x:0 0 0 2024-04-24T08:24:37.829 INFO:tasks.workunit.client.1.smithi149.stdout:4/159: fdatasync d2/fc 0 2024-04-24T08:24:37.830 INFO:tasks.workunit.client.1.smithi149.stdout:9/154: write d1/d2/d15/f23 [51152,38876] 0 2024-04-24T08:24:37.833 INFO:tasks.workunit.client.1.smithi149.stdout:2/161: creat d0/d6/db/d27/f37 x:0 0 0 2024-04-24T08:24:37.834 INFO:tasks.workunit.client.1.smithi149.stdout:9/155: dread - d1/da/dd/f24 zero size 2024-04-24T08:24:37.840 INFO:tasks.workunit.client.1.smithi149.stdout:4/160: truncate d2/d18/d1e/f33 706588 0 2024-04-24T08:24:37.840 INFO:tasks.workunit.client.1.smithi149.stdout:3/162: creat d0/d26/f37 x:0 0 0 2024-04-24T08:24:37.844 INFO:tasks.workunit.client.1.smithi149.stdout:8/113: symlink d1/l1c 0 2024-04-24T08:24:37.847 INFO:tasks.workunit.client.1.smithi149.stdout:0/144: readlink d2/d9/l14 0 2024-04-24T08:24:37.850 INFO:tasks.workunit.client.1.smithi149.stdout:9/156: mkdir d1/d2/d33 0 2024-04-24T08:24:37.850 INFO:tasks.workunit.client.1.smithi149.stdout:1/247: dwrite d0/d5/df/d12/d19/f23 [0,4194304] 0 2024-04-24T08:24:37.854 INFO:tasks.workunit.client.1.smithi149.stdout:6/131: rename d0/d1a/l24 to d0/l2a 0 2024-04-24T08:24:37.855 INFO:tasks.workunit.client.1.smithi149.stdout:8/114: creat d1/de/d18/f1d x:0 0 0 2024-04-24T08:24:37.858 INFO:tasks.workunit.client.1.smithi149.stdout:2/162: truncate d0/d6/db/d28/f2c 978605 0 2024-04-24T08:24:37.858 INFO:tasks.workunit.client.1.smithi149.stdout:8/115: write d1/de/f16 [856632,49982] 0 2024-04-24T08:24:37.858 INFO:tasks.workunit.client.1.smithi149.stdout:0/145: unlink d2/d9/c1c 0 2024-04-24T08:24:37.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/161: mkdir d2/d10/d1a/d34 0 2024-04-24T08:24:37.868 INFO:tasks.workunit.client.1.smithi149.stdout:1/248: rmdir d0/d5/df 39 2024-04-24T08:24:37.869 INFO:tasks.workunit.client.1.smithi149.stdout:6/132: creat d0/d1a/f2b x:0 0 0 2024-04-24T08:24:37.870 INFO:tasks.workunit.client.1.smithi149.stdout:7/175: dwrite d9/fd [4194304,4194304] 0 2024-04-24T08:24:37.873 INFO:tasks.workunit.client.1.smithi149.stdout:3/163: mkdir d0/d5/da/d19/d1b/d22/d38 0 2024-04-24T08:24:37.877 INFO:tasks.workunit.client.1.smithi149.stdout:7/176: chown d9 15 1 2024-04-24T08:24:37.877 INFO:tasks.workunit.client.1.smithi149.stdout:2/163: creat d0/d6/db/d28/f38 x:0 0 0 2024-04-24T08:24:37.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/116: truncate d1/fc 1180413 0 2024-04-24T08:24:37.881 INFO:tasks.workunit.client.1.smithi149.stdout:3/164: fsync d0/d5/da/f24 0 2024-04-24T08:24:37.887 INFO:tasks.workunit.client.1.smithi149.stdout:3/165: fdatasync d0/d5/f29 0 2024-04-24T08:24:37.891 INFO:tasks.workunit.client.1.smithi149.stdout:9/157: dwrite d1/de/d12/f1d [0,4194304] 0 2024-04-24T08:24:37.900 INFO:tasks.workunit.client.1.smithi149.stdout:4/162: dread d2/d18/d1e/f15 [0,4194304] 0 2024-04-24T08:24:37.900 INFO:tasks.workunit.client.1.smithi149.stdout:1/249: dread - d0/d5/db/f1c zero size 2024-04-24T08:24:37.901 INFO:tasks.workunit.client.1.smithi149.stdout:7/177: creat d9/dc/d11/f3b x:0 0 0 2024-04-24T08:24:37.902 INFO:tasks.workunit.client.1.smithi149.stdout:8/117: fsync d1/f14 0 2024-04-24T08:24:37.903 INFO:tasks.workunit.client.1.smithi149.stdout:0/146: dwrite d2/fb [4194304,4194304] 0 2024-04-24T08:24:37.904 INFO:tasks.workunit.client.1.smithi149.stdout:9/158: chown d1/da/d13/f1e 118950588 1 2024-04-24T08:24:37.913 INFO:tasks.workunit.client.1.smithi149.stdout:6/133: write d0/f26 [1547851,14464] 0 2024-04-24T08:24:37.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/118: symlink d1/de/d18/l1e 0 2024-04-24T08:24:37.917 INFO:tasks.workunit.client.1.smithi149.stdout:4/163: mknod d2/d10/c35 0 2024-04-24T08:24:37.923 INFO:tasks.workunit.client.1.smithi149.stdout:0/147: read d2/d17/f23 [129491,85851] 0 2024-04-24T08:24:37.924 INFO:tasks.workunit.client.1.smithi149.stdout:9/159: mkdir d1/d2/d34 0 2024-04-24T08:24:37.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/166: getdents d0/d5/d1a/d1c 0 2024-04-24T08:24:37.927 INFO:tasks.workunit.client.1.smithi149.stdout:8/119: chown d1/fd 26 1 2024-04-24T08:24:37.928 INFO:tasks.workunit.client.1.smithi149.stdout:6/134: creat d0/d1a/d28/f2c x:0 0 0 2024-04-24T08:24:37.929 INFO:tasks.workunit.client.1.smithi149.stdout:0/148: readlink d2/d9/df/l20 0 2024-04-24T08:24:37.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/135: readlink d0/ld 0 2024-04-24T08:24:37.931 INFO:tasks.workunit.client.1.smithi149.stdout:7/178: dread f3 [4194304,4194304] 0 2024-04-24T08:24:37.935 INFO:tasks.workunit.client.1.smithi149.stdout:3/167: rmdir d0 39 2024-04-24T08:24:37.941 INFO:tasks.workunit.client.1.smithi149.stdout:4/164: truncate d2/f7 2774564 0 2024-04-24T08:24:37.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/164: dwrite d0/d6/f16 [0,4194304] 0 2024-04-24T08:24:37.944 INFO:tasks.workunit.client.1.smithi149.stdout:0/149: unlink c1 0 2024-04-24T08:24:37.946 INFO:tasks.workunit.client.1.smithi149.stdout:6/136: truncate d0/f1f 353615 0 2024-04-24T08:24:37.950 INFO:tasks.workunit.client.1.smithi149.stdout:7/179: mkdir d9/dc/d2e/d3c 0 2024-04-24T08:24:37.951 INFO:tasks.workunit.client.1.smithi149.stdout:2/165: write d0/f22 [1539568,6945] 0 2024-04-24T08:24:37.954 INFO:tasks.workunit.client.1.smithi149.stdout:1/250: write d0/d5/df/d12/d19/f33 [1332961,3999] 0 2024-04-24T08:24:37.955 INFO:tasks.workunit.client.1.smithi149.stdout:6/137: symlink d0/d1a/l2d 0 2024-04-24T08:24:37.960 INFO:tasks.workunit.client.1.smithi149.stdout:4/165: creat d2/d10/d1a/f36 x:0 0 0 2024-04-24T08:24:37.963 INFO:tasks.workunit.client.1.smithi149.stdout:7/180: symlink d9/dc/d2e/l3d 0 2024-04-24T08:24:37.963 INFO:tasks.workunit.client.1.smithi149.stdout:3/168: fdatasync d0/f11 0 2024-04-24T08:24:37.964 INFO:tasks.workunit.client.1.smithi149.stdout:6/138: rmdir d0/d1a/d28 39 2024-04-24T08:24:37.966 INFO:tasks.workunit.client.1.smithi149.stdout:9/160: dwrite d1/de/f22 [0,4194304] 0 2024-04-24T08:24:37.967 INFO:tasks.workunit.client.1.smithi149.stdout:4/166: chown d2/d18/d1e/f33 4 1 2024-04-24T08:24:37.971 INFO:tasks.workunit.client.1.smithi149.stdout:1/251: unlink d0/d1b/f1f 0 2024-04-24T08:24:37.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/150: rmdir d2/d17/d26 0 2024-04-24T08:24:37.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/151: fdatasync d2/d17/f24 0 2024-04-24T08:24:37.981 INFO:tasks.workunit.client.1.smithi149.stdout:8/120: dwrite d1/f14 [0,4194304] 0 2024-04-24T08:24:37.986 INFO:tasks.workunit.client.1.smithi149.stdout:7/181: write d9/dc/f35 [4303716,70539] 0 2024-04-24T08:24:37.987 INFO:tasks.workunit.client.1.smithi149.stdout:2/166: mkdir d0/d6/db/d39 0 2024-04-24T08:24:37.989 INFO:tasks.workunit.client.1.smithi149.stdout:8/121: truncate d1/de/d18/f1d 496233 0 2024-04-24T08:24:37.990 INFO:tasks.workunit.client.1.smithi149.stdout:2/167: chown d0/d6/db/c30 24735 1 2024-04-24T08:24:37.998 INFO:tasks.workunit.client.1.smithi149.stdout:5/225: sync 2024-04-24T08:24:37.998 INFO:tasks.workunit.client.1.smithi149.stdout:3/169: mkdir d0/d5/d1a/d1c/d39 0 2024-04-24T08:24:38.002 INFO:tasks.workunit.client.1.smithi149.stdout:4/167: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:24:38.002 INFO:tasks.workunit.client.1.smithi149.stdout:1/252: dwrite d0/d5/db/d18/f45 [0,4194304] 0 2024-04-24T08:24:38.018 INFO:tasks.workunit.client.1.smithi149.stdout:5/226: rmdir db 39 2024-04-24T08:24:38.025 INFO:tasks.workunit.client.1.smithi149.stdout:9/161: creat d1/d2/d34/f35 x:0 0 0 2024-04-24T08:24:38.026 INFO:tasks.workunit.client.1.smithi149.stdout:2/168: creat d0/d6/db/d28/f3a x:0 0 0 2024-04-24T08:24:38.028 INFO:tasks.workunit.client.1.smithi149.stdout:8/122: dwrite d1/de/f1a [0,4194304] 0 2024-04-24T08:24:38.031 INFO:tasks.workunit.client.1.smithi149.stdout:6/139: creat d0/d1a/f2e x:0 0 0 2024-04-24T08:24:38.032 INFO:tasks.workunit.client.1.smithi149.stdout:0/152: link d2/d9/l14 d2/d9/d1a/l27 0 2024-04-24T08:24:38.036 INFO:tasks.workunit.client.1.smithi149.stdout:4/168: creat d2/d10/d1a/d34/f37 x:0 0 0 2024-04-24T08:24:38.039 INFO:tasks.workunit.client.1.smithi149.stdout:4/169: fdatasync d2/ff 0 2024-04-24T08:24:38.040 INFO:tasks.workunit.client.1.smithi149.stdout:9/162: mknod d1/da/d13/c36 0 2024-04-24T08:24:38.048 INFO:tasks.workunit.client.1.smithi149.stdout:2/169: creat d0/d6/db/d27/f3b x:0 0 0 2024-04-24T08:24:38.049 INFO:tasks.workunit.client.1.smithi149.stdout:1/253: mknod d0/d5/db/d18/d4d/d50/c52 0 2024-04-24T08:24:38.053 INFO:tasks.workunit.client.1.smithi149.stdout:8/123: mkdir d1/d1f 0 2024-04-24T08:24:38.060 INFO:tasks.workunit.client.1.smithi149.stdout:5/227: dwrite db/f16 [0,4194304] 0 2024-04-24T08:24:38.064 INFO:tasks.workunit.client.1.smithi149.stdout:0/153: rename d2/d9/d1a/l27 to d2/d9/l28 0 2024-04-24T08:24:38.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/140: rmdir d0/d1a 39 2024-04-24T08:24:38.065 INFO:tasks.workunit.client.1.smithi149.stdout:9/163: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:38.065 INFO:tasks.workunit.client.1.smithi149.stdout:4/170: getdents d2/d24/d2e 0 2024-04-24T08:24:38.067 INFO:tasks.workunit.client.1.smithi149.stdout:6/141: readlink d0/l1e 0 2024-04-24T08:24:38.067 INFO:tasks.workunit.client.1.smithi149.stdout:4/171: readlink d2/l3 0 2024-04-24T08:24:38.070 INFO:tasks.workunit.client.1.smithi149.stdout:1/254: mkdir d0/d39/d53 0 2024-04-24T08:24:38.070 INFO:tasks.workunit.client.1.smithi149.stdout:5/228: rmdir db/d12 39 2024-04-24T08:24:38.075 INFO:tasks.workunit.client.1.smithi149.stdout:2/170: dread d0/d6/db/f1f [0,4194304] 0 2024-04-24T08:24:38.076 INFO:tasks.workunit.client.1.smithi149.stdout:5/229: fsync db/d1c/f21 0 2024-04-24T08:24:38.078 INFO:tasks.workunit.client.1.smithi149.stdout:3/170: dread d0/d5/da/d19/f20 [8388608,4194304] 0 2024-04-24T08:24:38.078 INFO:tasks.workunit.client.1.smithi149.stdout:9/164: fdatasync d1/d2/d15/d1c/f20 0 2024-04-24T08:24:38.080 INFO:tasks.workunit.client.1.smithi149.stdout:2/171: chown d0/d6/db/f26 875170 1 2024-04-24T08:24:38.081 INFO:tasks.workunit.client.1.smithi149.stdout:6/142: chown d0/d1a/f1b 4845 1 2024-04-24T08:24:38.083 INFO:tasks.workunit.client.1.smithi149.stdout:2/172: write d0/d6/f12 [2347212,27173] 0 2024-04-24T08:24:38.088 INFO:tasks.workunit.client.1.smithi149.stdout:6/143: chown d0/d1a/f2b 31811630 1 2024-04-24T08:24:38.094 INFO:tasks.workunit.client.1.smithi149.stdout:0/154: dwrite d2/d17/f1f [0,4194304] 0 2024-04-24T08:24:38.096 INFO:tasks.workunit.client.1.smithi149.stdout:0/155: chown d2/lc 24243808 1 2024-04-24T08:24:38.100 INFO:tasks.workunit.client.1.smithi149.stdout:2/173: read d0/d6/f12 [4546190,87993] 0 2024-04-24T08:24:38.104 INFO:tasks.workunit.client.1.smithi149.stdout:4/172: dwrite d2/d18/d1e/f21 [0,4194304] 0 2024-04-24T08:24:38.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/255: dwrite d0/d5/db/d18/f45 [0,4194304] 0 2024-04-24T08:24:38.105 INFO:tasks.workunit.client.1.smithi149.stdout:4/173: chown d2/ff 50 1 2024-04-24T08:24:38.111 INFO:tasks.workunit.client.1.smithi149.stdout:6/144: rmdir d0/d1a 39 2024-04-24T08:24:38.113 INFO:tasks.workunit.client.1.smithi149.stdout:1/256: fdatasync d0/d5/df/d12/d19/f33 0 2024-04-24T08:24:38.115 INFO:tasks.workunit.client.1.smithi149.stdout:8/124: link d1/l11 d1/l20 0 2024-04-24T08:24:38.122 INFO:tasks.workunit.client.1.smithi149.stdout:8/125: dread d1/de/d18/f1d [0,4194304] 0 2024-04-24T08:24:38.132 INFO:tasks.workunit.client.1.smithi149.stdout:2/174: dwrite d0/f22 [0,4194304] 0 2024-04-24T08:24:38.134 INFO:tasks.workunit.client.1.smithi149.stdout:2/175: chown d0/d6/db/f1c 0 1 2024-04-24T08:24:38.137 INFO:tasks.workunit.client.1.smithi149.stdout:2/176: chown d0/d6/db/f1f 316667 1 2024-04-24T08:24:38.137 INFO:tasks.workunit.client.1.smithi149.stdout:3/171: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:38.141 INFO:tasks.workunit.client.1.smithi149.stdout:3/172: write d0/d5/f29 [1246332,23974] 0 2024-04-24T08:24:38.142 INFO:tasks.workunit.client.1.smithi149.stdout:1/257: mknod d0/d5/df/d12/d19/c54 0 2024-04-24T08:24:38.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/258: chown d0/d1b/d4b 862324 1 2024-04-24T08:24:38.147 INFO:tasks.workunit.client.1.smithi149.stdout:9/165: link d1/da/d13/l2d d1/d2/d15/l37 0 2024-04-24T08:24:38.151 INFO:tasks.workunit.client.1.smithi149.stdout:4/174: mkdir d2/d24/d2e/d38 0 2024-04-24T08:24:38.154 INFO:tasks.workunit.client.1.smithi149.stdout:5/230: link db/d1f/f2a db/d1c/f42 0 2024-04-24T08:24:38.159 INFO:tasks.workunit.client.1.smithi149.stdout:0/156: write d2/d9/df/f1d [3848294,111811] 0 2024-04-24T08:24:38.160 INFO:tasks.workunit.client.1.smithi149.stdout:9/166: dread d1/d2/f2b [0,4194304] 0 2024-04-24T08:24:38.164 INFO:tasks.workunit.client.1.smithi149.stdout:3/173: rename d0/d5/da/d19/d1b/d2b to d0/d5/d1a/d1c/d34/d3a 0 2024-04-24T08:24:38.164 INFO:tasks.workunit.client.1.smithi149.stdout:2/177: dwrite d0/f35 [0,4194304] 0 2024-04-24T08:24:38.166 INFO:tasks.workunit.client.1.smithi149.stdout:8/126: dwrite d1/de/ff [0,4194304] 0 2024-04-24T08:24:38.168 INFO:tasks.workunit.client.1.smithi149.stdout:6/145: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:38.173 INFO:tasks.workunit.client.1.smithi149.stdout:2/178: truncate d0/d6/db/d27/f3b 15015 0 2024-04-24T08:24:38.173 INFO:tasks.workunit.client.1.smithi149.stdout:8/127: truncate d1/de/f16 1941563 0 2024-04-24T08:24:38.176 INFO:tasks.workunit.client.1.smithi149.stdout:2/179: fdatasync d0/f8 0 2024-04-24T08:24:38.184 INFO:tasks.workunit.client.1.smithi149.stdout:4/175: creat d2/d24/f39 x:0 0 0 2024-04-24T08:24:38.186 INFO:tasks.workunit.client.1.smithi149.stdout:0/157: write d2/fb [3768890,26247] 0 2024-04-24T08:24:38.196 INFO:tasks.workunit.client.1.smithi149.stdout:3/174: creat d0/d5/d1a/d1c/d34/f3b x:0 0 0 2024-04-24T08:24:38.208 INFO:tasks.workunit.client.1.smithi149.stdout:1/259: dwrite d0/d5/f41 [0,4194304] 0 2024-04-24T08:24:38.209 INFO:tasks.workunit.client.1.smithi149.stdout:5/231: creat db/d23/f43 x:0 0 0 2024-04-24T08:24:38.213 INFO:tasks.workunit.client.1.smithi149.stdout:0/158: mknod d2/d9/d1a/c29 0 2024-04-24T08:24:38.213 INFO:tasks.workunit.client.1.smithi149.stdout:5/232: dread - db/d23/f43 zero size 2024-04-24T08:24:38.220 INFO:tasks.workunit.client.1.smithi149.stdout:3/175: mkdir d0/d26/d3c 0 2024-04-24T08:24:38.222 INFO:tasks.workunit.client.1.smithi149.stdout:6/146: dwrite d0/d1a/f27 [4194304,4194304] 0 2024-04-24T08:24:38.226 INFO:tasks.workunit.client.1.smithi149.stdout:8/128: unlink d1/f8 0 2024-04-24T08:24:38.229 INFO:tasks.workunit.client.1.smithi149.stdout:7/182: sync 2024-04-24T08:24:38.238 INFO:tasks.workunit.client.1.smithi149.stdout:9/167: truncate d1/da/f27 5269976 0 2024-04-24T08:24:38.246 INFO:tasks.workunit.client.1.smithi149.stdout:7/183: dwrite d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:38.246 INFO:tasks.workunit.client.1.smithi149.stdout:0/159: rename d2/lc to d2/d17/l2a 0 2024-04-24T08:24:38.258 INFO:tasks.workunit.client.1.smithi149.stdout:4/176: creat d2/d10/f3a x:0 0 0 2024-04-24T08:24:38.261 INFO:tasks.workunit.client.1.smithi149.stdout:9/168: dwrite d1/da/f29 [0,4194304] 0 2024-04-24T08:24:38.264 INFO:tasks.workunit.client.1.smithi149.stdout:9/169: chown d1/da/c31 2 1 2024-04-24T08:24:38.268 INFO:tasks.workunit.client.1.smithi149.stdout:8/129: rename d1/f14 to d1/de/d18/f21 0 2024-04-24T08:24:38.275 INFO:tasks.workunit.client.1.smithi149.stdout:0/160: dwrite d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:38.281 INFO:tasks.workunit.client.1.smithi149.stdout:6/147: mkdir d0/d1a/d2f 0 2024-04-24T08:24:38.284 INFO:tasks.workunit.client.1.smithi149.stdout:2/180: link d0/l18 d0/d6/db/d27/l3c 0 2024-04-24T08:24:38.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/260: truncate d0/fe 1451426 0 2024-04-24T08:24:38.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/177: write d2/f19 [3765343,79796] 0 2024-04-24T08:24:38.302 INFO:tasks.workunit.client.1.smithi149.stdout:0/161: mknod d2/d17/c2b 0 2024-04-24T08:24:38.304 INFO:tasks.workunit.client.1.smithi149.stdout:2/181: dwrite d0/d6/db/f15 [0,4194304] 0 2024-04-24T08:24:38.307 INFO:tasks.workunit.client.1.smithi149.stdout:9/170: symlink d1/l38 0 2024-04-24T08:24:38.309 INFO:tasks.workunit.client.1.smithi149.stdout:1/261: symlink d0/d39/l55 0 2024-04-24T08:24:38.309 INFO:tasks.workunit.client.1.smithi149.stdout:9/171: chown d1/l38 8 1 2024-04-24T08:24:38.311 INFO:tasks.workunit.client.1.smithi149.stdout:4/178: unlink d2/d10/f3a 0 2024-04-24T08:24:38.316 INFO:tasks.workunit.client.1.smithi149.stdout:8/130: mkdir d1/d1f/d22 0 2024-04-24T08:24:38.317 INFO:tasks.workunit.client.1.smithi149.stdout:3/176: link d0/l1f d0/d5/d1a/d1c/d39/l3d 0 2024-04-24T08:24:38.319 INFO:tasks.workunit.client.1.smithi149.stdout:0/162: mknod d2/d9/df/c2c 0 2024-04-24T08:24:38.321 INFO:tasks.workunit.client.1.smithi149.stdout:0/163: chown d2/d9/df/f1d 663440925 1 2024-04-24T08:24:38.322 INFO:tasks.workunit.client.1.smithi149.stdout:6/148: symlink d0/d1a/l30 0 2024-04-24T08:24:38.325 INFO:tasks.workunit.client.1.smithi149.stdout:6/149: truncate d0/d1a/f25 398122 0 2024-04-24T08:24:38.326 INFO:tasks.workunit.client.1.smithi149.stdout:5/233: getdents db/d12/d22 0 2024-04-24T08:24:38.330 INFO:tasks.workunit.client.1.smithi149.stdout:1/262: creat d0/d5/db/d18/d34/f56 x:0 0 0 2024-04-24T08:24:38.331 INFO:tasks.workunit.client.1.smithi149.stdout:3/177: dread d0/fc [0,4194304] 0 2024-04-24T08:24:38.335 INFO:tasks.workunit.client.1.smithi149.stdout:4/179: mkdir d2/d18/d1e/d3b 0 2024-04-24T08:24:38.338 INFO:tasks.workunit.client.1.smithi149.stdout:7/184: getdents d9/dc/de 0 2024-04-24T08:24:38.339 INFO:tasks.workunit.client.1.smithi149.stdout:8/131: stat d1/l6 0 2024-04-24T08:24:38.339 INFO:tasks.workunit.client.1.smithi149.stdout:7/185: chown d9/dc/de/f33 3 1 2024-04-24T08:24:38.342 INFO:tasks.workunit.client.1.smithi149.stdout:8/132: write d1/de/f16 [109348,50567] 0 2024-04-24T08:24:38.344 INFO:tasks.workunit.client.1.smithi149.stdout:0/164: creat d2/d9/f2d x:0 0 0 2024-04-24T08:24:38.349 INFO:tasks.workunit.client.1.smithi149.stdout:2/182: mkdir d0/d6/db/d39/d3d 0 2024-04-24T08:24:38.350 INFO:tasks.workunit.client.1.smithi149.stdout:3/178: dwrite d0/d26/f37 [0,4194304] 0 2024-04-24T08:24:38.352 INFO:tasks.workunit.client.1.smithi149.stdout:5/234: creat db/d23/f44 x:0 0 0 2024-04-24T08:24:38.359 INFO:tasks.workunit.client.1.smithi149.stdout:4/180: unlink d2/ce 0 2024-04-24T08:24:38.360 INFO:tasks.workunit.client.1.smithi149.stdout:4/181: fsync d2/d18/d1e/f21 0 2024-04-24T08:24:38.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/183: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:38.367 INFO:tasks.workunit.client.1.smithi149.stdout:6/150: mkdir d0/d1a/d28/d31 0 2024-04-24T08:24:38.373 INFO:tasks.workunit.client.1.smithi149.stdout:5/235: dwrite db/d1c/f21 [0,4194304] 0 2024-04-24T08:24:38.380 INFO:tasks.workunit.client.1.smithi149.stdout:3/179: dread d0/d5/f16 [0,4194304] 0 2024-04-24T08:24:38.382 INFO:tasks.workunit.client.1.smithi149.stdout:7/186: mknod d9/c3e 0 2024-04-24T08:24:38.383 INFO:tasks.workunit.client.1.smithi149.stdout:9/172: rename d1/da/d13/c36 to d1/da/c39 0 2024-04-24T08:24:38.391 INFO:tasks.workunit.client.1.smithi149.stdout:8/133: symlink d1/d1f/d22/l23 0 2024-04-24T08:24:38.392 INFO:tasks.workunit.client.1.smithi149.stdout:4/182: mkdir d2/d18/d1e/d3c 0 2024-04-24T08:24:38.394 INFO:tasks.workunit.client.1.smithi149.stdout:4/183: stat d2/l26 0 2024-04-24T08:24:38.398 INFO:tasks.workunit.client.1.smithi149.stdout:0/165: rename d2/d17/f1f to d2/d9/d1a/f2e 0 2024-04-24T08:24:38.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/151: dwrite d0/d1a/d28/f2c [0,4194304] 0 2024-04-24T08:24:38.403 INFO:tasks.workunit.client.1.smithi149.stdout:9/173: creat d1/d2/d15/d1c/f3a x:0 0 0 2024-04-24T08:24:38.407 INFO:tasks.workunit.client.1.smithi149.stdout:2/184: dwrite d0/d6/db/d28/f38 [0,4194304] 0 2024-04-24T08:24:38.412 INFO:tasks.workunit.client.1.smithi149.stdout:7/187: dread d9/fd [0,4194304] 0 2024-04-24T08:24:38.414 INFO:tasks.workunit.client.1.smithi149.stdout:9/174: read d1/de/f18 [2441510,10660] 0 2024-04-24T08:24:38.415 INFO:tasks.workunit.client.1.smithi149.stdout:8/134: symlink d1/d1f/d22/l24 0 2024-04-24T08:24:38.416 INFO:tasks.workunit.client.1.smithi149.stdout:5/236: rename db/d1c to db/d12/d22/d45 0 2024-04-24T08:24:38.421 INFO:tasks.workunit.client.1.smithi149.stdout:0/166: mkdir d2/d9/df/d2f 0 2024-04-24T08:24:38.424 INFO:tasks.workunit.client.1.smithi149.stdout:6/152: chown d0/l2a 6302 1 2024-04-24T08:24:38.428 INFO:tasks.workunit.client.1.smithi149.stdout:4/184: dwrite d2/d10/f31 [0,4194304] 0 2024-04-24T08:24:38.428 INFO:tasks.workunit.client.1.smithi149.stdout:6/153: write d0/d1a/d28/f2c [254607,2267] 0 2024-04-24T08:24:38.437 INFO:tasks.workunit.client.1.smithi149.stdout:7/188: symlink d9/dc/de/l3f 0 2024-04-24T08:24:38.438 INFO:tasks.workunit.client.1.smithi149.stdout:9/175: mknod d1/da/d13/c3b 0 2024-04-24T08:24:38.440 INFO:tasks.workunit.client.1.smithi149.stdout:8/135: dread d1/de/f1a [0,4194304] 0 2024-04-24T08:24:38.444 INFO:tasks.workunit.client.1.smithi149.stdout:5/237: dread db/d12/d22/d45/f21 [0,4194304] 0 2024-04-24T08:24:38.446 INFO:tasks.workunit.client.1.smithi149.stdout:3/180: dread d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:38.451 INFO:tasks.workunit.client.1.smithi149.stdout:4/185: creat d2/f3d x:0 0 0 2024-04-24T08:24:38.453 INFO:tasks.workunit.client.1.smithi149.stdout:3/181: truncate d0/d5/da/fb 1671883 0 2024-04-24T08:24:38.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/176: creat d1/d2/d15/f3c x:0 0 0 2024-04-24T08:24:38.459 INFO:tasks.workunit.client.1.smithi149.stdout:1/263: write d0/d5/df/f28 [190146,128375] 0 2024-04-24T08:24:38.460 INFO:tasks.workunit.client.1.smithi149.stdout:8/136: mkdir d1/d1f/d25 0 2024-04-24T08:24:38.463 INFO:tasks.workunit.client.1.smithi149.stdout:0/167: creat d2/d9/df/d2f/f30 x:0 0 0 2024-04-24T08:24:38.466 INFO:tasks.workunit.client.1.smithi149.stdout:5/238: creat db/d12/d2f/f46 x:0 0 0 2024-04-24T08:24:38.470 INFO:tasks.workunit.client.1.smithi149.stdout:3/182: mknod d0/d5/da/c3e 0 2024-04-24T08:24:38.471 INFO:tasks.workunit.client.1.smithi149.stdout:7/189: mkdir d9/d40 0 2024-04-24T08:24:38.475 INFO:tasks.workunit.client.1.smithi149.stdout:9/177: dread d1/de/f22 [0,4194304] 0 2024-04-24T08:24:38.476 INFO:tasks.workunit.client.1.smithi149.stdout:3/183: chown d0/d5/d1a/d1c/d34/d3a/f33 2436 1 2024-04-24T08:24:38.477 INFO:tasks.workunit.client.1.smithi149.stdout:8/137: symlink d1/de/d18/l26 0 2024-04-24T08:24:38.481 INFO:tasks.workunit.client.1.smithi149.stdout:0/168: mknod d2/d9/df/d2f/c31 0 2024-04-24T08:24:38.483 INFO:tasks.workunit.client.1.smithi149.stdout:9/178: write d1/da/f2c [384389,94888] 0 2024-04-24T08:24:38.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/239: mknod db/d12/d22/c47 0 2024-04-24T08:24:38.486 INFO:tasks.workunit.client.1.smithi149.stdout:4/186: link d2/d24/f2a d2/d10/d1a/f3e 0 2024-04-24T08:24:38.487 INFO:tasks.workunit.client.1.smithi149.stdout:6/154: dread d0/fa [0,4194304] 0 2024-04-24T08:24:38.490 INFO:tasks.workunit.client.1.smithi149.stdout:4/187: read d2/d18/d1e/f15 [1234679,109295] 0 2024-04-24T08:24:38.491 INFO:tasks.workunit.client.1.smithi149.stdout:1/264: dwrite d0/d5/df/d12/d19/f33 [0,4194304] 0 2024-04-24T08:24:38.491 INFO:tasks.workunit.client.1.smithi149.stdout:5/240: truncate db/d12/d1e/f3c 44911 0 2024-04-24T08:24:38.493 INFO:tasks.workunit.client.1.smithi149.stdout:7/190: write d9/dc/f1b [384095,108060] 0 2024-04-24T08:24:38.494 INFO:tasks.workunit.client.1.smithi149.stdout:5/241: fdatasync db/d12/d22/f31 0 2024-04-24T08:24:38.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/188: chown d2/d10/d1a/f36 14 1 2024-04-24T08:24:38.504 INFO:tasks.workunit.client.1.smithi149.stdout:8/138: creat d1/d1f/f27 x:0 0 0 2024-04-24T08:24:38.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/185: write d0/d6/f12 [3561584,51574] 0 2024-04-24T08:24:38.505 INFO:tasks.workunit.client.1.smithi149.stdout:3/184: creat d0/d5/d1a/d1c/f3f x:0 0 0 2024-04-24T08:24:38.513 INFO:tasks.workunit.client.1.smithi149.stdout:8/139: write d1/fd [60767,36796] 0 2024-04-24T08:24:38.514 INFO:tasks.workunit.client.1.smithi149.stdout:4/189: mkdir d2/d24/d2e/d3f 0 2024-04-24T08:24:38.516 INFO:tasks.workunit.client.1.smithi149.stdout:4/190: readlink d2/l12 0 2024-04-24T08:24:38.520 INFO:tasks.workunit.client.1.smithi149.stdout:3/185: symlink d0/d5/l40 0 2024-04-24T08:24:38.523 INFO:tasks.workunit.client.1.smithi149.stdout:7/191: dread f1 [4194304,4194304] 0 2024-04-24T08:24:38.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:38 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:24:38.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:38 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:24:38.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:38 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:24:38.529 INFO:tasks.workunit.client.1.smithi149.stdout:0/169: rmdir d2/d9/df/d2f 39 2024-04-24T08:24:38.530 INFO:tasks.workunit.client.1.smithi149.stdout:0/170: fdatasync d2/d9/df/f16 0 2024-04-24T08:24:38.531 INFO:tasks.workunit.client.1.smithi149.stdout:1/265: truncate d0/fd 1544081 0 2024-04-24T08:24:38.533 INFO:tasks.workunit.client.1.smithi149.stdout:1/266: stat d0/d39/d53 0 2024-04-24T08:24:38.538 INFO:tasks.workunit.client.1.smithi149.stdout:7/192: creat d9/dc/d2e/f41 x:0 0 0 2024-04-24T08:24:38.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:38 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:24:38.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:38 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:24:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:38 smithi090 ceph-mon[25016]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:24:38.544 INFO:tasks.workunit.client.1.smithi149.stdout:0/171: mknod d2/d9/d1a/c32 0 2024-04-24T08:24:38.545 INFO:tasks.workunit.client.1.smithi149.stdout:6/155: write d0/f4 [1823989,41030] 0 2024-04-24T08:24:38.546 INFO:tasks.workunit.client.1.smithi149.stdout:1/267: creat d0/d5/df/d12/d19/f57 x:0 0 0 2024-04-24T08:24:38.546 INFO:tasks.workunit.client.1.smithi149.stdout:5/242: write db/f20 [252998,98153] 0 2024-04-24T08:24:38.549 INFO:tasks.workunit.client.1.smithi149.stdout:9/179: write d1/d2/f19 [625906,119669] 0 2024-04-24T08:24:38.550 INFO:tasks.workunit.client.1.smithi149.stdout:2/186: link d0/d6/db/d28/f38 d0/d6/db/f3e 0 2024-04-24T08:24:38.555 INFO:tasks.workunit.client.1.smithi149.stdout:7/193: unlink f3 0 2024-04-24T08:24:38.555 INFO:tasks.workunit.client.1.smithi149.stdout:1/268: truncate d0/d5/db/d18/f2c 106276 0 2024-04-24T08:24:38.558 INFO:tasks.workunit.client.1.smithi149.stdout:0/172: rename d2/d17/c2b to d2/d17/c33 0 2024-04-24T08:24:38.558 INFO:tasks.workunit.client.1.smithi149.stdout:7/194: fsync d9/dc/f1b 0 2024-04-24T08:24:38.559 INFO:tasks.workunit.client.1.smithi149.stdout:9/180: fsync d1/de/f1a 0 2024-04-24T08:24:38.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/140: truncate d1/de/f1a 2681593 0 2024-04-24T08:24:38.569 INFO:tasks.workunit.client.1.smithi149.stdout:4/191: write d2/f7 [727871,24996] 0 2024-04-24T08:24:38.569 INFO:tasks.workunit.client.1.smithi149.stdout:2/187: write d0/d6/db/f17 [364952,70961] 0 2024-04-24T08:24:38.571 INFO:tasks.workunit.client.1.smithi149.stdout:6/156: dwrite d0/f1f [0,4194304] 0 2024-04-24T08:24:38.574 INFO:tasks.workunit.client.1.smithi149.stdout:9/181: creat d1/de/d12/f3d x:0 0 0 2024-04-24T08:24:38.575 INFO:tasks.workunit.client.1.smithi149.stdout:6/157: stat d0/f26 0 2024-04-24T08:24:38.576 INFO:tasks.workunit.client.1.smithi149.stdout:0/173: dread d2/f8 [0,4194304] 0 2024-04-24T08:24:38.576 INFO:tasks.workunit.client.1.smithi149.stdout:2/188: dread - d0/d6/db/d27/f37 zero size 2024-04-24T08:24:38.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/186: dwrite d0/d5/da/d19/f20 [8388608,4194304] 0 2024-04-24T08:24:38.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/174: write d2/d9/d1a/f25 [617782,131044] 0 2024-04-24T08:24:38.586 INFO:tasks.workunit.client.1.smithi149.stdout:1/269: creat d0/d1b/d4b/f58 x:0 0 0 2024-04-24T08:24:38.591 INFO:tasks.workunit.client.1.smithi149.stdout:1/270: write d0/f4 [5320942,8782] 0 2024-04-24T08:24:38.591 INFO:tasks.workunit.client.1.smithi149.stdout:9/182: unlink d1/f26 0 2024-04-24T08:24:38.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/243: getdents db/d1f 0 2024-04-24T08:24:38.605 INFO:tasks.workunit.client.1.smithi149.stdout:7/195: link d9/fd d9/dc/d11/f42 0 2024-04-24T08:24:38.607 INFO:tasks.workunit.client.1.smithi149.stdout:8/141: fsync d1/de/d18/f21 0 2024-04-24T08:24:38.607 INFO:tasks.workunit.client.1.smithi149.stdout:4/192: creat d2/d24/d2e/d3f/f40 x:0 0 0 2024-04-24T08:24:38.607 INFO:tasks.workunit.client.1.smithi149.stdout:7/196: chown c8 1 1 2024-04-24T08:24:38.609 INFO:tasks.workunit.client.1.smithi149.stdout:2/189: truncate d0/d6/db/f3e 3701775 0 2024-04-24T08:24:38.620 INFO:tasks.workunit.client.1.smithi149.stdout:2/190: symlink d0/d6/db/d39/l3f 0 2024-04-24T08:24:38.622 INFO:tasks.workunit.client.1.smithi149.stdout:8/142: chown d1/fc 5536 1 2024-04-24T08:24:38.623 INFO:tasks.workunit.client.1.smithi149.stdout:6/158: truncate d0/f26 1464607 0 2024-04-24T08:24:38.623 INFO:tasks.workunit.client.1.smithi149.stdout:0/175: dwrite d2/fb [4194304,4194304] 0 2024-04-24T08:24:38.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/159: write d0/f11 [2187946,51672] 0 2024-04-24T08:24:38.639 INFO:tasks.workunit.client.1.smithi149.stdout:6/160: chown d0/f10 366306993 1 2024-04-24T08:24:38.640 INFO:tasks.workunit.client.1.smithi149.stdout:9/183: dwrite d1/de/ff [0,4194304] 0 2024-04-24T08:24:38.641 INFO:tasks.workunit.client.1.smithi149.stdout:0/176: write d2/d9/df/f1d [518767,17054] 0 2024-04-24T08:24:38.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/187: truncate d0/d5/da/d19/f20 9126618 0 2024-04-24T08:24:38.646 INFO:tasks.workunit.client.1.smithi149.stdout:4/193: dwrite d2/d10/d1a/d34/f37 [0,4194304] 0 2024-04-24T08:24:38.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/271: rename d0/d5/df/l22 to d0/d1b/l59 0 2024-04-24T08:24:38.656 INFO:tasks.workunit.client.1.smithi149.stdout:7/197: dwrite d9/dc/de/f12 [0,4194304] 0 2024-04-24T08:24:38.661 INFO:tasks.workunit.client.1.smithi149.stdout:7/198: chown d9/dc/de/f22 710202061 1 2024-04-24T08:24:38.669 INFO:tasks.workunit.client.1.smithi149.stdout:8/143: unlink d1/de/d18/f21 0 2024-04-24T08:24:38.680 INFO:tasks.workunit.client.1.smithi149.stdout:2/191: symlink d0/l40 0 2024-04-24T08:24:38.680 INFO:tasks.workunit.client.1.smithi149.stdout:5/244: getdents db/d12/d2f 0 2024-04-24T08:24:38.687 INFO:tasks.workunit.client.1.smithi149.stdout:8/144: creat d1/d1f/f28 x:0 0 0 2024-04-24T08:24:38.687 INFO:tasks.workunit.client.1.smithi149.stdout:4/194: dwrite d2/d24/f39 [0,4194304] 0 2024-04-24T08:24:38.694 INFO:tasks.workunit.client.1.smithi149.stdout:4/195: chown d2/d10/f31 58254253 1 2024-04-24T08:24:38.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/184: dread d1/d2/d15/d1c/f2a [0,4194304] 0 2024-04-24T08:24:38.701 INFO:tasks.workunit.client.1.smithi149.stdout:2/192: chown d0/d6/c23 492 1 2024-04-24T08:24:38.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/196: chown d2/l26 207 1 2024-04-24T08:24:38.707 INFO:tasks.workunit.client.1.smithi149.stdout:9/185: fsync d1/d2/d34/f35 0 2024-04-24T08:24:38.707 INFO:tasks.workunit.client.1.smithi149.stdout:7/199: mkdir d9/d43 0 2024-04-24T08:24:38.712 INFO:tasks.workunit.client.1.smithi149.stdout:6/161: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:38.713 INFO:tasks.workunit.client.1.smithi149.stdout:2/193: fsync d0/d6/db/d27/f31 0 2024-04-24T08:24:38.715 INFO:tasks.workunit.client.1.smithi149.stdout:3/188: dwrite d0/d5/d1a/d1c/d34/f3b [0,4194304] 0 2024-04-24T08:24:38.715 INFO:tasks.workunit.client.1.smithi149.stdout:0/177: dwrite d2/d9/f13 [0,4194304] 0 2024-04-24T08:24:38.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/186: fsync d1/d2/d15/d1c/f3a 0 2024-04-24T08:24:38.728 INFO:tasks.workunit.client.1.smithi149.stdout:9/187: stat d1/da/fc 0 2024-04-24T08:24:38.728 INFO:tasks.workunit.client.1.smithi149.stdout:3/189: write d0/d5/d1a/f2d [1569566,109259] 0 2024-04-24T08:24:38.730 INFO:tasks.workunit.client.1.smithi149.stdout:0/178: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:38.743 INFO:tasks.workunit.client.1.smithi149.stdout:7/200: rename d9/d40 to d9/dc/de/d44 0 2024-04-24T08:24:38.744 INFO:tasks.workunit.client.1.smithi149.stdout:5/245: dwrite db/f14 [0,4194304] 0 2024-04-24T08:24:38.750 INFO:tasks.workunit.client.1.smithi149.stdout:4/197: mknod d2/d18/d1e/d3b/c41 0 2024-04-24T08:24:38.750 INFO:tasks.workunit.client.1.smithi149.stdout:5/246: readlink db/d1f/l3a 0 2024-04-24T08:24:38.754 INFO:tasks.workunit.client.1.smithi149.stdout:5/247: write db/f20 [682987,93977] 0 2024-04-24T08:24:38.755 INFO:tasks.workunit.client.1.smithi149.stdout:3/190: dread d0/d5/da/d19/d1b/d25/f30 [0,4194304] 0 2024-04-24T08:24:38.757 INFO:tasks.workunit.client.1.smithi149.stdout:2/194: dwrite d0/d6/f12 [0,4194304] 0 2024-04-24T08:24:38.776 INFO:tasks.workunit.client.1.smithi149.stdout:3/191: dread d0/d5/f29 [0,4194304] 0 2024-04-24T08:24:38.776 INFO:tasks.workunit.client.1.smithi149.stdout:7/201: symlink d9/dc/de/l45 0 2024-04-24T08:24:38.780 INFO:tasks.workunit.client.1.smithi149.stdout:5/248: mknod db/d12/d22/d45/c48 0 2024-04-24T08:24:38.788 INFO:tasks.workunit.client.1.smithi149.stdout:0/179: creat d2/d9/df/d2f/f34 x:0 0 0 2024-04-24T08:24:38.792 INFO:tasks.workunit.client.1.smithi149.stdout:6/162: rmdir d0/d1a/d2f 0 2024-04-24T08:24:38.794 INFO:tasks.workunit.client.1.smithi149.stdout:3/192: dread d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:38.795 INFO:tasks.workunit.client.1.smithi149.stdout:7/202: mknod d9/dc/de/c46 0 2024-04-24T08:24:38.796 INFO:tasks.workunit.client.1.smithi149.stdout:1/272: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:24:38.797 INFO:tasks.workunit.client.1.smithi149.stdout:7/203: stat d9/f19 0 2024-04-24T08:24:38.801 INFO:tasks.workunit.client.1.smithi149.stdout:9/188: write d1/de/f1a [2658490,5427] 0 2024-04-24T08:24:38.804 INFO:tasks.workunit.client.1.smithi149.stdout:8/145: link d1/c15 d1/c29 0 2024-04-24T08:24:38.807 INFO:tasks.workunit.client.1.smithi149.stdout:8/146: fdatasync d1/d1f/f27 0 2024-04-24T08:24:38.816 INFO:tasks.workunit.client.1.smithi149.stdout:5/249: link db/d23/f44 db/d12/d1e/f49 0 2024-04-24T08:24:38.816 INFO:tasks.workunit.client.1.smithi149.stdout:7/204: dread d9/dc/d11/f42 [0,4194304] 0 2024-04-24T08:24:38.818 INFO:tasks.workunit.client.1.smithi149.stdout:8/147: symlink d1/d1f/d22/l2a 0 2024-04-24T08:24:38.822 INFO:tasks.workunit.client.1.smithi149.stdout:7/205: write d9/dc/de/f16 [7287774,9378] 0 2024-04-24T08:24:38.823 INFO:tasks.workunit.client.1.smithi149.stdout:3/193: dwrite d0/d5/da/f28 [0,4194304] 0 2024-04-24T08:24:38.824 INFO:tasks.workunit.client.1.smithi149.stdout:7/206: stat d9/dc/de/f12 0 2024-04-24T08:24:38.832 INFO:tasks.workunit.client.1.smithi149.stdout:6/163: rename d0/d1a/f1b to d0/d1a/f32 0 2024-04-24T08:24:38.834 INFO:tasks.workunit.client.1.smithi149.stdout:5/250: creat db/d12/d1e/f4a x:0 0 0 2024-04-24T08:24:38.836 INFO:tasks.workunit.client.1.smithi149.stdout:4/198: truncate d2/f7 1472380 0 2024-04-24T08:24:38.838 INFO:tasks.workunit.client.1.smithi149.stdout:9/189: dwrite d1/de/f1a [0,4194304] 0 2024-04-24T08:24:38.842 INFO:tasks.workunit.client.1.smithi149.stdout:4/199: read - d2/f3d zero size 2024-04-24T08:24:38.842 INFO:tasks.workunit.client.1.smithi149.stdout:5/251: rename db/d12 to db/d12/d22/d45/d4b 22 2024-04-24T08:24:38.843 INFO:tasks.workunit.client.1.smithi149.stdout:8/148: truncate d1/fc 2214960 0 2024-04-24T08:24:38.843 INFO:tasks.workunit.client.1.smithi149.stdout:2/195: write d0/d6/db/f3e [651215,125534] 0 2024-04-24T08:24:38.846 INFO:tasks.workunit.client.1.smithi149.stdout:2/196: readlink d0/d6/l34 0 2024-04-24T08:24:38.847 INFO:tasks.workunit.client.1.smithi149.stdout:1/273: dwrite d0/d5/db/d18/f3a [0,4194304] 0 2024-04-24T08:24:38.851 INFO:tasks.workunit.client.1.smithi149.stdout:3/194: creat d0/d5/da/d19/d1b/d22/f41 x:0 0 0 2024-04-24T08:24:38.852 INFO:tasks.workunit.client.1.smithi149.stdout:0/180: write d2/f8 [1540166,119923] 0 2024-04-24T08:24:38.861 INFO:tasks.workunit.client.1.smithi149.stdout:3/195: dread - d0/d5/d1a/d1c/d34/d3a/f36 zero size 2024-04-24T08:24:38.869 INFO:tasks.workunit.client.1.smithi149.stdout:9/190: rmdir d1/da 39 2024-04-24T08:24:38.872 INFO:tasks.workunit.client.1.smithi149.stdout:7/207: dwrite d9/dc/f17 [0,4194304] 0 2024-04-24T08:24:38.873 INFO:tasks.workunit.client.1.smithi149.stdout:4/200: creat d2/d18/d1e/d3b/f42 x:0 0 0 2024-04-24T08:24:38.897 INFO:tasks.workunit.client.1.smithi149.stdout:8/149: rename d1/fc to d1/de/f2b 0 2024-04-24T08:24:38.899 INFO:tasks.workunit.client.1.smithi149.stdout:5/252: dwrite db/f16 [0,4194304] 0 2024-04-24T08:24:38.906 INFO:tasks.workunit.client.1.smithi149.stdout:2/197: dwrite d0/d6/db/d28/f38 [0,4194304] 0 2024-04-24T08:24:38.907 INFO:tasks.workunit.client.1.smithi149.stdout:0/181: rename d2/c6 to d2/d9/df/d2f/c35 0 2024-04-24T08:24:38.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/150: rmdir d1/de/d18 39 2024-04-24T08:24:38.916 INFO:tasks.workunit.client.1.smithi149.stdout:5/253: chown db/d12/d1e/f3c 73 1 2024-04-24T08:24:38.917 INFO:tasks.workunit.client.1.smithi149.stdout:3/196: dwrite d0/d5/d1a/d1c/d34/f35 [0,4194304] 0 2024-04-24T08:24:38.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/198: unlink d0/d6/f7 0 2024-04-24T08:24:38.928 INFO:tasks.workunit.client.1.smithi149.stdout:7/208: rename d9/dc/de/l3f to d9/dc/d11/d26/l47 0 2024-04-24T08:24:38.929 INFO:tasks.workunit.client.1.smithi149.stdout:1/274: write d0/fd [1575457,14463] 0 2024-04-24T08:24:38.929 INFO:tasks.workunit.client.1.smithi149.stdout:0/182: mkdir d2/d9/d1a/d36 0 2024-04-24T08:24:38.929 INFO:tasks.workunit.client.1.smithi149.stdout:4/201: write d2/d10/f17 [1182517,50766] 0 2024-04-24T08:24:38.932 INFO:tasks.workunit.client.1.smithi149.stdout:6/164: dwrite d0/fe [4194304,4194304] 0 2024-04-24T08:24:38.933 INFO:tasks.workunit.client.1.smithi149.stdout:8/151: creat d1/d1f/f2c x:0 0 0 2024-04-24T08:24:38.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/209: readlink d9/dc/de/l1c 0 2024-04-24T08:24:38.940 INFO:tasks.workunit.client.1.smithi149.stdout:4/202: write d2/f19 [1311790,30000] 0 2024-04-24T08:24:38.941 INFO:tasks.workunit.client.1.smithi149.stdout:0/183: write d2/d9/f13 [361581,20299] 0 2024-04-24T08:24:38.943 INFO:tasks.workunit.client.1.smithi149.stdout:5/254: creat db/d35/f4c x:0 0 0 2024-04-24T08:24:38.957 INFO:tasks.workunit.client.1.smithi149.stdout:9/191: dread d1/da/f29 [0,4194304] 0 2024-04-24T08:24:38.967 INFO:tasks.workunit.client.1.smithi149.stdout:3/197: rename d0/c17 to d0/d5/d1a/d1c/c42 0 2024-04-24T08:24:38.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/199: dwrite d0/f8 [0,4194304] 0 2024-04-24T08:24:38.969 INFO:tasks.workunit.client.1.smithi149.stdout:0/184: symlink d2/d9/d1a/d36/l37 0 2024-04-24T08:24:38.975 INFO:tasks.workunit.client.1.smithi149.stdout:0/185: fdatasync d2/fa 0 2024-04-24T08:24:38.976 INFO:tasks.workunit.client.1.smithi149.stdout:6/165: truncate d0/f16 461288 0 2024-04-24T08:24:38.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/203: creat d2/d18/d1e/d3c/f43 x:0 0 0 2024-04-24T08:24:38.978 INFO:tasks.workunit.client.1.smithi149.stdout:3/198: write d0/d5/da/fb [2554235,28929] 0 2024-04-24T08:24:38.982 INFO:tasks.workunit.client.1.smithi149.stdout:7/210: dread d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:38.986 INFO:tasks.workunit.client.1.smithi149.stdout:1/275: dwrite d0/d5/db/d18/f45 [0,4194304] 0 2024-04-24T08:24:38.993 INFO:tasks.workunit.client.1.smithi149.stdout:2/200: creat d0/d6/f41 x:0 0 0 2024-04-24T08:24:38.995 INFO:tasks.workunit.client.1.smithi149.stdout:6/166: mkdir d0/d1a/d28/d31/d33 0 2024-04-24T08:24:39.000 INFO:tasks.workunit.client.1.smithi149.stdout:5/255: dwrite db/d12/d22/f31 [0,4194304] 0 2024-04-24T08:24:39.001 INFO:tasks.workunit.client.1.smithi149.stdout:1/276: stat d0/d5/db/d18/d34 0 2024-04-24T08:24:39.001 INFO:tasks.workunit.client.1.smithi149.stdout:9/192: dread d1/da/f29 [0,4194304] 0 2024-04-24T08:24:39.007 INFO:tasks.workunit.client.1.smithi149.stdout:6/167: write d0/f1f [55155,1646] 0 2024-04-24T08:24:39.007 INFO:tasks.workunit.client.1.smithi149.stdout:8/152: rmdir d1 39 2024-04-24T08:24:39.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/201: truncate d0/d6/db/f17 4628542 0 2024-04-24T08:24:39.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/186: dwrite d2/fb [4194304,4194304] 0 2024-04-24T08:24:39.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/199: rename d0/d5/da/d19/d1b/d25 to d0/d5/da/d43 0 2024-04-24T08:24:39.021 INFO:tasks.workunit.client.1.smithi149.stdout:5/256: read db/d12/d22/f33 [1164979,81644] 0 2024-04-24T08:24:39.032 INFO:tasks.workunit.client.1.smithi149.stdout:7/211: truncate d9/dc/d11/f42 5638416 0 2024-04-24T08:24:39.037 INFO:tasks.workunit.client.1.smithi149.stdout:4/204: dwrite d2/d18/d1e/f21 [0,4194304] 0 2024-04-24T08:24:39.041 INFO:tasks.workunit.client.1.smithi149.stdout:0/187: creat d2/d9/d1a/d36/f38 x:0 0 0 2024-04-24T08:24:39.046 INFO:tasks.workunit.client.1.smithi149.stdout:9/193: dread d1/da/f29 [0,4194304] 0 2024-04-24T08:24:39.051 INFO:tasks.workunit.client.1.smithi149.stdout:8/153: unlink d1/fd 0 2024-04-24T08:24:39.054 INFO:tasks.workunit.client.1.smithi149.stdout:6/168: truncate d0/f17 157301 0 2024-04-24T08:24:39.056 INFO:tasks.workunit.client.1.smithi149.stdout:6/169: read - d0/d1a/f2e zero size 2024-04-24T08:24:39.061 INFO:tasks.workunit.client.1.smithi149.stdout:4/205: rename d2/d18/d1e/f21 to d2/f44 0 2024-04-24T08:24:39.063 INFO:tasks.workunit.client.1.smithi149.stdout:2/202: dwrite d0/f2 [4194304,4194304] 0 2024-04-24T08:24:39.077 INFO:tasks.workunit.client.1.smithi149.stdout:3/200: mkdir d0/d5/d1a/d1c/d39/d44 0 2024-04-24T08:24:39.078 INFO:tasks.workunit.client.1.smithi149.stdout:1/277: getdents d0/d5/db 0 2024-04-24T08:24:39.082 INFO:tasks.workunit.client.1.smithi149.stdout:8/154: symlink d1/de/d18/l2d 0 2024-04-24T08:24:39.083 INFO:tasks.workunit.client.1.smithi149.stdout:5/257: dwrite db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:39.084 INFO:tasks.workunit.client.1.smithi149.stdout:3/201: chown d0/d5/d1a/c13 34105188 1 2024-04-24T08:24:39.088 INFO:tasks.workunit.client.1.smithi149.stdout:1/278: unlink d0/f4 0 2024-04-24T08:24:39.088 INFO:tasks.workunit.client.1.smithi149.stdout:9/194: creat d1/da/f3e x:0 0 0 2024-04-24T08:24:39.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/195: chown d1/da/c25 2674 1 2024-04-24T08:24:39.092 INFO:tasks.workunit.client.1.smithi149.stdout:1/279: dread - d0/f25 zero size 2024-04-24T08:24:39.092 INFO:tasks.workunit.client.1.smithi149.stdout:3/202: rmdir d0 39 2024-04-24T08:24:39.094 INFO:tasks.workunit.client.1.smithi149.stdout:8/155: truncate d1/d1f/f27 348474 0 2024-04-24T08:24:39.098 INFO:tasks.workunit.client.1.smithi149.stdout:5/258: symlink db/d35/l4d 0 2024-04-24T08:24:39.107 INFO:tasks.workunit.client.1.smithi149.stdout:5/259: chown db/d12/d1e/f4a 101183317 1 2024-04-24T08:24:39.110 INFO:tasks.workunit.client.1.smithi149.stdout:3/203: fsync d0/d5/da/d19/d1b/f2c 0 2024-04-24T08:24:39.113 INFO:tasks.workunit.client.1.smithi149.stdout:7/212: dread d9/dc/f18 [0,4194304] 0 2024-04-24T08:24:39.115 INFO:tasks.workunit.client.1.smithi149.stdout:4/206: dwrite d2/d18/d1e/d3b/f42 [0,4194304] 0 2024-04-24T08:24:39.115 INFO:tasks.workunit.client.1.smithi149.stdout:0/188: dread d2/fa [0,4194304] 0 2024-04-24T08:24:39.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/203: dwrite d0/d6/db/d28/f3a [0,4194304] 0 2024-04-24T08:24:39.138 INFO:tasks.workunit.client.1.smithi149.stdout:4/207: symlink d2/l45 0 2024-04-24T08:24:39.139 INFO:tasks.workunit.client.1.smithi149.stdout:3/204: symlink d0/d5/da/d19/l45 0 2024-04-24T08:24:39.139 INFO:tasks.workunit.client.1.smithi149.stdout:0/189: fdatasync d2/d17/f23 0 2024-04-24T08:24:39.145 INFO:tasks.workunit.client.1.smithi149.stdout:1/280: rename d0/d5/df/d12/d19/f33 to d0/f5a 0 2024-04-24T08:24:39.147 INFO:tasks.workunit.client.1.smithi149.stdout:8/156: dwrite d1/d1f/f28 [0,4194304] 0 2024-04-24T08:24:39.149 INFO:tasks.workunit.client.1.smithi149.stdout:9/196: dwrite d1/d2/d15/f3c [0,4194304] 0 2024-04-24T08:24:39.149 INFO:tasks.workunit.client.1.smithi149.stdout:4/208: chown d2/f14 7 1 2024-04-24T08:24:39.150 INFO:tasks.workunit.client.1.smithi149.stdout:3/205: chown d0/d26/f2a 14 1 2024-04-24T08:24:39.153 INFO:tasks.workunit.client.1.smithi149.stdout:1/281: chown d0/f25 26581061 1 2024-04-24T08:24:39.163 INFO:tasks.workunit.client.1.smithi149.stdout:9/197: dread - d1/de/f2e zero size 2024-04-24T08:24:39.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/260: dwrite db/d12/d22/d45/f21 [4194304,4194304] 0 2024-04-24T08:24:39.175 INFO:tasks.workunit.client.1.smithi149.stdout:0/190: mknod d2/d9/df/d2f/c39 0 2024-04-24T08:24:39.179 INFO:tasks.workunit.client.1.smithi149.stdout:2/204: dwrite d0/d6/db/d28/f2c [0,4194304] 0 2024-04-24T08:24:39.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/213: rename c8 to d9/dc/d2e/d3c/c48 0 2024-04-24T08:24:39.194 INFO:tasks.workunit.client.1.smithi149.stdout:7/214: write d9/dc/de/f31 [34494,94728] 0 2024-04-24T08:24:39.198 INFO:tasks.workunit.client.1.smithi149.stdout:9/198: dwrite d1/d2/d15/d1c/f3a [0,4194304] 0 2024-04-24T08:24:39.200 INFO:tasks.workunit.client.1.smithi149.stdout:4/209: mkdir d2/d24/d2e/d3f/d46 0 2024-04-24T08:24:39.203 INFO:tasks.workunit.client.1.smithi149.stdout:9/199: read d1/d2/d15/d1c/f3a [3159125,89317] 0 2024-04-24T08:24:39.207 INFO:tasks.workunit.client.1.smithi149.stdout:2/205: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:39.207 INFO:tasks.workunit.client.1.smithi149.stdout:9/200: chown d1/da/f3e 603025 1 2024-04-24T08:24:39.215 INFO:tasks.workunit.client.1.smithi149.stdout:7/215: dwrite d9/dc/d2e/f39 [0,4194304] 0 2024-04-24T08:24:39.217 INFO:tasks.workunit.client.1.smithi149.stdout:1/282: mknod d0/d1b/d4b/c5b 0 2024-04-24T08:24:39.231 INFO:tasks.workunit.client.1.smithi149.stdout:5/261: rmdir db/d23 39 2024-04-24T08:24:39.233 INFO:tasks.workunit.client.1.smithi149.stdout:5/262: dread - db/d12/f3d zero size 2024-04-24T08:24:39.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/191: symlink d2/d17/l3a 0 2024-04-24T08:24:39.238 INFO:tasks.workunit.client.1.smithi149.stdout:0/192: chown d2/d17/l3a 1959520 1 2024-04-24T08:24:39.238 INFO:tasks.workunit.client.1.smithi149.stdout:1/283: dwrite d0/d5/df/d12/d19/f23 [0,4194304] 0 2024-04-24T08:24:39.248 INFO:tasks.workunit.client.1.smithi149.stdout:4/210: symlink d2/d18/d1e/d3c/l47 0 2024-04-24T08:24:39.251 INFO:tasks.workunit.client.1.smithi149.stdout:4/211: write d2/d10/f17 [2257995,86820] 0 2024-04-24T08:24:39.267 INFO:tasks.workunit.client.1.smithi149.stdout:7/216: creat d9/dc/de/d44/f49 x:0 0 0 2024-04-24T08:24:39.268 INFO:tasks.workunit.client.1.smithi149.stdout:1/284: dwrite d0/d5/f41 [4194304,4194304] 0 2024-04-24T08:24:39.273 INFO:tasks.workunit.client.1.smithi149.stdout:1/285: dread - d0/d5/db/d18/f29 zero size 2024-04-24T08:24:39.293 INFO:tasks.workunit.client.1.smithi149.stdout:7/217: dread d9/dc/de/f12 [0,4194304] 0 2024-04-24T08:24:39.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/212: mkdir d2/d18/d1e/d3c/d48 0 2024-04-24T08:24:39.297 INFO:tasks.workunit.client.1.smithi149.stdout:7/218: fdatasync d9/dc/de/f22 0 2024-04-24T08:24:39.299 INFO:tasks.workunit.client.1.smithi149.stdout:3/206: dwrite d0/d5/da/d19/f20 [8388608,4194304] 0 2024-04-24T08:24:39.299 INFO:tasks.workunit.client.1.smithi149.stdout:4/213: write d2/fc [1898258,98193] 0 2024-04-24T08:24:39.300 INFO:tasks.workunit.client.1.smithi149.stdout:7/219: chown d9/dc/d2e/c2f 9 1 2024-04-24T08:24:39.302 INFO:tasks.workunit.client.1.smithi149.stdout:7/220: readlink d9/dc/d11/d26/l36 0 2024-04-24T08:24:39.307 INFO:tasks.workunit.client.1.smithi149.stdout:5/263: unlink db/d1f/f2a 0 2024-04-24T08:24:39.311 INFO:tasks.workunit.client.1.smithi149.stdout:0/193: write d2/d9/fe [244836,63737] 0 2024-04-24T08:24:39.313 INFO:tasks.workunit.client.1.smithi149.stdout:8/157: getdents d1/de 0 2024-04-24T08:24:39.314 INFO:tasks.workunit.client.1.smithi149.stdout:0/194: write d2/d9/df/f16 [987175,19056] 0 2024-04-24T08:24:39.322 INFO:tasks.workunit.client.1.smithi149.stdout:3/207: rename d0/d5/f29 to d0/d5/d1a/f46 0 2024-04-24T08:24:39.322 INFO:tasks.workunit.client.1.smithi149.stdout:4/214: write d2/d24/d2e/d3f/f40 [2258,109554] 0 2024-04-24T08:24:39.324 INFO:tasks.workunit.client.1.smithi149.stdout:9/201: truncate d1/de/ff 1946106 0 2024-04-24T08:24:39.325 INFO:tasks.workunit.client.1.smithi149.stdout:3/208: write d0/f11 [1561292,121971] 0 2024-04-24T08:24:39.327 INFO:tasks.workunit.client.1.smithi149.stdout:9/202: read d1/da/d13/f1e [244767,33378] 0 2024-04-24T08:24:39.331 INFO:tasks.workunit.client.1.smithi149.stdout:0/195: symlink d2/d9/d1a/d36/l3b 0 2024-04-24T08:24:39.333 INFO:tasks.workunit.client.1.smithi149.stdout:4/215: symlink d2/d10/d1a/d34/l49 0 2024-04-24T08:24:39.337 INFO:tasks.workunit.client.1.smithi149.stdout:1/286: truncate d0/d5/db/d18/f3a 1721432 0 2024-04-24T08:24:39.339 INFO:tasks.workunit.client.1.smithi149.stdout:3/209: dread d0/d5/f16 [0,4194304] 0 2024-04-24T08:24:39.339 INFO:tasks.workunit.client.1.smithi149.stdout:1/287: fdatasync d0/fe 0 2024-04-24T08:24:39.342 INFO:tasks.workunit.client.1.smithi149.stdout:1/288: write d0/fd [1747558,27231] 0 2024-04-24T08:24:39.344 INFO:tasks.workunit.client.1.smithi149.stdout:0/196: unlink d2/d9/d1a/f2e 0 2024-04-24T08:24:39.347 INFO:tasks.workunit.client.1.smithi149.stdout:1/289: write d0/d5/db/d18/d34/f4e [232332,117263] 0 2024-04-24T08:24:39.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/203: mknod d1/da/c3f 0 2024-04-24T08:24:39.357 INFO:tasks.workunit.client.1.smithi149.stdout:3/210: creat d0/d5/d1a/d1c/f47 x:0 0 0 2024-04-24T08:24:39.361 INFO:tasks.workunit.client.1.smithi149.stdout:7/221: getdents d9/dc/d11/d26 0 2024-04-24T08:24:39.364 INFO:tasks.workunit.client.1.smithi149.stdout:1/290: mkdir d0/d5/df/d12/d19/d5c 0 2024-04-24T08:24:39.367 INFO:tasks.workunit.client.1.smithi149.stdout:5/264: getdents db/d12/d22 0 2024-04-24T08:24:39.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/216: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:39.370 INFO:tasks.workunit.client.1.smithi149.stdout:8/158: link d1/l12 d1/de/l2e 0 2024-04-24T08:24:39.372 INFO:tasks.workunit.client.1.smithi149.stdout:9/204: rename d1/de/f1a to d1/d2/d34/f40 0 2024-04-24T08:24:39.373 INFO:tasks.workunit.client.1.smithi149.stdout:4/217: stat d2/d10/d1a/d34/l49 0 2024-04-24T08:24:39.374 INFO:tasks.workunit.client.1.smithi149.stdout:3/211: creat d0/d5/da/d19/f48 x:0 0 0 2024-04-24T08:24:39.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/197: mkdir d2/d3c 0 2024-04-24T08:24:39.386 INFO:tasks.workunit.client.1.smithi149.stdout:1/291: dwrite d0/d5/df/d12/d19/f23 [0,4194304] 0 2024-04-24T08:24:39.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/292: read - d0/d1b/d4b/f58 zero size 2024-04-24T08:24:39.390 INFO:tasks.workunit.client.1.smithi149.stdout:5/265: rmdir db/d12/d2f 39 2024-04-24T08:24:39.397 INFO:tasks.workunit.client.1.smithi149.stdout:4/218: dwrite d2/d24/f39 [0,4194304] 0 2024-04-24T08:24:39.401 INFO:tasks.workunit.client.1.smithi149.stdout:0/198: creat d2/d9/df/d2f/f3d x:0 0 0 2024-04-24T08:24:39.405 INFO:tasks.workunit.client.1.smithi149.stdout:3/212: fsync d0/f4 0 2024-04-24T08:24:39.409 INFO:tasks.workunit.client.1.smithi149.stdout:4/219: rmdir d2/d24 39 2024-04-24T08:24:39.409 INFO:tasks.workunit.client.1.smithi149.stdout:7/222: dwrite d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:39.414 INFO:tasks.workunit.client.1.smithi149.stdout:8/159: fsync d1/de/f1a 0 2024-04-24T08:24:39.415 INFO:tasks.workunit.client.1.smithi149.stdout:5/266: chown db/d12/d22/d45/f42 50303 1 2024-04-24T08:24:39.417 INFO:tasks.workunit.client.1.smithi149.stdout:9/205: getdents d1/da 0 2024-04-24T08:24:39.421 INFO:tasks.workunit.client.1.smithi149.stdout:3/213: unlink d0/d26/f37 0 2024-04-24T08:24:39.424 INFO:tasks.workunit.client.1.smithi149.stdout:3/214: fsync d0/d5/da/f24 0 2024-04-24T08:24:39.429 INFO:tasks.workunit.client.1.smithi149.stdout:1/293: dwrite d0/d5/f8 [0,4194304] 0 2024-04-24T08:24:39.433 INFO:tasks.workunit.client.1.smithi149.stdout:1/294: chown d0/d5/db/d18/d4d/d50 960 1 2024-04-24T08:24:39.436 INFO:tasks.workunit.client.1.smithi149.stdout:4/220: creat d2/d24/d2e/f4a x:0 0 0 2024-04-24T08:24:39.440 INFO:tasks.workunit.client.1.smithi149.stdout:0/199: mknod d2/d9/df/c3e 0 2024-04-24T08:24:39.441 INFO:tasks.workunit.client.1.smithi149.stdout:3/215: creat d0/d5/d1a/d1c/d39/f49 x:0 0 0 2024-04-24T08:24:39.444 INFO:tasks.workunit.client.1.smithi149.stdout:9/206: getdents d1/d2/d33 0 2024-04-24T08:24:39.449 INFO:tasks.workunit.client.1.smithi149.stdout:1/295: mkdir d0/d1b/d5d 0 2024-04-24T08:24:39.450 INFO:tasks.workunit.client.1.smithi149.stdout:4/221: unlink d2/d18/d1e/d3c/l47 0 2024-04-24T08:24:39.451 INFO:tasks.workunit.client.1.smithi149.stdout:9/207: creat d1/d2/f41 x:0 0 0 2024-04-24T08:24:39.455 INFO:tasks.workunit.client.1.smithi149.stdout:8/160: dwrite d1/f9 [0,4194304] 0 2024-04-24T08:24:39.462 INFO:tasks.workunit.client.1.smithi149.stdout:4/222: creat d2/d24/d2e/d3f/f4b x:0 0 0 2024-04-24T08:24:39.462 INFO:tasks.workunit.client.1.smithi149.stdout:9/208: truncate d1/d2/f2b 1341212 0 2024-04-24T08:24:39.468 INFO:tasks.workunit.client.1.smithi149.stdout:3/216: link d0/d5/da/d19/l45 d0/d5/da/d19/d1b/d22/d38/l4a 0 2024-04-24T08:24:39.471 INFO:tasks.workunit.client.1.smithi149.stdout:1/296: creat d0/d5/df/d12/d19/d5c/f5e x:0 0 0 2024-04-24T08:24:39.473 INFO:tasks.workunit.client.1.smithi149.stdout:9/209: mkdir d1/d2/d15/d1c/d42 0 2024-04-24T08:24:39.477 INFO:tasks.workunit.client.1.smithi149.stdout:0/200: dwrite d2/d9/df/d2f/f34 [0,4194304] 0 2024-04-24T08:24:39.477 INFO:tasks.workunit.client.1.smithi149.stdout:1/297: chown d0/d5/df/l3b 86141 1 2024-04-24T08:24:39.478 INFO:tasks.workunit.client.1.smithi149.stdout:3/217: symlink d0/d5/da/d43/l4b 0 2024-04-24T08:24:39.480 INFO:tasks.workunit.client.1.smithi149.stdout:1/298: chown d0/d5/f8 26 1 2024-04-24T08:24:39.483 INFO:tasks.workunit.client.1.smithi149.stdout:0/201: write d2/fb [8488906,53715] 0 2024-04-24T08:24:39.484 INFO:tasks.workunit.client.1.smithi149.stdout:1/299: write d0/d1b/d2a/f4a [342744,22826] 0 2024-04-24T08:24:39.485 INFO:tasks.workunit.client.1.smithi149.stdout:8/161: creat d1/de/f2f x:0 0 0 2024-04-24T08:24:39.486 INFO:tasks.workunit.client.1.smithi149.stdout:5/267: truncate db/f16 1435997 0 2024-04-24T08:24:39.490 INFO:tasks.workunit.client.1.smithi149.stdout:0/202: fsync d2/d9/df/d2f/f3d 0 2024-04-24T08:24:39.491 INFO:tasks.workunit.client.1.smithi149.stdout:7/223: dread d9/dc/f35 [0,4194304] 0 2024-04-24T08:24:39.491 INFO:tasks.workunit.client.1.smithi149.stdout:3/218: fsync d0/d5/da/d43/f30 0 2024-04-24T08:24:39.492 INFO:tasks.workunit.client.1.smithi149.stdout:8/162: write d1/d1f/f2c [978463,30267] 0 2024-04-24T08:24:39.493 INFO:tasks.workunit.client.1.smithi149.stdout:5/268: fdatasync db/d35/f4c 0 2024-04-24T08:24:39.497 INFO:tasks.workunit.client.1.smithi149.stdout:0/203: chown d2/d9/d1a/c29 2960 1 2024-04-24T08:24:39.500 INFO:tasks.workunit.client.1.smithi149.stdout:4/223: write d2/d18/f20 [1521496,78583] 0 2024-04-24T08:24:39.501 INFO:tasks.workunit.client.1.smithi149.stdout:8/163: write d1/de/f2f [10672,122795] 0 2024-04-24T08:24:39.504 INFO:tasks.workunit.client.1.smithi149.stdout:3/219: mkdir d0/d5/d1a/d1c/d4c 0 2024-04-24T08:24:39.505 INFO:tasks.workunit.client.1.smithi149.stdout:7/224: creat d9/dc/d11/d26/f4a x:0 0 0 2024-04-24T08:24:39.508 INFO:tasks.workunit.client.1.smithi149.stdout:3/220: mknod d0/d5/d1a/d1c/c4d 0 2024-04-24T08:24:39.508 INFO:tasks.workunit.client.1.smithi149.stdout:7/225: readlink d9/dc/l25 0 2024-04-24T08:24:39.513 INFO:tasks.workunit.client.1.smithi149.stdout:9/210: dwrite d1/d2/f41 [0,4194304] 0 2024-04-24T08:24:39.513 INFO:tasks.workunit.client.1.smithi149.stdout:3/221: readlink d0/d5/da/d19/l27 0 2024-04-24T08:24:39.516 INFO:tasks.workunit.client.1.smithi149.stdout:7/226: fsync d9/dc/d11/f37 0 2024-04-24T08:24:39.517 INFO:tasks.workunit.client.1.smithi149.stdout:7/227: fdatasync d9/dc/f29 0 2024-04-24T08:24:39.518 INFO:tasks.workunit.client.1.smithi149.stdout:1/300: rename d0/d5/db/d18/f3a to d0/f5f 0 2024-04-24T08:24:39.520 INFO:tasks.workunit.client.1.smithi149.stdout:0/204: dwrite d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:39.520 INFO:tasks.workunit.client.1.smithi149.stdout:8/164: creat d1/d1f/d22/f30 x:0 0 0 2024-04-24T08:24:39.521 INFO:tasks.workunit.client.1.smithi149.stdout:1/301: dread - d0/f44 zero size 2024-04-24T08:24:39.525 INFO:tasks.workunit.client.1.smithi149.stdout:0/205: chown d2/d9/d1a/d36/l37 18688529 1 2024-04-24T08:24:39.525 INFO:tasks.workunit.client.1.smithi149.stdout:9/211: fsync d1/de/f18 0 2024-04-24T08:24:39.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:39 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:24:39.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:39 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:24:39.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:39 smithi149 ceph-mon[31282]: pgmap v12: 65 pgs: 65 active+clean; 500 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 MiB/s rd, 45 MiB/s wr, 244 op/s 2024-04-24T08:24:39.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:39 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:39.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:39 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:39.531 INFO:tasks.workunit.client.1.smithi149.stdout:0/206: write d2/d17/f24 [1297777,56118] 0 2024-04-24T08:24:39.531 INFO:tasks.workunit.client.1.smithi149.stdout:8/165: mkdir d1/d1f/d31 0 2024-04-24T08:24:39.532 INFO:tasks.workunit.client.1.smithi149.stdout:7/228: mknod d9/dc/d2e/d3c/c4b 0 2024-04-24T08:24:39.542 INFO:tasks.workunit.client.1.smithi149.stdout:0/207: creat d2/d9/d1a/f3f x:0 0 0 2024-04-24T08:24:39.542 INFO:tasks.workunit.client.1.smithi149.stdout:8/166: mkdir d1/de/d18/d32 0 2024-04-24T08:24:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:39 smithi090 ceph-mon[25016]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:24:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:39 smithi090 ceph-mon[25016]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:24:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:39 smithi090 ceph-mon[25016]: pgmap v12: 65 pgs: 65 active+clean; 500 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 MiB/s rd, 45 MiB/s wr, 244 op/s 2024-04-24T08:24:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:39 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:39 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:39.544 INFO:tasks.workunit.client.1.smithi149.stdout:1/302: getdents d0/d5/df/d12/d19/d5c 0 2024-04-24T08:24:39.545 INFO:tasks.workunit.client.1.smithi149.stdout:0/208: fsync d2/d9/df/f1d 0 2024-04-24T08:24:39.549 INFO:tasks.workunit.client.1.smithi149.stdout:1/303: fdatasync d0/da/fc 0 2024-04-24T08:24:39.553 INFO:tasks.workunit.client.1.smithi149.stdout:1/304: write d0/d5/df/d12/d19/f57 [350668,33784] 0 2024-04-24T08:24:39.555 INFO:tasks.workunit.client.1.smithi149.stdout:1/305: readlink d0/d5/df/d12/l42 0 2024-04-24T08:24:39.556 INFO:tasks.workunit.client.1.smithi149.stdout:4/224: truncate d2/fc 1855538 0 2024-04-24T08:24:39.561 INFO:tasks.workunit.client.1.smithi149.stdout:7/229: dread d9/dc/de/f33 [0,4194304] 0 2024-04-24T08:24:39.561 INFO:tasks.workunit.client.1.smithi149.stdout:1/306: fsync d0/f5a 0 2024-04-24T08:24:39.565 INFO:tasks.workunit.client.1.smithi149.stdout:0/209: getdents d2 0 2024-04-24T08:24:39.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/307: dread - d0/d5/db/d18/d34/f56 zero size 2024-04-24T08:24:39.568 INFO:tasks.workunit.client.1.smithi149.stdout:7/230: write d9/dc/d2e/f41 [944971,19175] 0 2024-04-24T08:24:39.571 INFO:tasks.workunit.client.1.smithi149.stdout:4/225: link d2/ff d2/d10/d1a/d34/f4c 0 2024-04-24T08:24:39.573 INFO:tasks.workunit.client.1.smithi149.stdout:3/222: dwrite d0/fe [0,4194304] 0 2024-04-24T08:24:39.582 INFO:tasks.workunit.client.1.smithi149.stdout:0/210: mkdir d2/d9/df/d40 0 2024-04-24T08:24:39.584 INFO:tasks.workunit.client.1.smithi149.stdout:4/226: dread d2/d10/d1a/f3e [0,4194304] 0 2024-04-24T08:24:39.584 INFO:tasks.workunit.client.1.smithi149.stdout:8/167: dwrite d1/de/f2b [0,4194304] 0 2024-04-24T08:24:39.588 INFO:tasks.workunit.client.1.smithi149.stdout:9/212: dread d1/de/f22 [0,4194304] 0 2024-04-24T08:24:39.590 INFO:tasks.workunit.client.1.smithi149.stdout:7/231: write d9/dc/d11/f42 [165665,90387] 0 2024-04-24T08:24:39.590 INFO:tasks.workunit.client.1.smithi149.stdout:3/223: rename d0/d5 to d0/d5/d1a/d1c/d4c/d4e 22 2024-04-24T08:24:39.591 INFO:tasks.workunit.client.1.smithi149.stdout:0/211: mkdir d2/d17/d41 0 2024-04-24T08:24:39.594 INFO:tasks.workunit.client.1.smithi149.stdout:8/168: chown d1/de/d18/l2d 30 1 2024-04-24T08:24:39.595 INFO:tasks.workunit.client.1.smithi149.stdout:0/212: chown d2/d9/fe 331086600 1 2024-04-24T08:24:39.597 INFO:tasks.workunit.client.1.smithi149.stdout:4/227: creat d2/d24/d2e/d3f/f4d x:0 0 0 2024-04-24T08:24:39.600 INFO:tasks.workunit.client.1.smithi149.stdout:9/213: symlink d1/da/d13/l43 0 2024-04-24T08:24:39.600 INFO:tasks.workunit.client.1.smithi149.stdout:3/224: symlink d0/d5/d1a/d1c/l4f 0 2024-04-24T08:24:39.602 INFO:tasks.workunit.client.1.smithi149.stdout:1/308: dwrite d0/d5/df/d12/d19/f23 [0,4194304] 0 2024-04-24T08:24:39.602 INFO:tasks.workunit.client.1.smithi149.stdout:7/232: rmdir d9/dc/d2e 39 2024-04-24T08:24:39.604 INFO:tasks.workunit.client.1.smithi149.stdout:4/228: truncate d2/d18/d1e/f2c 174423 0 2024-04-24T08:24:39.610 INFO:tasks.workunit.client.1.smithi149.stdout:0/213: symlink d2/d9/d1a/l42 0 2024-04-24T08:24:39.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/169: rename d1/de/ff to d1/d1f/d22/f33 0 2024-04-24T08:24:39.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/229: dwrite d2/f9 [0,4194304] 0 2024-04-24T08:24:39.631 INFO:tasks.workunit.client.1.smithi149.stdout:0/214: creat d2/d9/df/d2f/f43 x:0 0 0 2024-04-24T08:24:39.635 INFO:tasks.workunit.client.1.smithi149.stdout:3/225: rename d0/d5/da/d19/f20 to d0/d5/da/d19/d1b/d22/d38/f50 0 2024-04-24T08:24:39.635 INFO:tasks.workunit.client.1.smithi149.stdout:8/170: creat d1/d1f/d25/f34 x:0 0 0 2024-04-24T08:24:39.638 INFO:tasks.workunit.client.1.smithi149.stdout:1/309: rename d0/d5/df/d12 to d0/d5/d4f/d60 0 2024-04-24T08:24:39.642 INFO:tasks.workunit.client.1.smithi149.stdout:1/310: dread - d0/f25 zero size 2024-04-24T08:24:39.649 INFO:tasks.workunit.client.1.smithi149.stdout:9/214: dwrite d1/da/f27 [4194304,4194304] 0 2024-04-24T08:24:39.650 INFO:tasks.workunit.client.1.smithi149.stdout:4/230: mkdir d2/d24/d4e 0 2024-04-24T08:24:39.654 INFO:tasks.workunit.client.1.smithi149.stdout:1/311: chown d0/d1b/l40 7 1 2024-04-24T08:24:39.655 INFO:tasks.workunit.client.1.smithi149.stdout:4/231: read d2/d10/d1a/f3e [88441,2182] 0 2024-04-24T08:24:39.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/215: link d2/d9/d1a/f25 d2/d9/d1a/f44 0 2024-04-24T08:24:39.665 INFO:tasks.workunit.client.1.smithi149.stdout:4/232: read d2/d18/d1e/d3b/f42 [2008763,96381] 0 2024-04-24T08:24:39.667 INFO:tasks.workunit.client.1.smithi149.stdout:0/216: dread - d2/d9/df/d2f/f30 zero size 2024-04-24T08:24:39.670 INFO:tasks.workunit.client.1.smithi149.stdout:7/233: dwrite d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:39.670 INFO:tasks.workunit.client.1.smithi149.stdout:4/233: read - d2/d10/f30 zero size 2024-04-24T08:24:39.674 INFO:tasks.workunit.client.1.smithi149.stdout:4/234: write d2/d18/f1b [1959253,95179] 0 2024-04-24T08:24:39.676 INFO:tasks.workunit.client.1.smithi149.stdout:3/226: dwrite d0/d5/d1a/d1c/d34/f3b [0,4194304] 0 2024-04-24T08:24:39.678 INFO:tasks.workunit.client.1.smithi149.stdout:4/235: write d2/d18/f1b [2230122,84374] 0 2024-04-24T08:24:39.686 INFO:tasks.workunit.client.1.smithi149.stdout:1/312: truncate d0/d5/db/f1e 2586412 0 2024-04-24T08:24:39.686 INFO:tasks.workunit.client.1.smithi149.stdout:0/217: mknod d2/d9/df/d40/c45 0 2024-04-24T08:24:39.687 INFO:tasks.workunit.client.1.smithi149.stdout:8/171: truncate d1/d1f/f2c 689362 0 2024-04-24T08:24:39.691 INFO:tasks.workunit.client.1.smithi149.stdout:3/227: unlink d0/d26/f2a 0 2024-04-24T08:24:39.692 INFO:tasks.workunit.client.1.smithi149.stdout:8/172: write d1/d1f/f28 [4393276,69542] 0 2024-04-24T08:24:39.701 INFO:tasks.workunit.client.1.smithi149.stdout:0/218: mknod d2/d17/d41/c46 0 2024-04-24T08:24:39.702 INFO:tasks.workunit.client.1.smithi149.stdout:1/313: dread d0/d5/d4f/d60/d19/f57 [0,4194304] 0 2024-04-24T08:24:39.706 INFO:tasks.workunit.client.1.smithi149.stdout:9/215: dwrite d1/de/f22 [0,4194304] 0 2024-04-24T08:24:39.706 INFO:tasks.workunit.client.1.smithi149.stdout:7/234: truncate d9/dc/de/f22 3791482 0 2024-04-24T08:24:39.708 INFO:tasks.workunit.client.1.smithi149.stdout:4/236: getdents d2/d18 0 2024-04-24T08:24:39.709 INFO:tasks.workunit.client.1.smithi149.stdout:8/173: creat d1/d1f/d31/f35 x:0 0 0 2024-04-24T08:24:39.710 INFO:tasks.workunit.client.1.smithi149.stdout:1/314: read d0/d5/d4f/d60/d19/f57 [81622,97342] 0 2024-04-24T08:24:39.712 INFO:tasks.workunit.client.1.smithi149.stdout:4/237: stat d2/d10/d1a/d34/l49 0 2024-04-24T08:24:39.717 INFO:tasks.workunit.client.1.smithi149.stdout:0/219: link d2/d9/df/f1d d2/d9/d1a/d36/f47 0 2024-04-24T08:24:39.719 INFO:tasks.workunit.client.1.smithi149.stdout:4/238: chown d2/d10/d1a/d34/f37 117141 1 2024-04-24T08:24:39.722 INFO:tasks.workunit.client.1.smithi149.stdout:1/315: dread d0/da/f24 [0,4194304] 0 2024-04-24T08:24:39.732 INFO:tasks.workunit.client.1.smithi149.stdout:4/239: read d2/d24/d2e/d3f/f40 [30812,111929] 0 2024-04-24T08:24:39.737 INFO:tasks.workunit.client.1.smithi149.stdout:4/240: stat d2/d24/d2e/d3f/f40 0 2024-04-24T08:24:39.741 INFO:tasks.workunit.client.1.smithi149.stdout:3/228: dwrite d0/d5/da/d19/f48 [0,4194304] 0 2024-04-24T08:24:39.745 INFO:tasks.workunit.client.1.smithi149.stdout:8/174: dwrite d1/d1f/d22/f30 [0,4194304] 0 2024-04-24T08:24:39.747 INFO:tasks.workunit.client.1.smithi149.stdout:8/175: chown d1/d1f/d25 22823 1 2024-04-24T08:24:39.749 INFO:tasks.workunit.client.1.smithi149.stdout:1/316: mkdir d0/d39/d53/d61 0 2024-04-24T08:24:39.749 INFO:tasks.workunit.client.1.smithi149.stdout:9/216: dread d1/de/f22 [0,4194304] 0 2024-04-24T08:24:39.751 INFO:tasks.workunit.client.1.smithi149.stdout:9/217: fsync d1/da/f27 0 2024-04-24T08:24:39.757 INFO:tasks.workunit.client.1.smithi149.stdout:3/229: rmdir d0/d5/da/d43 39 2024-04-24T08:24:39.762 INFO:tasks.workunit.client.1.smithi149.stdout:3/230: write d0/d5/da/f28 [496511,57179] 0 2024-04-24T08:24:39.762 INFO:tasks.workunit.client.1.smithi149.stdout:8/176: write d1/de/f16 [1396742,17355] 0 2024-04-24T08:24:39.770 INFO:tasks.workunit.client.1.smithi149.stdout:8/177: chown d1/d1f/d22/f30 48 1 2024-04-24T08:24:39.775 INFO:tasks.workunit.client.1.smithi149.stdout:8/178: write d1/d1f/f27 [770932,5868] 0 2024-04-24T08:24:39.785 INFO:tasks.workunit.client.1.smithi149.stdout:1/317: dwrite d0/d5/d4f/d60/d19/f2b [0,4194304] 0 2024-04-24T08:24:39.792 INFO:tasks.workunit.client.1.smithi149.stdout:9/218: dread d1/d2/f41 [0,4194304] 0 2024-04-24T08:24:39.794 INFO:tasks.workunit.client.1.smithi149.stdout:0/220: dwrite d2/d17/f23 [0,4194304] 0 2024-04-24T08:24:39.797 INFO:tasks.workunit.client.1.smithi149.stdout:9/219: chown d1/da/d13/l43 2722 1 2024-04-24T08:24:39.800 INFO:tasks.workunit.client.1.smithi149.stdout:0/221: write d2/f8 [198678,64267] 0 2024-04-24T08:24:39.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/222: chown d2/d9/df/d40 2 1 2024-04-24T08:24:39.813 INFO:tasks.workunit.client.1.smithi149.stdout:3/231: dwrite d0/d5/d1a/d1c/d34/d3a/f33 [0,4194304] 0 2024-04-24T08:24:39.815 INFO:tasks.workunit.client.1.smithi149.stdout:4/241: dwrite d2/ff [0,4194304] 0 2024-04-24T08:24:39.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/220: mknod d1/d2/c44 0 2024-04-24T08:24:39.830 INFO:tasks.workunit.client.1.smithi149.stdout:9/221: stat d1/d2/d34/f40 0 2024-04-24T08:24:39.833 INFO:tasks.workunit.client.1.smithi149.stdout:3/232: symlink d0/d5/l51 0 2024-04-24T08:24:39.836 INFO:tasks.workunit.client.1.smithi149.stdout:1/318: truncate d0/da/f3f 3757579 0 2024-04-24T08:24:39.837 INFO:tasks.workunit.client.1.smithi149.stdout:0/223: mkdir d2/d9/df/d48 0 2024-04-24T08:24:39.839 INFO:tasks.workunit.client.1.smithi149.stdout:8/179: dwrite d1/de/d18/f1d [0,4194304] 0 2024-04-24T08:24:39.846 INFO:tasks.workunit.client.1.smithi149.stdout:9/222: dread d1/d2/f2b [0,4194304] 0 2024-04-24T08:24:39.862 INFO:tasks.workunit.client.1.smithi149.stdout:2/206: sync 2024-04-24T08:24:39.862 INFO:tasks.workunit.client.1.smithi149.stdout:6/170: sync 2024-04-24T08:24:39.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/224: rmdir d2/d9/d1a/d36 39 2024-04-24T08:24:39.872 INFO:tasks.workunit.client.1.smithi149.stdout:3/233: unlink d0/l1f 0 2024-04-24T08:24:39.872 INFO:tasks.workunit.client.1.smithi149.stdout:4/242: rmdir d2/d24/d4e 0 2024-04-24T08:24:39.873 INFO:tasks.workunit.client.1.smithi149.stdout:6/171: write d0/f1f [1565447,105729] 0 2024-04-24T08:24:39.877 INFO:tasks.workunit.client.1.smithi149.stdout:3/234: chown d0/d5/f16 30959 1 2024-04-24T08:24:39.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/172: chown d0/f3 139177 1 2024-04-24T08:24:39.884 INFO:tasks.workunit.client.1.smithi149.stdout:2/207: unlink d0/d6/db/l25 0 2024-04-24T08:24:39.886 INFO:tasks.workunit.client.1.smithi149.stdout:4/243: symlink d2/d18/d1e/d3c/l4f 0 2024-04-24T08:24:39.894 INFO:tasks.workunit.client.1.smithi149.stdout:1/319: dwrite d0/d5/d4f/d60/d19/f57 [0,4194304] 0 2024-04-24T08:24:39.896 INFO:tasks.workunit.client.1.smithi149.stdout:3/235: symlink d0/d5/d1a/d1c/l52 0 2024-04-24T08:24:39.897 INFO:tasks.workunit.client.1.smithi149.stdout:0/225: dread d2/d17/f24 [0,4194304] 0 2024-04-24T08:24:39.897 INFO:tasks.workunit.client.1.smithi149.stdout:9/223: dwrite d1/d2/d15/f3c [0,4194304] 0 2024-04-24T08:24:39.900 INFO:tasks.workunit.client.1.smithi149.stdout:6/173: chown d0/lf 1202299614 1 2024-04-24T08:24:39.901 INFO:tasks.workunit.client.1.smithi149.stdout:6/174: readlink d0/lf 0 2024-04-24T08:24:39.903 INFO:tasks.workunit.client.1.smithi149.stdout:3/236: fdatasync d0/d5/d1a/d1c/d34/d3a/f33 0 2024-04-24T08:24:39.905 INFO:tasks.workunit.client.1.smithi149.stdout:6/175: dread - d0/d1a/f32 zero size 2024-04-24T08:24:39.907 INFO:tasks.workunit.client.1.smithi149.stdout:4/244: rename d2/d18/d1e/l1f to d2/d18/l50 0 2024-04-24T08:24:39.913 INFO:tasks.workunit.client.1.smithi149.stdout:1/320: creat d0/d5/db/d18/d4d/f62 x:0 0 0 2024-04-24T08:24:39.914 INFO:tasks.workunit.client.1.smithi149.stdout:2/208: dwrite d0/f35 [0,4194304] 0 2024-04-24T08:24:39.914 INFO:tasks.workunit.client.1.smithi149.stdout:9/224: creat d1/da/d13/f45 x:0 0 0 2024-04-24T08:24:39.930 INFO:tasks.workunit.client.1.smithi149.stdout:3/237: rename d0/d5/d1a/d1c to d0/d5/d53 0 2024-04-24T08:24:39.936 INFO:tasks.workunit.client.1.smithi149.stdout:1/321: mkdir d0/d5/db/d63 0 2024-04-24T08:24:39.936 INFO:tasks.workunit.client.1.smithi149.stdout:9/225: rmdir d1/da/dd 39 2024-04-24T08:24:39.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/180: link d1/l12 d1/l36 0 2024-04-24T08:24:39.938 INFO:tasks.workunit.client.1.smithi149.stdout:4/245: link d2/d10/f23 d2/d24/d2e/f51 0 2024-04-24T08:24:39.939 INFO:tasks.workunit.client.1.smithi149.stdout:9/226: chown d1/d2/d15/d1c 63981605 1 2024-04-24T08:24:39.939 INFO:tasks.workunit.client.1.smithi149.stdout:1/322: stat d0/d5/l4c 0 2024-04-24T08:24:39.940 INFO:tasks.workunit.client.1.smithi149.stdout:1/323: read - d0/f25 zero size 2024-04-24T08:24:39.942 INFO:tasks.workunit.client.1.smithi149.stdout:6/176: getdents d0/d1a/d28 0 2024-04-24T08:24:39.944 INFO:tasks.workunit.client.1.smithi149.stdout:4/246: write d2/d18/d1e/f2c [635818,8765] 0 2024-04-24T08:24:39.953 INFO:tasks.workunit.client.1.smithi149.stdout:2/209: creat d0/d6/db/f42 x:0 0 0 2024-04-24T08:24:39.953 INFO:tasks.workunit.client.1.smithi149.stdout:4/247: write d2/d24/d2e/f4a [562215,110306] 0 2024-04-24T08:24:39.954 INFO:tasks.workunit.client.1.smithi149.stdout:8/181: symlink d1/de/l37 0 2024-04-24T08:24:39.955 INFO:tasks.workunit.client.1.smithi149.stdout:0/226: truncate d2/f8 1163963 0 2024-04-24T08:24:39.958 INFO:tasks.workunit.client.1.smithi149.stdout:4/248: creat d2/d18/d1e/d3c/f52 x:0 0 0 2024-04-24T08:24:39.958 INFO:tasks.workunit.client.1.smithi149.stdout:9/227: mknod d1/d2/d15/d1c/d42/c46 0 2024-04-24T08:24:39.961 INFO:tasks.workunit.client.1.smithi149.stdout:8/182: write d1/f9 [699319,86691] 0 2024-04-24T08:24:39.962 INFO:tasks.workunit.client.1.smithi149.stdout:3/238: dwrite d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:39.967 INFO:tasks.workunit.client.1.smithi149.stdout:4/249: symlink d2/d18/d1e/d3b/l53 0 2024-04-24T08:24:39.967 INFO:tasks.workunit.client.1.smithi149.stdout:8/183: fdatasync d1/d1f/f28 0 2024-04-24T08:24:39.978 INFO:tasks.workunit.client.1.smithi149.stdout:4/250: creat d2/d18/f54 x:0 0 0 2024-04-24T08:24:39.980 INFO:tasks.workunit.client.1.smithi149.stdout:1/324: dwrite d0/d5/d4f/d60/d19/f23 [0,4194304] 0 2024-04-24T08:24:39.981 INFO:tasks.workunit.client.1.smithi149.stdout:6/177: dwrite d0/d1a/f25 [0,4194304] 0 2024-04-24T08:24:39.984 INFO:tasks.workunit.client.1.smithi149.stdout:1/325: chown d0/d5/db/d18/d4d 0 1 2024-04-24T08:24:39.990 INFO:tasks.workunit.client.1.smithi149.stdout:0/227: getdents d2/d9/df 0 2024-04-24T08:24:39.997 INFO:tasks.workunit.client.1.smithi149.stdout:3/239: dread - d0/d5/d53/d39/f49 zero size 2024-04-24T08:24:40.001 INFO:tasks.workunit.client.1.smithi149.stdout:9/228: dwrite d1/d2/d15/f2f [0,4194304] 0 2024-04-24T08:24:40.001 INFO:tasks.workunit.client.1.smithi149.stdout:3/240: chown d0/d5/d53 45 1 2024-04-24T08:24:40.003 INFO:tasks.workunit.client.1.smithi149.stdout:6/178: write d0/fa [1316546,5871] 0 2024-04-24T08:24:40.008 INFO:tasks.workunit.client.1.smithi149.stdout:6/179: fdatasync d0/f3 0 2024-04-24T08:24:40.010 INFO:tasks.workunit.client.1.smithi149.stdout:1/326: creat d0/d1b/f64 x:0 0 0 2024-04-24T08:24:40.012 INFO:tasks.workunit.client.1.smithi149.stdout:0/228: truncate d2/d9/d1a/d36/f38 633612 0 2024-04-24T08:24:40.016 INFO:tasks.workunit.client.1.smithi149.stdout:3/241: symlink d0/d5/d53/d34/d3a/l54 0 2024-04-24T08:24:40.021 INFO:tasks.workunit.client.1.smithi149.stdout:8/184: dread d1/de/f1a [0,4194304] 0 2024-04-24T08:24:40.022 INFO:tasks.workunit.client.1.smithi149.stdout:1/327: mkdir d0/d5/d4f/d65 0 2024-04-24T08:24:40.025 INFO:tasks.workunit.client.1.smithi149.stdout:0/229: chown d2/d17/c33 127882 1 2024-04-24T08:24:40.028 INFO:tasks.workunit.client.1.smithi149.stdout:1/328: write d0/f5a [3077409,64585] 0 2024-04-24T08:24:40.030 INFO:tasks.workunit.client.1.smithi149.stdout:6/180: symlink d0/d1a/l34 0 2024-04-24T08:24:40.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/242: dread d0/d5/da/d19/f48 [0,4194304] 0 2024-04-24T08:24:40.035 INFO:tasks.workunit.client.1.smithi149.stdout:4/251: dwrite d2/d10/f23 [0,4194304] 0 2024-04-24T08:24:40.050 INFO:tasks.workunit.client.1.smithi149.stdout:4/252: creat d2/d18/f55 x:0 0 0 2024-04-24T08:24:40.050 INFO:tasks.workunit.client.1.smithi149.stdout:6/181: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:40.051 INFO:tasks.workunit.client.1.smithi149.stdout:1/329: mknod d0/d1b/c66 0 2024-04-24T08:24:40.056 INFO:tasks.workunit.client.1.smithi149.stdout:8/185: creat d1/de/f38 x:0 0 0 2024-04-24T08:24:40.058 INFO:tasks.workunit.client.1.smithi149.stdout:9/229: rename d1/da/c25 to d1/c47 0 2024-04-24T08:24:40.060 INFO:tasks.workunit.client.1.smithi149.stdout:3/243: creat d0/d5/da/d43/f55 x:0 0 0 2024-04-24T08:24:40.063 INFO:tasks.workunit.client.1.smithi149.stdout:3/244: dread - d0/d5/da/d19/d1b/d22/f41 zero size 2024-04-24T08:24:40.064 INFO:tasks.workunit.client.1.smithi149.stdout:4/253: mknod d2/d18/d1e/d3c/c56 0 2024-04-24T08:24:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/245: chown d0/d5/da/d19/f48 14471912 1 2024-04-24T08:24:40.066 INFO:tasks.workunit.client.1.smithi149.stdout:4/254: dread - d2/d10/f30 zero size 2024-04-24T08:24:40.068 INFO:tasks.workunit.client.1.smithi149.stdout:0/230: truncate d2/d17/f23 3700097 0 2024-04-24T08:24:40.071 INFO:tasks.workunit.client.1.smithi149.stdout:4/255: write d2/d18/f55 [10966,10244] 0 2024-04-24T08:24:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:0/231: chown d2/d9/d1a/d36/f38 916 1 2024-04-24T08:24:40.076 INFO:tasks.workunit.client.1.smithi149.stdout:0/232: stat d2/l3 0 2024-04-24T08:24:40.077 INFO:tasks.workunit.client.1.smithi149.stdout:1/330: dwrite d0/d5/d4f/d60/d19/f2b [0,4194304] 0 2024-04-24T08:24:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:8/186: truncate d1/de/d18/f1d 3709270 0 2024-04-24T08:24:40.087 INFO:tasks.workunit.client.1.smithi149.stdout:6/182: write d0/f17 [289231,18560] 0 2024-04-24T08:24:40.094 INFO:tasks.workunit.client.1.smithi149.stdout:4/256: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:24:40.096 INFO:tasks.workunit.client.1.smithi149.stdout:9/230: link d1/da/d13/f1e d1/d2/f48 0 2024-04-24T08:24:40.097 INFO:tasks.workunit.client.1.smithi149.stdout:6/183: symlink d0/d1a/d28/d31/l35 0 2024-04-24T08:24:40.099 INFO:tasks.workunit.client.1.smithi149.stdout:3/246: rmdir d0/d5/d53/d39/d44 0 2024-04-24T08:24:40.102 INFO:tasks.workunit.client.1.smithi149.stdout:4/257: dread d2/d18/f55 [0,4194304] 0 2024-04-24T08:24:40.104 INFO:tasks.workunit.client.1.smithi149.stdout:9/231: fdatasync d1/de/f18 0 2024-04-24T08:24:40.107 INFO:tasks.workunit.client.1.smithi149.stdout:1/331: link d0/da/c47 d0/d5/db/d18/d2f/c67 0 2024-04-24T08:24:40.109 INFO:tasks.workunit.client.1.smithi149.stdout:6/184: read d0/fc [345793,53271] 0 2024-04-24T08:24:40.109 INFO:tasks.workunit.client.1.smithi149.stdout:3/247: unlink d0/d5/d53/d39/f49 0 2024-04-24T08:24:40.110 INFO:tasks.workunit.client.1.smithi149.stdout:0/233: dwrite d2/d17/f24 [0,4194304] 0 2024-04-24T08:24:40.122 INFO:tasks.workunit.client.1.smithi149.stdout:8/187: dwrite d1/de/f2f [0,4194304] 0 2024-04-24T08:24:40.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/248: dwrite d0/d5/da/fb [0,4194304] 0 2024-04-24T08:24:40.157 INFO:tasks.workunit.client.1.smithi149.stdout:4/258: dwrite d2/d24/d2e/d3f/f4b [0,4194304] 0 2024-04-24T08:24:40.162 INFO:tasks.workunit.client.1.smithi149.stdout:8/188: creat d1/f39 x:0 0 0 2024-04-24T08:24:40.165 INFO:tasks.workunit.client.1.smithi149.stdout:1/332: dwrite d0/d5/db/d18/f29 [0,4194304] 0 2024-04-24T08:24:40.174 INFO:tasks.workunit.client.1.smithi149.stdout:0/234: dwrite d2/d9/df/f1b [4194304,4194304] 0 2024-04-24T08:24:40.174 INFO:tasks.workunit.client.1.smithi149.stdout:1/333: truncate d0/d5/db/f1c 644357 0 2024-04-24T08:24:40.179 INFO:tasks.workunit.client.1.smithi149.stdout:1/334: write d0/d5/f41 [1365153,89677] 0 2024-04-24T08:24:40.184 INFO:tasks.workunit.client.1.smithi149.stdout:0/235: write d2/d9/df/f16 [3864328,100443] 0 2024-04-24T08:24:40.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/335: chown d0/d5/db/d18/d34/f4e 2024 1 2024-04-24T08:24:40.190 INFO:tasks.workunit.client.1.smithi149.stdout:9/232: dread d1/de/d12/f1d [0,4194304] 0 2024-04-24T08:24:40.190 INFO:tasks.workunit.client.1.smithi149.stdout:0/236: dread - d2/d9/df/d2f/f30 zero size 2024-04-24T08:24:40.190 INFO:tasks.workunit.client.1.smithi149.stdout:4/259: dread d2/d24/d2e/d3f/f40 [0,4194304] 0 2024-04-24T08:24:40.191 INFO:tasks.workunit.client.1.smithi149.stdout:6/185: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:40.198 INFO:tasks.workunit.client.1.smithi149.stdout:8/189: dread d1/de/f2b [0,4194304] 0 2024-04-24T08:24:40.201 INFO:tasks.workunit.client.1.smithi149.stdout:3/249: creat d0/d5/d53/d39/f56 x:0 0 0 2024-04-24T08:24:40.203 INFO:tasks.workunit.client.1.smithi149.stdout:0/237: chown d2/l7 5534 1 2024-04-24T08:24:40.205 INFO:tasks.workunit.client.1.smithi149.stdout:7/235: sync 2024-04-24T08:24:40.205 INFO:tasks.workunit.client.1.smithi149.stdout:2/210: sync 2024-04-24T08:24:40.205 INFO:tasks.workunit.client.1.smithi149.stdout:5/269: sync 2024-04-24T08:24:40.205 INFO:tasks.workunit.client.1.smithi149.stdout:3/250: chown d0/d5/da 55 1 2024-04-24T08:24:40.209 INFO:tasks.workunit.client.1.smithi149.stdout:7/236: chown d9/dc/d11/d38 896 1 2024-04-24T08:24:40.209 INFO:tasks.workunit.client.1.smithi149.stdout:3/251: stat d0/d5/da/d19/l27 0 2024-04-24T08:24:40.210 INFO:tasks.workunit.client.1.smithi149.stdout:9/233: rmdir d1 39 2024-04-24T08:24:40.212 INFO:tasks.workunit.client.1.smithi149.stdout:7/237: chown d9/c2d 0 1 2024-04-24T08:24:40.212 INFO:tasks.workunit.client.1.smithi149.stdout:3/252: fsync d0/d5/da/f24 0 2024-04-24T08:24:40.215 INFO:tasks.workunit.client.1.smithi149.stdout:7/238: chown d9/dc/de/f16 202 1 2024-04-24T08:24:40.215 INFO:tasks.workunit.client.1.smithi149.stdout:6/186: write d0/d1a/f29 [1982945,53794] 0 2024-04-24T08:24:40.216 INFO:tasks.workunit.client.1.smithi149.stdout:3/253: dread - d0/d5/d53/f3f zero size 2024-04-24T08:24:40.217 INFO:tasks.workunit.client.1.smithi149.stdout:4/260: mknod d2/d24/d2e/c57 0 2024-04-24T08:24:40.220 INFO:tasks.workunit.client.1.smithi149.stdout:4/261: write d2/d18/d1e/f2c [1054751,65377] 0 2024-04-24T08:24:40.225 INFO:tasks.workunit.client.1.smithi149.stdout:5/270: truncate db/d12/d2f/f36 696850 0 2024-04-24T08:24:40.227 INFO:tasks.workunit.client.1.smithi149.stdout:8/190: dwrite d1/de/f16 [0,4194304] 0 2024-04-24T08:24:40.234 INFO:tasks.workunit.client.1.smithi149.stdout:1/336: mknod d0/d5/db/d18/d34/d51/c68 0 2024-04-24T08:24:40.241 INFO:tasks.workunit.client.1.smithi149.stdout:0/238: link d2/d9/df/f16 d2/d3c/f49 0 2024-04-24T08:24:40.247 INFO:tasks.workunit.client.1.smithi149.stdout:5/271: symlink db/d12/d22/l4e 0 2024-04-24T08:24:40.249 INFO:tasks.workunit.client.1.smithi149.stdout:8/191: symlink d1/de/d18/l3a 0 2024-04-24T08:24:40.251 INFO:tasks.workunit.client.1.smithi149.stdout:5/272: truncate db/d12/d2f/f46 67611 0 2024-04-24T08:24:40.252 INFO:tasks.workunit.client.1.smithi149.stdout:7/239: mkdir d9/d43/d4c 0 2024-04-24T08:24:40.253 INFO:tasks.workunit.client.1.smithi149.stdout:1/337: dwrite d0/d5/db/d18/d34/f4e [0,4194304] 0 2024-04-24T08:24:40.261 INFO:tasks.workunit.client.1.smithi149.stdout:4/262: creat d2/d24/d2e/d3f/d46/f58 x:0 0 0 2024-04-24T08:24:40.264 INFO:tasks.workunit.client.1.smithi149.stdout:0/239: mkdir d2/d9/df/d40/d4a 0 2024-04-24T08:24:40.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/234: fsync d1/da/f8 0 2024-04-24T08:24:40.272 INFO:tasks.workunit.client.1.smithi149.stdout:7/240: dwrite d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:24:40.280 INFO:tasks.workunit.client.1.smithi149.stdout:1/338: rename d0/d1b/d2a/f4a to d0/d1b/d5d/f69 0 2024-04-24T08:24:40.280 INFO:tasks.workunit.client.1.smithi149.stdout:5/273: creat db/d12/d1e/f4f x:0 0 0 2024-04-24T08:24:40.282 INFO:tasks.workunit.client.1.smithi149.stdout:2/211: link d0/d6/lc d0/d6/l43 0 2024-04-24T08:24:40.283 INFO:tasks.workunit.client.1.smithi149.stdout:4/263: fdatasync d2/d18/d1e/f15 0 2024-04-24T08:24:40.302 INFO:tasks.workunit.client.1.smithi149.stdout:6/187: write d0/f26 [2280887,124654] 0 2024-04-24T08:24:40.305 INFO:tasks.workunit.client.1.smithi149.stdout:3/254: truncate d0/fe 3050256 0 2024-04-24T08:24:40.306 INFO:tasks.workunit.client.1.smithi149.stdout:2/212: mkdir d0/d6/db/d39/d3d/d44 0 2024-04-24T08:24:40.313 INFO:tasks.workunit.client.1.smithi149.stdout:5/274: dwrite db/d12/d1e/f3c [0,4194304] 0 2024-04-24T08:24:40.316 INFO:tasks.workunit.client.1.smithi149.stdout:5/275: chown db/d12/c38 1 1 2024-04-24T08:24:40.317 INFO:tasks.workunit.client.1.smithi149.stdout:4/264: rename d2/d24/d2e/d3f/d46 to d2/d18/d1e/d3c/d59 0 2024-04-24T08:24:40.319 INFO:tasks.workunit.client.1.smithi149.stdout:9/235: dwrite d1/da/f2c [0,4194304] 0 2024-04-24T08:24:40.322 INFO:tasks.workunit.client.1.smithi149.stdout:5/276: chown db/d12/d2f 223261 1 2024-04-24T08:24:40.324 INFO:tasks.workunit.client.1.smithi149.stdout:8/192: write d1/de/f16 [5108544,55693] 0 2024-04-24T08:24:40.324 INFO:tasks.workunit.client.1.smithi149.stdout:2/213: creat d0/d6/db/d27/f45 x:0 0 0 2024-04-24T08:24:40.326 INFO:tasks.workunit.client.1.smithi149.stdout:2/214: chown d0/f22 3 1 2024-04-24T08:24:40.330 INFO:tasks.workunit.client.1.smithi149.stdout:0/240: write d2/d9/d1a/f25 [1556259,69279] 0 2024-04-24T08:24:40.333 INFO:tasks.workunit.client.1.smithi149.stdout:5/277: symlink db/d12/d2f/l50 0 2024-04-24T08:24:40.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/255: dread d0/d5/da/fb [0,4194304] 0 2024-04-24T08:24:40.338 INFO:tasks.workunit.client.1.smithi149.stdout:1/339: truncate d0/d5/db/d18/f29 3966766 0 2024-04-24T08:24:40.340 INFO:tasks.workunit.client.1.smithi149.stdout:4/265: mknod d2/d24/d2e/d38/c5a 0 2024-04-24T08:24:40.341 INFO:tasks.workunit.client.1.smithi149.stdout:6/188: link d0/l22 d0/d1a/d28/l36 0 2024-04-24T08:24:40.341 INFO:tasks.workunit.client.1.smithi149.stdout:9/236: mkdir d1/da/d49 0 2024-04-24T08:24:40.343 INFO:tasks.workunit.client.1.smithi149.stdout:0/241: mknod d2/d9/df/d2f/c4b 0 2024-04-24T08:24:40.344 INFO:tasks.workunit.client.1.smithi149.stdout:3/256: dread - d0/d5/da/f14 zero size 2024-04-24T08:24:40.347 INFO:tasks.workunit.client.1.smithi149.stdout:1/340: symlink d0/d5/d4f/l6a 0 2024-04-24T08:24:40.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/237: mkdir d1/da/d4a 0 2024-04-24T08:24:40.352 INFO:tasks.workunit.client.1.smithi149.stdout:9/238: stat d1/d2/d15/f2f 0 2024-04-24T08:24:40.354 INFO:tasks.workunit.client.1.smithi149.stdout:8/193: dread d1/f9 [0,4194304] 0 2024-04-24T08:24:40.356 INFO:tasks.workunit.client.1.smithi149.stdout:6/189: creat d0/d1a/d28/d31/f37 x:0 0 0 2024-04-24T08:24:40.358 INFO:tasks.workunit.client.1.smithi149.stdout:0/242: rmdir d2/d9/df/d40 39 2024-04-24T08:24:40.360 INFO:tasks.workunit.client.1.smithi149.stdout:4/266: rename d2/d18/f54 to d2/d24/d2e/f5b 0 2024-04-24T08:24:40.361 INFO:tasks.workunit.client.1.smithi149.stdout:5/278: dwrite db/d12/f3d [0,4194304] 0 2024-04-24T08:24:40.361 INFO:tasks.workunit.client.1.smithi149.stdout:4/267: chown d2/d18 338827 1 2024-04-24T08:24:40.366 INFO:tasks.workunit.client.1.smithi149.stdout:4/268: chown d2/d18/d1e/f15 65 1 2024-04-24T08:24:40.371 INFO:tasks.workunit.client.1.smithi149.stdout:8/194: symlink d1/d1f/l3b 0 2024-04-24T08:24:40.375 INFO:tasks.workunit.client.1.smithi149.stdout:2/215: truncate d0/d6/db/d28/f3a 2004204 0 2024-04-24T08:24:40.379 INFO:tasks.workunit.client.1.smithi149.stdout:6/190: mkdir d0/d1a/d28/d31/d38 0 2024-04-24T08:24:40.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/239: fdatasync d1/da/d13/f1e 0 2024-04-24T08:24:40.390 INFO:tasks.workunit.client.1.smithi149.stdout:4/269: dwrite d2/d10/f30 [0,4194304] 0 2024-04-24T08:24:40.390 INFO:tasks.workunit.client.1.smithi149.stdout:5/279: symlink db/l51 0 2024-04-24T08:24:40.392 INFO:tasks.workunit.client.1.smithi149.stdout:5/280: fdatasync db/d12/d1e/f27 0 2024-04-24T08:24:40.396 INFO:tasks.workunit.client.1.smithi149.stdout:6/191: rename d0/d1a to d0/d1a/d39 22 2024-04-24T08:24:40.396 INFO:tasks.workunit.client.1.smithi149.stdout:1/341: dread d0/fe [0,4194304] 0 2024-04-24T08:24:40.400 INFO:tasks.workunit.client.1.smithi149.stdout:8/195: mknod d1/de/c3c 0 2024-04-24T08:24:40.403 INFO:tasks.workunit.client.1.smithi149.stdout:3/257: link d0/d5/da/d43/l4b d0/l57 0 2024-04-24T08:24:40.407 INFO:tasks.workunit.client.1.smithi149.stdout:4/270: creat d2/d24/d2e/d38/f5c x:0 0 0 2024-04-24T08:24:40.409 INFO:tasks.workunit.client.1.smithi149.stdout:1/342: fdatasync d0/da/f24 0 2024-04-24T08:24:40.412 INFO:tasks.workunit.client.1.smithi149.stdout:4/271: truncate d2/d10/f27 1091394 0 2024-04-24T08:24:40.412 INFO:tasks.workunit.client.1.smithi149.stdout:0/243: creat d2/d9/df/d48/f4c x:0 0 0 2024-04-24T08:24:40.419 INFO:tasks.workunit.client.1.smithi149.stdout:4/272: read d2/f14 [4182591,2883] 0 2024-04-24T08:24:40.419 INFO:tasks.workunit.client.1.smithi149.stdout:3/258: symlink d0/d5/da/d43/l58 0 2024-04-24T08:24:40.419 INFO:tasks.workunit.client.1.smithi149.stdout:2/216: dwrite d0/f22 [0,4194304] 0 2024-04-24T08:24:40.419 INFO:tasks.workunit.client.1.smithi149.stdout:1/343: write d0/d5/db/d18/d34/f4e [3061408,95856] 0 2024-04-24T08:24:40.420 INFO:tasks.workunit.client.1.smithi149.stdout:6/192: creat d0/d1a/d28/d31/d38/f3a x:0 0 0 2024-04-24T08:24:40.420 INFO:tasks.workunit.client.1.smithi149.stdout:5/281: mknod db/d23/c52 0 2024-04-24T08:24:40.420 INFO:tasks.workunit.client.1.smithi149.stdout:8/196: chown d1/l10 593 1 2024-04-24T08:24:40.426 INFO:tasks.workunit.client.1.smithi149.stdout:1/344: fsync d0/d5/d4f/d60/d19/f23 0 2024-04-24T08:24:40.426 INFO:tasks.workunit.client.1.smithi149.stdout:3/259: readlink d0/d5/d53/d34/d3a/l54 0 2024-04-24T08:24:40.443 INFO:tasks.workunit.client.1.smithi149.stdout:1/345: link d0/d5/f8 d0/d5/db/d63/f6b 0 2024-04-24T08:24:40.447 INFO:tasks.workunit.client.1.smithi149.stdout:5/282: dread db/d12/d1e/f3c [0,4194304] 0 2024-04-24T08:24:40.451 INFO:tasks.workunit.client.1.smithi149.stdout:0/244: dread d2/d9/df/d2f/f34 [0,4194304] 0 2024-04-24T08:24:40.451 INFO:tasks.workunit.client.1.smithi149.stdout:9/240: dread d1/da/fc [0,4194304] 0 2024-04-24T08:24:40.452 INFO:tasks.workunit.client.1.smithi149.stdout:0/245: fdatasync d2/d9/df/d2f/f43 0 2024-04-24T08:24:40.454 INFO:tasks.workunit.client.1.smithi149.stdout:6/193: getdents d0/d1a/d28/d31 0 2024-04-24T08:24:40.457 INFO:tasks.workunit.client.1.smithi149.stdout:6/194: write d0/d1a/d28/d31/f37 [703726,49317] 0 2024-04-24T08:24:40.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/241: creat d1/d2/d15/d1c/d42/f4b x:0 0 0 2024-04-24T08:24:40.461 INFO:tasks.workunit.client.1.smithi149.stdout:9/242: readlink d1/da/d13/l1f 0 2024-04-24T08:24:40.464 INFO:tasks.workunit.client.1.smithi149.stdout:0/246: link d2/d9/d1a/f25 d2/d17/d41/f4d 0 2024-04-24T08:24:40.466 INFO:tasks.workunit.client.1.smithi149.stdout:4/273: truncate d2/d18/d1e/f33 599320 0 2024-04-24T08:24:40.468 INFO:tasks.workunit.client.1.smithi149.stdout:2/217: truncate d0/f8 1703865 0 2024-04-24T08:24:40.470 INFO:tasks.workunit.client.1.smithi149.stdout:3/260: truncate d0/d5/da/d19/f48 846306 0 2024-04-24T08:24:40.470 INFO:tasks.workunit.client.1.smithi149.stdout:8/197: dwrite d1/de/f1a [0,4194304] 0 2024-04-24T08:24:40.473 INFO:tasks.workunit.client.1.smithi149.stdout:9/243: mkdir d1/d4c 0 2024-04-24T08:24:40.474 INFO:tasks.workunit.client.1.smithi149.stdout:9/244: read d1/d2/d15/f3c [1213949,62629] 0 2024-04-24T08:24:40.476 INFO:tasks.workunit.client.1.smithi149.stdout:6/195: link d0/d1a/l30 d0/d1a/d28/d31/d38/l3b 0 2024-04-24T08:24:40.481 INFO:tasks.workunit.client.1.smithi149.stdout:4/274: mknod d2/c5d 0 2024-04-24T08:24:40.482 INFO:tasks.workunit.client.1.smithi149.stdout:3/261: mkdir d0/d5/da/d59 0 2024-04-24T08:24:40.484 INFO:tasks.workunit.client.1.smithi149.stdout:0/247: mkdir d2/d9/df/d40/d4a/d4e 0 2024-04-24T08:24:40.485 INFO:tasks.workunit.client.1.smithi149.stdout:6/196: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:40.486 INFO:tasks.workunit.client.1.smithi149.stdout:4/275: write d2/f19 [750616,115050] 0 2024-04-24T08:24:40.488 INFO:tasks.workunit.client.1.smithi149.stdout:8/198: dread d1/de/f16 [0,4194304] 0 2024-04-24T08:24:40.488 INFO:tasks.workunit.client.1.smithi149.stdout:7/241: sync 2024-04-24T08:24:40.490 INFO:tasks.workunit.client.1.smithi149.stdout:1/346: dwrite d0/da/f24 [0,4194304] 0 2024-04-24T08:24:40.493 INFO:tasks.workunit.client.1.smithi149.stdout:3/262: mkdir d0/d5/da/d5a 0 2024-04-24T08:24:40.495 INFO:tasks.workunit.client.1.smithi149.stdout:4/276: creat d2/d24/d2e/d38/f5e x:0 0 0 2024-04-24T08:24:40.495 INFO:tasks.workunit.client.1.smithi149.stdout:2/218: read d0/d6/db/f17 [2737295,8197] 0 2024-04-24T08:24:40.496 INFO:tasks.workunit.client.1.smithi149.stdout:8/199: write d1/d1f/d22/f30 [1944262,89338] 0 2024-04-24T08:24:40.496 INFO:tasks.workunit.client.1.smithi149.stdout:6/197: unlink d0/c18 0 2024-04-24T08:24:40.505 INFO:tasks.workunit.client.1.smithi149.stdout:1/347: write d0/d1b/f64 [147597,70940] 0 2024-04-24T08:24:40.505 INFO:tasks.workunit.client.1.smithi149.stdout:3/263: unlink d0/d5/d1a/c13 0 2024-04-24T08:24:40.506 INFO:tasks.workunit.client.1.smithi149.stdout:6/198: write d0/fe [8811470,53101] 0 2024-04-24T08:24:40.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/277: unlink d2/f19 0 2024-04-24T08:24:40.513 INFO:tasks.workunit.client.1.smithi149.stdout:8/200: mknod d1/de/d18/c3d 0 2024-04-24T08:24:40.513 INFO:tasks.workunit.client.1.smithi149.stdout:5/283: dread db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:40.518 INFO:tasks.workunit.client.1.smithi149.stdout:0/248: dread d2/d9/df/d2f/f34 [0,4194304] 0 2024-04-24T08:24:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:40 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:24:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.528 INFO:tasks.workunit.client.1.smithi149.stdout:7/242: link d9/c10 d9/dc/d11/c4d 0 2024-04-24T08:24:40.530 INFO:tasks.workunit.client.1.smithi149.stdout:5/284: mknod db/d35/c53 0 2024-04-24T08:24:40.532 INFO:tasks.workunit.client.1.smithi149.stdout:5/285: write db/d12/d2f/f46 [1074265,58053] 0 2024-04-24T08:24:40.534 INFO:tasks.workunit.client.1.smithi149.stdout:8/201: rename d1/f39 to d1/de/d18/f3e 0 2024-04-24T08:24:40.535 INFO:tasks.workunit.client.1.smithi149.stdout:0/249: creat d2/d9/df/d48/f4f x:0 0 0 2024-04-24T08:24:40.535 INFO:tasks.workunit.client.1.smithi149.stdout:9/245: truncate d1/da/f2c 1904270 0 2024-04-24T08:24:40.538 INFO:tasks.workunit.client.1.smithi149.stdout:6/199: mknod d0/c3c 0 2024-04-24T08:24:40.540 INFO:tasks.workunit.client.1.smithi149.stdout:4/278: creat d2/d18/d1e/d3c/d48/f5f x:0 0 0 2024-04-24T08:24:40.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:40 smithi090 ceph-mon[25016]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:24:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:24:40.545 INFO:tasks.workunit.client.1.smithi149.stdout:5/286: dread db/f10 [4194304,4194304] 0 2024-04-24T08:24:40.547 INFO:tasks.workunit.client.1.smithi149.stdout:8/202: fsync d1/de/f2b 0 2024-04-24T08:24:40.548 INFO:tasks.workunit.client.1.smithi149.stdout:8/203: write d1/de/f2b [3710317,52752] 0 2024-04-24T08:24:40.549 INFO:tasks.workunit.client.1.smithi149.stdout:7/243: rename d9/f19 to d9/dc/d11/d38/f4e 0 2024-04-24T08:24:40.556 INFO:tasks.workunit.client.1.smithi149.stdout:1/348: truncate d0/da/fc 581867 0 2024-04-24T08:24:40.559 INFO:tasks.workunit.client.1.smithi149.stdout:1/349: truncate d0/d5/db/f1c 1402452 0 2024-04-24T08:24:40.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/279: link d2/d18/d1e/d3c/d59/f58 d2/d18/d1e/d3b/f60 0 2024-04-24T08:24:40.566 INFO:tasks.workunit.client.1.smithi149.stdout:5/287: mknod db/c54 0 2024-04-24T08:24:40.569 INFO:tasks.workunit.client.1.smithi149.stdout:8/204: symlink d1/de/d18/d32/l3f 0 2024-04-24T08:24:40.571 INFO:tasks.workunit.client.1.smithi149.stdout:1/350: unlink d0/d5/db/d18/d34/c37 0 2024-04-24T08:24:40.571 INFO:tasks.workunit.client.1.smithi149.stdout:1/351: chown d0 3107 1 2024-04-24T08:24:40.580 INFO:tasks.workunit.client.1.smithi149.stdout:6/200: link d0/l22 d0/d1a/d28/d31/d33/l3d 0 2024-04-24T08:24:40.584 INFO:tasks.workunit.client.1.smithi149.stdout:5/288: rename l6 to db/d12/d22/d45/l55 0 2024-04-24T08:24:40.589 INFO:tasks.workunit.client.1.smithi149.stdout:5/289: chown db/d12/d1e/f41 41 1 2024-04-24T08:24:40.590 INFO:tasks.workunit.client.1.smithi149.stdout:0/250: dwrite d2/d17/d41/f4d [0,4194304] 0 2024-04-24T08:24:40.591 INFO:tasks.workunit.client.1.smithi149.stdout:7/244: dread d9/dc/d2e/f39 [0,4194304] 0 2024-04-24T08:24:40.594 INFO:tasks.workunit.client.1.smithi149.stdout:8/205: rename d1/de/f16 to d1/d1f/f40 0 2024-04-24T08:24:40.596 INFO:tasks.workunit.client.1.smithi149.stdout:0/251: rmdir d2/d17/d41 39 2024-04-24T08:24:40.597 INFO:tasks.workunit.client.1.smithi149.stdout:7/245: mkdir d9/d43/d4f 0 2024-04-24T08:24:40.600 INFO:tasks.workunit.client.1.smithi149.stdout:1/352: rename d0/d5/d4f/d60/d19/f23 to d0/d1b/d5d/f6c 0 2024-04-24T08:24:40.601 INFO:tasks.workunit.client.1.smithi149.stdout:5/290: rename db to db/d12/d1e/d56 22 2024-04-24T08:24:40.602 INFO:tasks.workunit.client.1.smithi149.stdout:4/280: dwrite d2/d18/d1e/d3c/d59/f58 [0,4194304] 0 2024-04-24T08:24:40.604 INFO:tasks.workunit.client.1.smithi149.stdout:8/206: mknod d1/d1f/c41 0 2024-04-24T08:24:40.606 INFO:tasks.workunit.client.1.smithi149.stdout:5/291: write db/d12/d2f/f46 [1540330,113867] 0 2024-04-24T08:24:40.607 INFO:tasks.workunit.client.1.smithi149.stdout:7/246: mknod d9/dc/de/d44/c50 0 2024-04-24T08:24:40.607 INFO:tasks.workunit.client.1.smithi149.stdout:6/201: getdents d0/d1a/d28/d31 0 2024-04-24T08:24:40.608 INFO:tasks.workunit.client.1.smithi149.stdout:1/353: mknod d0/d5/db/d18/d34/c6d 0 2024-04-24T08:24:40.609 INFO:tasks.workunit.client.1.smithi149.stdout:5/292: stat db/l51 0 2024-04-24T08:24:40.611 INFO:tasks.workunit.client.1.smithi149.stdout:8/207: creat d1/d1f/d25/f42 x:0 0 0 2024-04-24T08:24:40.612 INFO:tasks.workunit.client.1.smithi149.stdout:4/281: write d2/d18/f55 [730988,18163] 0 2024-04-24T08:24:40.615 INFO:tasks.workunit.client.1.smithi149.stdout:4/282: readlink d2/l12 0 2024-04-24T08:24:40.616 INFO:tasks.workunit.client.1.smithi149.stdout:8/208: dread - d1/de/f38 zero size 2024-04-24T08:24:40.618 INFO:tasks.workunit.client.1.smithi149.stdout:5/293: creat db/d12/d2f/f57 x:0 0 0 2024-04-24T08:24:40.619 INFO:tasks.workunit.client.1.smithi149.stdout:4/283: stat d2/d24/d2e/c57 0 2024-04-24T08:24:40.620 INFO:tasks.workunit.client.1.smithi149.stdout:5/294: readlink db/d12/l17 0 2024-04-24T08:24:40.620 INFO:tasks.workunit.client.1.smithi149.stdout:0/252: dwrite d2/fa [0,4194304] 0 2024-04-24T08:24:40.621 INFO:tasks.workunit.client.1.smithi149.stdout:1/354: creat d0/d5/db/d63/f6e x:0 0 0 2024-04-24T08:24:40.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/209: write d1/de/f1a [4343344,92559] 0 2024-04-24T08:24:40.627 INFO:tasks.workunit.client.1.smithi149.stdout:4/284: rename d2/d18/d1e/d3c to d2/d18/d1e/d3c/d48/d61 22 2024-04-24T08:24:40.629 INFO:tasks.workunit.client.1.smithi149.stdout:4/285: readlink d2/d10/d1a/l1c 0 2024-04-24T08:24:40.630 INFO:tasks.workunit.client.1.smithi149.stdout:8/210: mkdir d1/de/d18/d43 0 2024-04-24T08:24:40.631 INFO:tasks.workunit.client.1.smithi149.stdout:6/202: dread d0/d1a/f25 [0,4194304] 0 2024-04-24T08:24:40.634 INFO:tasks.workunit.client.1.smithi149.stdout:0/253: creat d2/d9/df/d2f/f50 x:0 0 0 2024-04-24T08:24:40.638 INFO:tasks.workunit.client.1.smithi149.stdout:8/211: mknod d1/d1f/d25/c44 0 2024-04-24T08:24:40.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/203: mknod d0/d1a/d28/d31/d33/c3e 0 2024-04-24T08:24:40.648 INFO:tasks.workunit.client.1.smithi149.stdout:7/247: truncate d9/dc/d2e/f39 521006 0 2024-04-24T08:24:40.648 INFO:tasks.workunit.client.1.smithi149.stdout:6/204: write d0/f3 [2890728,51807] 0 2024-04-24T08:24:40.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T08:24:40.654 INFO:tasks.workunit.client.1.smithi149.stdout:7/248: mkdir d9/dc/d2e/d51 0 2024-04-24T08:24:40.655 INFO:tasks.workunit.client.1.smithi149.stdout:5/295: truncate db/f14 484496 0 2024-04-24T08:24:40.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/249: write d9/dc/d2e/f41 [1249865,8217] 0 2024-04-24T08:24:40.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/254: getdents d2/d9/d1a 0 2024-04-24T08:24:40.661 INFO:tasks.workunit.client.1.smithi149.stdout:4/286: truncate d2/d18/d1e/d3c/d59/f58 3832539 0 2024-04-24T08:24:40.662 INFO:tasks.workunit.client.1.smithi149.stdout:8/212: dwrite d1/d1f/d22/f30 [4194304,4194304] 0 2024-04-24T08:24:40.664 INFO:tasks.workunit.client.1.smithi149.stdout:1/355: dwrite d0/d5/f8 [4194304,4194304] 0 2024-04-24T08:24:40.666 INFO:tasks.workunit.client.1.smithi149.stdout:4/287: truncate d2/f3d 481712 0 2024-04-24T08:24:40.670 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T08:24:40.670 INFO:tasks.workunit.client.0.smithi090.stderr:+ make 2024-04-24T08:24:40.672 INFO:tasks.workunit.client.1.smithi149.stdout:5/296: symlink db/d12/l58 0 2024-04-24T08:24:40.674 INFO:tasks.workunit.client.1.smithi149.stdout:6/205: link d0/f3 d0/f3f 0 2024-04-24T08:24:40.676 INFO:tasks.workunit.client.1.smithi149.stdout:6/206: stat d0/d1a/f2e 0 2024-04-24T08:24:40.676 INFO:tasks.workunit.client.1.smithi149.stdout:1/356: unlink d0/d5/db/d18/f45 0 2024-04-24T08:24:40.687 INFO:tasks.workunit.client.1.smithi149.stdout:4/288: creat d2/d10/d1a/d34/f62 x:0 0 0 2024-04-24T08:24:40.691 INFO:tasks.workunit.client.1.smithi149.stdout:0/255: creat d2/d3c/f51 x:0 0 0 2024-04-24T08:24:40.693 INFO:tasks.workunit.client.1.smithi149.stdout:8/213: rename d1/de/d18/f1d to d1/de/d18/f45 0 2024-04-24T08:24:40.697 INFO:tasks.workunit.client.1.smithi149.stdout:5/297: rename db/d12/d1e/f27 to db/d35/f59 0 2024-04-24T08:24:40.697 INFO:tasks.workunit.client.1.smithi149.stdout:8/214: truncate d1/de/d18/f3e 921918 0 2024-04-24T08:24:40.698 INFO:tasks.workunit.client.1.smithi149.stdout:1/357: creat d0/d5/db/d18/f6f x:0 0 0 2024-04-24T08:24:40.700 INFO:tasks.workunit.client.1.smithi149.stdout:4/289: mknod d2/d18/d1e/d3c/d59/c63 0 2024-04-24T08:24:40.700 INFO:tasks.workunit.client.1.smithi149.stdout:5/298: chown db/l51 18292494 1 2024-04-24T08:24:40.703 INFO:tasks.workunit.client.1.smithi149.stdout:6/207: dwrite d0/f11 [4194304,4194304] 0 2024-04-24T08:24:40.712 INFO:tasks.workunit.client.1.smithi149.stdout:7/250: dread d9/dc/f17 [0,4194304] 0 2024-04-24T08:24:40.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/299: mknod db/d1f/c5a 0 2024-04-24T08:24:40.720 INFO:tasks.workunit.client.1.smithi149.stdout:4/290: rename d2/d10/f30 to d2/d18/f64 0 2024-04-24T08:24:40.721 INFO:tasks.workunit.client.1.smithi149.stdout:1/358: creat d0/d39/d53/d61/f70 x:0 0 0 2024-04-24T08:24:40.723 INFO:tasks.workunit.client.1.smithi149.stdout:0/256: getdents d2/d9/df/d40 0 2024-04-24T08:24:40.723 INFO:tasks.workunit.client.1.smithi149.stdout:4/291: fsync d2/d24/d2e/f51 0 2024-04-24T08:24:40.725 INFO:tasks.workunit.client.1.smithi149.stdout:5/300: dread db/f1a [0,4194304] 0 2024-04-24T08:24:40.726 INFO:tasks.workunit.client.1.smithi149.stdout:6/208: dwrite d0/d1a/d28/f2c [4194304,4194304] 0 2024-04-24T08:24:40.728 INFO:tasks.workunit.client.1.smithi149.stdout:8/215: link l0 d1/l46 0 2024-04-24T08:24:40.728 INFO:tasks.workunit.client.1.smithi149.stdout:0/257: write d2/d3c/f51 [795392,115672] 0 2024-04-24T08:24:40.728 INFO:tasks.workunit.client.1.smithi149.stdout:4/292: symlink d2/d24/d2e/l65 0 2024-04-24T08:24:40.734 INFO:tasks.workunit.client.1.smithi149.stdout:4/293: dread d2/d24/d2e/d3f/f40 [0,4194304] 0 2024-04-24T08:24:40.737 INFO:tasks.workunit.client.1.smithi149.stdout:4/294: write d2/d24/d2e/d3f/f4b [4444220,12477] 0 2024-04-24T08:24:40.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/216: unlink d1/l46 0 2024-04-24T08:24:40.743 INFO:tasks.workunit.client.1.smithi149.stdout:6/209: symlink d0/l40 0 2024-04-24T08:24:40.745 INFO:tasks.workunit.client.1.smithi149.stdout:4/295: chown d2/d18/d1e/d3c/c56 65240 1 2024-04-24T08:24:40.747 INFO:tasks.workunit.client.1.smithi149.stdout:7/251: dread d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:40.755 INFO:tasks.workunit.client.1.smithi149.stdout:4/296: write d2/d18/d1e/d3b/f42 [249270,40788] 0 2024-04-24T08:24:40.756 INFO:tasks.workunit.client.1.smithi149.stdout:7/252: rmdir d9/d43 39 2024-04-24T08:24:40.769 INFO:tasks.workunit.client.1.smithi149.stdout:7/253: dread d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:24:40.799 INFO:tasks.workunit.client.1.smithi149.stdout:8/217: creat d1/de/f47 x:0 0 0 2024-04-24T08:24:40.800 INFO:tasks.workunit.client.1.smithi149.stdout:4/297: rmdir d2/d24/d2e/d3f 39 2024-04-24T08:24:40.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/301: link db/f14 db/d12/f5b 0 2024-04-24T08:24:40.807 INFO:tasks.workunit.client.1.smithi149.stdout:5/302: chown db/d12/d2f/f57 102395878 1 2024-04-24T08:24:40.818 INFO:tasks.workunit.client.1.smithi149.stdout:4/298: unlink d2/f3d 0 2024-04-24T08:24:40.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/258: truncate d2/fa 1486259 0 2024-04-24T08:24:40.825 INFO:tasks.workunit.client.1.smithi149.stdout:5/303: creat db/d1f/f5c x:0 0 0 2024-04-24T08:24:40.826 INFO:tasks.workunit.client.1.smithi149.stdout:8/218: truncate d1/de/d18/f45 1254882 0 2024-04-24T08:24:40.832 INFO:tasks.workunit.client.1.smithi149.stdout:0/259: creat d2/d9/df/d48/f52 x:0 0 0 2024-04-24T08:24:40.835 INFO:tasks.workunit.client.0.smithi090.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T08:24:40.836 INFO:tasks.workunit.client.1.smithi149.stdout:8/219: creat d1/de/d18/f48 x:0 0 0 2024-04-24T08:24:40.837 INFO:tasks.workunit.client.1.smithi149.stdout:8/220: stat d1/de/c3c 0 2024-04-24T08:24:40.841 INFO:tasks.workunit.client.1.smithi149.stdout:0/260: read d2/d9/df/f21 [605284,97134] 0 2024-04-24T08:24:40.842 INFO:tasks.workunit.client.1.smithi149.stdout:5/304: mkdir db/d12/d22/d45/d5d 0 2024-04-24T08:24:40.843 INFO:tasks.workunit.client.1.smithi149.stdout:8/221: fdatasync d1/de/d18/f3e 0 2024-04-24T08:24:40.857 INFO:tasks.workunit.client.1.smithi149.stdout:2/219: sync 2024-04-24T08:24:40.861 INFO:tasks.workunit.client.1.smithi149.stdout:0/261: dwrite d2/d9/d1a/f44 [4194304,4194304] 0 2024-04-24T08:24:40.871 INFO:tasks.workunit.client.1.smithi149.stdout:0/262: read d2/d9/d1a/d36/f47 [3602976,23689] 0 2024-04-24T08:24:40.871 INFO:tasks.workunit.client.1.smithi149.stdout:8/222: dwrite d1/d1f/d22/f30 [0,4194304] 0 2024-04-24T08:24:40.879 INFO:tasks.workunit.client.1.smithi149.stdout:0/263: mknod d2/d3c/c53 0 2024-04-24T08:24:40.883 INFO:tasks.workunit.client.1.smithi149.stdout:0/264: dread - d2/d9/f2d zero size 2024-04-24T08:24:40.885 INFO:tasks.workunit.client.1.smithi149.stdout:0/265: stat d2/d9/fe 0 2024-04-24T08:24:40.885 INFO:tasks.workunit.client.1.smithi149.stdout:6/210: rename d0/l1e to d0/d1a/l41 0 2024-04-24T08:24:40.889 INFO:tasks.workunit.client.1.smithi149.stdout:0/266: stat d2/d9/df/d2f/f3d 0 2024-04-24T08:24:40.895 INFO:tasks.workunit.client.1.smithi149.stdout:7/254: rename d9/dc/d11/f2a to d9/d43/d4c/f52 0 2024-04-24T08:24:40.897 INFO:tasks.workunit.client.1.smithi149.stdout:7/255: dread - d9/dc/de/d44/f49 zero size 2024-04-24T08:24:40.898 INFO:tasks.workunit.client.1.smithi149.stdout:8/223: dwrite d1/d1f/f27 [0,4194304] 0 2024-04-24T08:24:40.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/211: symlink d0/d1a/d28/d31/d33/l42 0 2024-04-24T08:24:40.900 INFO:tasks.workunit.client.1.smithi149.stdout:7/256: chown d9/d43 3 1 2024-04-24T08:24:40.904 INFO:tasks.workunit.client.1.smithi149.stdout:4/299: truncate d2/d24/f39 1598930 0 2024-04-24T08:24:40.905 INFO:tasks.workunit.client.1.smithi149.stdout:2/220: dread d0/f8 [0,4194304] 0 2024-04-24T08:24:40.907 INFO:tasks.workunit.client.1.smithi149.stdout:8/224: symlink d1/de/d18/l49 0 2024-04-24T08:24:40.907 INFO:tasks.workunit.client.1.smithi149.stdout:0/267: dread d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:40.910 INFO:tasks.workunit.client.1.smithi149.stdout:8/225: dread - d1/d1f/d25/f42 zero size 2024-04-24T08:24:40.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/226: mknod d1/de/d18/c4a 0 2024-04-24T08:24:40.917 INFO:tasks.workunit.client.1.smithi149.stdout:6/212: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:24:40.918 INFO:tasks.workunit.client.1.smithi149.stdout:4/300: creat d2/d24/d2e/f66 x:0 0 0 2024-04-24T08:24:40.920 INFO:tasks.workunit.client.1.smithi149.stdout:4/301: stat d2/d18/f20 0 2024-04-24T08:24:40.924 INFO:tasks.workunit.client.1.smithi149.stdout:6/213: read d0/f12 [441226,98958] 0 2024-04-24T08:24:40.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/257: getdents d9/d43 0 2024-04-24T08:24:40.928 INFO:tasks.workunit.client.1.smithi149.stdout:5/305: truncate db/d12/f3d 339865 0 2024-04-24T08:24:40.929 INFO:tasks.workunit.client.1.smithi149.stdout:8/227: fsync d1/f9 0 2024-04-24T08:24:40.932 INFO:tasks.workunit.client.1.smithi149.stdout:2/221: getdents d0/d6 0 2024-04-24T08:24:40.933 INFO:tasks.workunit.client.1.smithi149.stdout:8/228: chown d1/de/d18/c3d 10 1 2024-04-24T08:24:40.934 INFO:tasks.workunit.client.1.smithi149.stdout:8/229: chown d1 28 1 2024-04-24T08:24:40.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/230: fsync d1/d1f/f28 0 2024-04-24T08:24:40.935 INFO:tasks.workunit.client.1.smithi149.stdout:7/258: creat d9/dc/d11/f53 x:0 0 0 2024-04-24T08:24:40.937 INFO:tasks.workunit.client.1.smithi149.stdout:5/306: write db/d12/d1e/f3c [2869818,16081] 0 2024-04-24T08:24:40.937 INFO:tasks.workunit.client.1.smithi149.stdout:6/214: symlink d0/d1a/d28/d31/d38/l43 0 2024-04-24T08:24:40.942 INFO:tasks.workunit.client.1.smithi149.stdout:4/302: rename d2/d18/d1e/d3c/d59/f58 to d2/d24/d2e/d3f/f67 0 2024-04-24T08:24:40.943 INFO:tasks.workunit.client.1.smithi149.stdout:4/303: stat d2/d18/d1e/d3c/d59 0 2024-04-24T08:24:40.948 INFO:tasks.workunit.client.1.smithi149.stdout:2/222: unlink d0/d6/c1d 0 2024-04-24T08:24:40.949 INFO:tasks.workunit.client.1.smithi149.stdout:8/231: symlink d1/d1f/d31/l4b 0 2024-04-24T08:24:40.950 INFO:tasks.workunit.client.1.smithi149.stdout:5/307: symlink db/d12/d1e/l5e 0 2024-04-24T08:24:40.956 INFO:tasks.workunit.client.1.smithi149.stdout:5/308: mknod db/d35/c5f 0 2024-04-24T08:24:40.959 INFO:tasks.workunit.client.1.smithi149.stdout:7/259: link d9/dc/d11/c32 d9/dc/de/d44/c54 0 2024-04-24T08:24:40.961 INFO:tasks.workunit.client.1.smithi149.stdout:6/215: link d0/d1a/c1d d0/d1a/d28/d31/c44 0 2024-04-24T08:24:40.965 INFO:tasks.workunit.client.1.smithi149.stdout:4/304: rename d2/d10/d1a/d34/l49 to d2/d24/d2e/l68 0 2024-04-24T08:24:40.966 INFO:tasks.workunit.client.1.smithi149.stdout:8/232: dwrite d1/de/d18/f48 [0,4194304] 0 2024-04-24T08:24:40.967 INFO:tasks.workunit.client.1.smithi149.stdout:4/305: chown d2/d10/c35 1 1 2024-04-24T08:24:40.971 INFO:tasks.workunit.client.1.smithi149.stdout:7/260: mknod d9/dc/d11/c55 0 2024-04-24T08:24:40.973 INFO:tasks.workunit.client.1.smithi149.stdout:6/216: rename d0/d1a/l20 to d0/d1a/d28/l45 0 2024-04-24T08:24:40.983 INFO:tasks.workunit.client.1.smithi149.stdout:8/233: creat d1/de/d18/f4c x:0 0 0 2024-04-24T08:24:40.983 INFO:tasks.workunit.client.1.smithi149.stdout:4/306: dread d2/d18/d1e/f33 [0,4194304] 0 2024-04-24T08:24:40.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/261: creat d9/dc/d11/d38/f56 x:0 0 0 2024-04-24T08:24:40.986 INFO:tasks.workunit.client.1.smithi149.stdout:5/309: dwrite db/d12/d1e/f4a [0,4194304] 0 2024-04-24T08:24:40.986 INFO:tasks.workunit.client.1.smithi149.stdout:0/268: dwrite d2/f8 [0,4194304] 0 2024-04-24T08:24:40.993 INFO:tasks.workunit.client.1.smithi149.stdout:5/310: chown db/d12/d1e/f4f 10 1 2024-04-24T08:24:40.994 INFO:tasks.workunit.client.1.smithi149.stdout:0/269: write d2/d9/d1a/d36/f38 [767974,42889] 0 2024-04-24T08:24:41.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/307: mkdir d2/d18/d1e/d3c/d48/d69 0 2024-04-24T08:24:41.003 INFO:tasks.workunit.client.1.smithi149.stdout:4/308: rename d2 to d2/d24/d6a 22 2024-04-24T08:24:41.005 INFO:tasks.workunit.client.1.smithi149.stdout:2/223: creat d0/d6/db/d28/f46 x:0 0 0 2024-04-24T08:24:41.011 INFO:tasks.workunit.client.1.smithi149.stdout:4/309: mknod d2/d18/d1e/d3c/d48/c6b 0 2024-04-24T08:24:41.014 INFO:tasks.workunit.client.1.smithi149.stdout:8/234: link d1/de/d18/l26 d1/d1f/d25/l4d 0 2024-04-24T08:24:41.017 INFO:tasks.workunit.client.1.smithi149.stdout:4/310: rmdir d2/d10/d1a 39 2024-04-24T08:24:41.020 INFO:tasks.workunit.client.1.smithi149.stdout:6/217: truncate d0/f11 7446501 0 2024-04-24T08:24:41.020 INFO:tasks.workunit.client.1.smithi149.stdout:2/224: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:41.021 INFO:tasks.workunit.client.1.smithi149.stdout:4/311: write d2/d24/d2e/f66 [436362,94979] 0 2024-04-24T08:24:41.025 INFO:tasks.workunit.client.1.smithi149.stdout:2/225: chown d0/d6/f41 174 1 2024-04-24T08:24:41.027 INFO:tasks.workunit.client.1.smithi149.stdout:4/312: chown d2/d10/d1a/f3e 11888099 1 2024-04-24T08:24:41.030 INFO:tasks.workunit.client.1.smithi149.stdout:2/226: creat d0/d6/db/d27/f47 x:0 0 0 2024-04-24T08:24:41.031 INFO:tasks.workunit.client.1.smithi149.stdout:4/313: dread - d2/d24/d2e/f5b zero size 2024-04-24T08:24:41.033 INFO:tasks.workunit.client.1.smithi149.stdout:2/227: write d0/d6/db/f42 [720304,2639] 0 2024-04-24T08:24:41.034 INFO:tasks.workunit.client.1.smithi149.stdout:0/270: truncate d2/d9/d1a/f25 142640 0 2024-04-24T08:24:41.034 INFO:tasks.workunit.client.1.smithi149.stdout:7/262: dread d9/dc/de/f33 [4194304,4194304] 0 2024-04-24T08:24:41.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/228: write d0/d6/db/d27/f31 [170061,79785] 0 2024-04-24T08:24:41.038 INFO:tasks.workunit.client.1.smithi149.stdout:8/235: truncate d1/de/d18/f3e 849377 0 2024-04-24T08:24:41.041 INFO:tasks.workunit.client.1.smithi149.stdout:4/314: creat d2/f6c x:0 0 0 2024-04-24T08:24:41.043 INFO:tasks.workunit.client.1.smithi149.stdout:2/229: write d0/d6/db/d27/f45 [277611,24546] 0 2024-04-24T08:24:41.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/315: rename d2 to d2/d18/d1e/d3c/d48/d6d 22 2024-04-24T08:24:41.050 INFO:tasks.workunit.client.1.smithi149.stdout:2/230: creat d0/d6/db/d28/f48 x:0 0 0 2024-04-24T08:24:41.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/218: dwrite d0/f16 [0,4194304] 0 2024-04-24T08:24:41.059 INFO:tasks.workunit.client.1.smithi149.stdout:4/316: rename d2/d18/d1e/d3b/c41 to d2/d24/d2e/d38/c6e 0 2024-04-24T08:24:41.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/219: chown d0/d1a/d28/d31/l35 1 1 2024-04-24T08:24:41.062 INFO:tasks.workunit.client.1.smithi149.stdout:7/263: dwrite d9/dc/d11/d26/f4a [0,4194304] 0 2024-04-24T08:24:41.065 INFO:tasks.workunit.client.1.smithi149.stdout:8/236: dwrite d1/d1f/f28 [0,4194304] 0 2024-04-24T08:24:41.069 INFO:tasks.workunit.client.1.smithi149.stdout:0/271: dwrite d2/d3c/f49 [0,4194304] 0 2024-04-24T08:24:41.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/237: mknod d1/d1f/d25/c4e 0 2024-04-24T08:24:41.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/272: getdents d2 0 2024-04-24T08:24:41.086 INFO:tasks.workunit.client.1.smithi149.stdout:8/238: creat d1/f4f x:0 0 0 2024-04-24T08:24:41.088 INFO:tasks.workunit.client.1.smithi149.stdout:0/273: mknod d2/d3c/c54 0 2024-04-24T08:24:41.096 INFO:tasks.workunit.client.1.smithi149.stdout:4/317: dwrite d2/d18/f1b [0,4194304] 0 2024-04-24T08:24:41.098 INFO:tasks.workunit.client.1.smithi149.stdout:7/264: dwrite d9/dc/d11/d38/f56 [0,4194304] 0 2024-04-24T08:24:41.101 INFO:tasks.workunit.client.1.smithi149.stdout:8/239: link d1/d1f/f40 d1/d1f/f50 0 2024-04-24T08:24:41.102 INFO:tasks.workunit.client.1.smithi149.stdout:0/274: write d2/d9/df/d48/f4c [658215,101233] 0 2024-04-24T08:24:41.108 INFO:tasks.workunit.client.1.smithi149.stdout:4/318: creat d2/d10/d1a/f6f x:0 0 0 2024-04-24T08:24:41.112 INFO:tasks.workunit.client.1.smithi149.stdout:7/265: mknod d9/c57 0 2024-04-24T08:24:41.112 INFO:tasks.workunit.client.1.smithi149.stdout:4/319: stat d2/d18/f20 0 2024-04-24T08:24:41.112 INFO:tasks.workunit.client.1.smithi149.stdout:8/240: readlink d1/d1f/d25/l4d 0 2024-04-24T08:24:41.124 INFO:tasks.workunit.client.1.smithi149.stdout:8/241: symlink d1/d1f/d31/l51 0 2024-04-24T08:24:41.126 INFO:tasks.workunit.client.1.smithi149.stdout:0/275: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:41.126 INFO:tasks.workunit.client.1.smithi149.stdout:7/266: creat d9/dc/d11/f58 x:0 0 0 2024-04-24T08:24:41.129 INFO:tasks.workunit.client.1.smithi149.stdout:4/320: dread d2/d24/d2e/f4a [0,4194304] 0 2024-04-24T08:24:41.130 INFO:tasks.workunit.client.1.smithi149.stdout:4/321: dread - d2/d10/d1a/f6f zero size 2024-04-24T08:24:41.132 INFO:tasks.workunit.client.1.smithi149.stdout:8/242: creat d1/d1f/f52 x:0 0 0 2024-04-24T08:24:41.136 INFO:tasks.workunit.client.1.smithi149.stdout:7/267: unlink d9/dc/d11/f53 0 2024-04-24T08:24:41.139 INFO:tasks.workunit.client.1.smithi149.stdout:8/243: readlink d1/l5 0 2024-04-24T08:24:41.141 INFO:tasks.workunit.client.1.smithi149.stdout:4/322: creat d2/d18/d1e/d3c/d48/d69/f70 x:0 0 0 2024-04-24T08:24:41.143 INFO:tasks.workunit.client.1.smithi149.stdout:8/244: symlink d1/de/d18/d32/l53 0 2024-04-24T08:24:41.144 INFO:tasks.workunit.client.1.smithi149.stdout:8/245: dread - d1/de/f38 zero size 2024-04-24T08:24:41.145 INFO:tasks.workunit.client.1.smithi149.stdout:4/323: unlink d2/d24/d2e/l65 0 2024-04-24T08:24:41.146 INFO:tasks.workunit.client.1.smithi149.stdout:4/324: chown d2/d18/d1e/l22 4 1 2024-04-24T08:24:41.157 INFO:tasks.workunit.client.1.smithi149.stdout:7/268: dwrite d9/dc/d11/d38/f4e [0,4194304] 0 2024-04-24T08:24:41.160 INFO:tasks.workunit.client.1.smithi149.stdout:7/269: write d9/dc/de/d44/f49 [904842,111217] 0 2024-04-24T08:24:41.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/270: rmdir d9/d43 39 2024-04-24T08:24:41.180 INFO:tasks.workunit.client.1.smithi149.stdout:7/271: dwrite d9/dc/f18 [0,4194304] 0 2024-04-24T08:24:41.195 INFO:tasks.workunit.client.1.smithi149.stdout:3/264: sync 2024-04-24T08:24:41.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/265: chown d0/d5/d53/d34/f3b 473276558 1 2024-04-24T08:24:41.199 INFO:tasks.workunit.client.1.smithi149.stdout:7/272: creat d9/dc/d2e/d51/f59 x:0 0 0 2024-04-24T08:24:41.201 INFO:tasks.workunit.client.1.smithi149.stdout:3/266: truncate d0/d5/d53/d39/f56 284659 0 2024-04-24T08:24:41.201 INFO:tasks.workunit.client.1.smithi149.stdout:7/273: write d9/dc/f1b [1591006,129815] 0 2024-04-24T08:24:41.203 INFO:tasks.workunit.client.1.smithi149.stdout:7/274: write d9/f2c [345619,57042] 0 2024-04-24T08:24:41.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/267: symlink d0/d5/d53/d34/l5b 0 2024-04-24T08:24:41.215 INFO:tasks.workunit.client.1.smithi149.stdout:7/275: symlink d9/d43/l5a 0 2024-04-24T08:24:41.216 INFO:tasks.workunit.client.1.smithi149.stdout:8/246: dwrite d1/d1f/f2c [0,4194304] 0 2024-04-24T08:24:41.217 INFO:tasks.workunit.client.1.smithi149.stdout:8/247: chown d1/de/d18/l3a 40829608 1 2024-04-24T08:24:41.219 INFO:tasks.workunit.client.1.smithi149.stdout:8/248: dread - d1/de/d18/f4c zero size 2024-04-24T08:24:41.221 INFO:tasks.workunit.client.1.smithi149.stdout:7/276: link d9/dc/f18 d9/f5b 0 2024-04-24T08:24:41.224 INFO:tasks.workunit.client.1.smithi149.stdout:8/249: rmdir d1/de/d18 39 2024-04-24T08:24:41.294 INFO:tasks.workunit.client.1.smithi149.stdout:3/268: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:41.298 INFO:tasks.workunit.client.1.smithi149.stdout:3/269: mkdir d0/d26/d5c 0 2024-04-24T08:24:41.298 INFO:tasks.workunit.client.1.smithi149.stdout:3/270: chown d0/d5/d53/d39 1510243 1 2024-04-24T08:24:41.301 INFO:tasks.workunit.client.1.smithi149.stdout:8/250: dwrite d1/d1f/f28 [0,4194304] 0 2024-04-24T08:24:41.302 INFO:tasks.workunit.client.1.smithi149.stdout:8/251: fdatasync d1/f4f 0 2024-04-24T08:24:41.305 INFO:tasks.workunit.client.1.smithi149.stdout:8/252: write d1/d1f/f2c [2703670,8411] 0 2024-04-24T08:24:41.305 INFO:tasks.workunit.client.1.smithi149.stdout:3/271: creat d0/d5/d1a/f5d x:0 0 0 2024-04-24T08:24:41.306 INFO:tasks.workunit.client.1.smithi149.stdout:3/272: chown d0/f4 111317820 1 2024-04-24T08:24:41.311 INFO:tasks.workunit.client.1.smithi149.stdout:3/273: read d0/d5/d53/d34/d3a/f33 [2256069,101329] 0 2024-04-24T08:24:41.314 INFO:tasks.workunit.client.1.smithi149.stdout:8/253: rename d1/d1f/f27 to d1/de/f54 0 2024-04-24T08:24:41.319 INFO:tasks.workunit.client.1.smithi149.stdout:3/274: rename d0/d5/da/d43 to d0/d5/da/d5e 0 2024-04-24T08:24:41.370 INFO:tasks.workunit.client.1.smithi149.stdout:9/246: sync 2024-04-24T08:24:41.370 INFO:tasks.workunit.client.1.smithi149.stdout:1/359: sync 2024-04-24T08:24:41.373 INFO:tasks.workunit.client.1.smithi149.stdout:1/360: readlink d0/d5/df/l3b 0 2024-04-24T08:24:41.375 INFO:tasks.workunit.client.1.smithi149.stdout:3/275: write d0/fe [1732878,22308] 0 2024-04-24T08:24:41.377 INFO:tasks.workunit.client.1.smithi149.stdout:9/247: mknod d1/da/dd/c4d 0 2024-04-24T08:24:41.378 INFO:tasks.workunit.client.1.smithi149.stdout:1/361: write d0/d1b/d5d/f6c [2324817,32203] 0 2024-04-24T08:24:41.380 INFO:tasks.workunit.client.1.smithi149.stdout:3/276: mkdir d0/d5/d53/d4c/d5f 0 2024-04-24T08:24:41.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/248: creat d1/d2/d33/f4e x:0 0 0 2024-04-24T08:24:41.383 INFO:tasks.workunit.client.1.smithi149.stdout:3/277: mknod d0/d26/c60 0 2024-04-24T08:24:41.387 INFO:tasks.workunit.client.1.smithi149.stdout:1/362: link d0/d1b/f64 d0/d39/d53/d61/f71 0 2024-04-24T08:24:41.387 INFO:tasks.workunit.client.1.smithi149.stdout:9/249: fsync d1/de/d12/f1d 0 2024-04-24T08:24:41.389 INFO:tasks.workunit.client.1.smithi149.stdout:3/278: mkdir d0/d5/da/d19/d1b/d61 0 2024-04-24T08:24:41.391 INFO:tasks.workunit.client.1.smithi149.stdout:1/363: creat d0/d5/db/f72 x:0 0 0 2024-04-24T08:24:41.392 INFO:tasks.workunit.client.1.smithi149.stdout:9/250: symlink d1/da/l4f 0 2024-04-24T08:24:41.394 INFO:tasks.workunit.client.1.smithi149.stdout:9/251: chown d1/d2/d15 182578 1 2024-04-24T08:24:41.395 INFO:tasks.workunit.client.1.smithi149.stdout:1/364: symlink d0/d39/d53/l73 0 2024-04-24T08:24:41.398 INFO:tasks.workunit.client.1.smithi149.stdout:3/279: link d0/d5/da/d19/l45 d0/d26/d3c/l62 0 2024-04-24T08:24:41.400 INFO:tasks.workunit.client.1.smithi149.stdout:9/252: creat d1/da/f50 x:0 0 0 2024-04-24T08:24:41.401 INFO:tasks.workunit.client.1.smithi149.stdout:1/365: mknod d0/d5/db/d18/d4d/d50/c74 0 2024-04-24T08:24:41.402 INFO:tasks.workunit.client.1.smithi149.stdout:9/253: chown d1/da/d13/l43 0 1 2024-04-24T08:24:41.403 INFO:tasks.workunit.client.1.smithi149.stdout:3/280: mkdir d0/d5/da/d5a/d63 0 2024-04-24T08:24:41.403 INFO:tasks.workunit.client.1.smithi149.stdout:9/254: chown d1/d2/d15/d1c 34254 1 2024-04-24T08:24:41.405 INFO:tasks.workunit.client.1.smithi149.stdout:1/366: symlink d0/d5/db/d18/d34/d51/l75 0 2024-04-24T08:24:41.405 INFO:tasks.workunit.client.1.smithi149.stdout:9/255: write d1/de/f2e [787907,99355] 0 2024-04-24T08:24:41.407 INFO:tasks.workunit.client.1.smithi149.stdout:3/281: write d0/d5/d53/d39/f56 [924324,34798] 0 2024-04-24T08:24:41.409 INFO:tasks.workunit.client.1.smithi149.stdout:1/367: rmdir d0/d1b/d5d 39 2024-04-24T08:24:41.410 INFO:tasks.workunit.client.1.smithi149.stdout:9/256: write d1/da/d13/f45 [931250,53837] 0 2024-04-24T08:24:41.413 INFO:tasks.workunit.client.1.smithi149.stdout:3/282: getdents d0 0 2024-04-24T08:24:41.414 INFO:tasks.workunit.client.1.smithi149.stdout:1/368: rename d0/d5/d4f/d65 to d0/d39/d76 0 2024-04-24T08:24:41.415 INFO:tasks.workunit.client.1.smithi149.stdout:9/257: symlink d1/d2/d15/l51 0 2024-04-24T08:24:41.420 INFO:tasks.workunit.client.1.smithi149.stdout:1/369: link d0/d5/db/f72 d0/d5/d4f/f77 0 2024-04-24T08:24:41.441 INFO:tasks.workunit.client.1.smithi149.stdout:3/283: dwrite d0/d5/da/d19/f48 [0,4194304] 0 2024-04-24T08:24:41.441 INFO:tasks.workunit.client.1.smithi149.stdout:9/258: dwrite d1/d2/f2b [0,4194304] 0 2024-04-24T08:24:41.444 INFO:tasks.workunit.client.1.smithi149.stdout:9/259: mknod d1/de/d12/c52 0 2024-04-24T08:24:41.449 INFO:tasks.workunit.client.1.smithi149.stdout:3/284: link d0/d5/d53/f47 d0/d5/da/d19/d1b/d22/d38/f64 0 2024-04-24T08:24:41.452 INFO:tasks.workunit.client.1.smithi149.stdout:9/260: rename d1/de/l14 to d1/d2/d15/d1c/l53 0 2024-04-24T08:24:41.457 INFO:tasks.workunit.client.1.smithi149.stdout:9/261: fdatasync d1/de/f18 0 2024-04-24T08:24:41.457 INFO:tasks.workunit.client.1.smithi149.stdout:3/285: creat d0/d26/d3c/f65 x:0 0 0 2024-04-24T08:24:41.469 INFO:tasks.workunit.client.1.smithi149.stdout:9/262: dread d1/da/fc [0,4194304] 0 2024-04-24T08:24:41.478 INFO:tasks.workunit.client.1.smithi149.stdout:3/286: truncate d0/d5/d53/d34/f3b 3199292 0 2024-04-24T08:24:41.481 INFO:tasks.workunit.client.1.smithi149.stdout:3/287: creat d0/d5/f66 x:0 0 0 2024-04-24T08:24:41.481 INFO:tasks.workunit.client.1.smithi149.stdout:3/288: chown d0/d26/d3c 89 1 2024-04-24T08:24:41.483 INFO:tasks.workunit.client.1.smithi149.stdout:9/263: dwrite d1/d2/f2b [0,4194304] 0 2024-04-24T08:24:41.484 INFO:tasks.workunit.client.1.smithi149.stdout:3/289: unlink d0/l18 0 2024-04-24T08:24:41.485 INFO:tasks.workunit.client.1.smithi149.stdout:9/264: chown d1/da/dd/l16 1503263086 1 2024-04-24T08:24:41.498 INFO:tasks.workunit.client.1.smithi149.stdout:3/290: stat d0/d5/d53/d39 0 2024-04-24T08:24:41.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/265: dwrite d1/d2/f41 [0,4194304] 0 2024-04-24T08:24:41.505 INFO:tasks.workunit.client.1.smithi149.stdout:3/291: rename d0/d5/da/d19/l27 to d0/d5/da/d5a/l67 0 2024-04-24T08:24:41.507 INFO:tasks.workunit.client.1.smithi149.stdout:3/292: chown d0/d5/d53/d34/d3a/f36 15 1 2024-04-24T08:24:41.508 INFO:tasks.workunit.client.1.smithi149.stdout:9/266: rename d1/da/f29 to d1/d4c/f54 0 2024-04-24T08:24:41.512 INFO:tasks.workunit.client.1.smithi149.stdout:9/267: mkdir d1/da/dd/d55 0 2024-04-24T08:24:41.514 INFO:tasks.workunit.client.1.smithi149.stdout:3/293: getdents d0/d5 0 2024-04-24T08:24:41.519 INFO:tasks.workunit.client.1.smithi149.stdout:3/294: unlink d0/d5/da/d5e/f55 0 2024-04-24T08:24:41.521 INFO:tasks.workunit.client.1.smithi149.stdout:9/268: creat d1/da/d49/f56 x:0 0 0 2024-04-24T08:24:41.522 INFO:tasks.workunit.client.1.smithi149.stdout:3/295: symlink d0/l68 0 2024-04-24T08:24:41.523 INFO:tasks.workunit.client.1.smithi149.stdout:9/269: symlink d1/l57 0 2024-04-24T08:24:41.525 INFO:tasks.workunit.client.1.smithi149.stdout:3/296: rmdir d0/d5 39 2024-04-24T08:24:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:41 smithi149 ceph-mon[31282]: pgmap v13: 65 pgs: 65 active+clean; 500 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 44 MiB/s wr, 150 op/s 2024-04-24T08:24:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:41 smithi149 ceph-mon[31282]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:24:41.526 INFO:tasks.workunit.client.1.smithi149.stdout:9/270: creat d1/d2/d15/f58 x:0 0 0 2024-04-24T08:24:41.528 INFO:tasks.workunit.client.1.smithi149.stdout:9/271: chown d1/da/dd/c1b 3371 1 2024-04-24T08:24:41.528 INFO:tasks.workunit.client.1.smithi149.stdout:3/297: stat d0/d5/d53/c42 0 2024-04-24T08:24:41.531 INFO:tasks.workunit.client.1.smithi149.stdout:3/298: mknod d0/d5/da/d19/d1b/c69 0 2024-04-24T08:24:41.532 INFO:tasks.workunit.client.1.smithi149.stdout:3/299: readlink d0/d5/d53/l52 0 2024-04-24T08:24:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:41 smithi090 ceph-mon[25016]: pgmap v13: 65 pgs: 65 active+clean; 500 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 44 MiB/s wr, 150 op/s 2024-04-24T08:24:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:41 smithi090 ceph-mon[25016]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-24T08:24:41.543 INFO:tasks.workunit.client.1.smithi149.stdout:9/272: dread d1/da/f8 [0,4194304] 0 2024-04-24T08:24:41.547 INFO:tasks.workunit.client.1.smithi149.stdout:9/273: write d1/d2/d15/d1c/f2a [496030,80581] 0 2024-04-24T08:24:41.549 INFO:tasks.workunit.client.1.smithi149.stdout:9/274: fsync d1/d4c/f54 0 2024-04-24T08:24:41.550 INFO:tasks.workunit.client.1.smithi149.stdout:3/300: dwrite d0/d5/d1a/f1e [0,4194304] 0 2024-04-24T08:24:41.553 INFO:tasks.workunit.client.1.smithi149.stdout:9/275: write d1/da/f3e [851577,31861] 0 2024-04-24T08:24:41.554 INFO:tasks.workunit.client.1.smithi149.stdout:9/276: chown d1/d2 525 1 2024-04-24T08:24:41.555 INFO:tasks.workunit.client.1.smithi149.stdout:3/301: mkdir d0/d5/da/d5a/d63/d6a 0 2024-04-24T08:24:41.558 INFO:tasks.workunit.client.1.smithi149.stdout:3/302: symlink d0/d5/da/d19/d1b/d22/l6b 0 2024-04-24T08:24:41.567 INFO:tasks.workunit.client.1.smithi149.stdout:3/303: truncate d0/d5/da/fb 454312 0 2024-04-24T08:24:41.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/311: sync 2024-04-24T08:24:41.572 INFO:tasks.workunit.client.1.smithi149.stdout:9/277: dwrite d1/d2/d15/f30 [0,4194304] 0 2024-04-24T08:24:41.577 INFO:tasks.workunit.client.1.smithi149.stdout:5/312: getdents db/d23 0 2024-04-24T08:24:41.582 INFO:tasks.workunit.client.1.smithi149.stdout:9/278: dread d1/da/f8 [0,4194304] 0 2024-04-24T08:24:41.583 INFO:tasks.workunit.client.1.smithi149.stdout:5/313: mkdir db/d12/d60 0 2024-04-24T08:24:41.587 INFO:tasks.workunit.client.1.smithi149.stdout:5/314: creat db/d35/f61 x:0 0 0 2024-04-24T08:24:41.588 INFO:tasks.workunit.client.1.smithi149.stdout:9/279: creat d1/f59 x:0 0 0 2024-04-24T08:24:41.589 INFO:tasks.workunit.client.1.smithi149.stdout:9/280: chown d1/de/f18 1252 1 2024-04-24T08:24:41.590 INFO:tasks.workunit.client.1.smithi149.stdout:3/304: dwrite d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:41.590 INFO:tasks.workunit.client.1.smithi149.stdout:5/315: creat db/d12/d2f/f62 x:0 0 0 2024-04-24T08:24:41.592 INFO:tasks.workunit.client.1.smithi149.stdout:9/281: write d1/d2/d34/f40 [4440780,18941] 0 2024-04-24T08:24:41.592 INFO:tasks.workunit.client.1.smithi149.stdout:5/316: chown db/c15 1 1 2024-04-24T08:24:41.596 INFO:tasks.workunit.client.1.smithi149.stdout:9/282: mknod d1/d2/c5a 0 2024-04-24T08:24:41.601 INFO:tasks.workunit.client.1.smithi149.stdout:9/283: unlink d1/da/c39 0 2024-04-24T08:24:41.602 INFO:tasks.workunit.client.1.smithi149.stdout:9/284: readlink d1/l38 0 2024-04-24T08:24:41.603 INFO:tasks.workunit.client.1.smithi149.stdout:3/305: write d0/d5/da/d19/d1b/d22/d38/f64 [518070,14498] 0 2024-04-24T08:24:41.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/306: symlink d0/d5/d53/d34/l6c 0 2024-04-24T08:24:41.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/317: dread db/d12/d2f/f46 [0,4194304] 0 2024-04-24T08:24:41.612 INFO:tasks.workunit.client.1.smithi149.stdout:5/318: creat db/d12/d22/d45/d5d/f63 x:0 0 0 2024-04-24T08:24:41.615 INFO:tasks.workunit.client.1.smithi149.stdout:9/285: truncate d1/d2/d15/d1c/f2a 951072 0 2024-04-24T08:24:41.617 INFO:tasks.workunit.client.1.smithi149.stdout:5/319: link db/d12/d2f/f62 db/d12/d1e/f64 0 2024-04-24T08:24:41.618 INFO:tasks.workunit.client.1.smithi149.stdout:9/286: mknod d1/de/c5b 0 2024-04-24T08:24:41.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/320: read - db/d23/f43 zero size 2024-04-24T08:24:41.620 INFO:tasks.workunit.client.1.smithi149.stdout:9/287: write d1/d2/f41 [4082370,105727] 0 2024-04-24T08:24:41.622 INFO:tasks.workunit.client.1.smithi149.stdout:5/321: mkdir db/d12/d1e/d65 0 2024-04-24T08:24:41.624 INFO:tasks.workunit.client.1.smithi149.stdout:3/307: dwrite d0/d5/d53/d39/f56 [0,4194304] 0 2024-04-24T08:24:41.624 INFO:tasks.workunit.client.1.smithi149.stdout:9/288: rmdir d1/da 39 2024-04-24T08:24:41.626 INFO:tasks.workunit.client.1.smithi149.stdout:9/289: truncate d1/f59 547319 0 2024-04-24T08:24:41.627 INFO:tasks.workunit.client.1.smithi149.stdout:5/322: creat db/d12/d22/d45/d5d/f66 x:0 0 0 2024-04-24T08:24:41.628 INFO:tasks.workunit.client.1.smithi149.stdout:3/308: mkdir d0/d5/d53/d6d 0 2024-04-24T08:24:41.633 INFO:tasks.workunit.client.1.smithi149.stdout:5/323: dread db/d35/f59 [0,4194304] 0 2024-04-24T08:24:41.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/290: rmdir d1/da/d13 39 2024-04-24T08:24:41.638 INFO:tasks.workunit.client.1.smithi149.stdout:3/309: creat d0/d5/da/f6e x:0 0 0 2024-04-24T08:24:41.639 INFO:tasks.workunit.client.1.smithi149.stdout:5/324: rename db/f10 to db/d12/d60/f67 0 2024-04-24T08:24:41.640 INFO:tasks.workunit.client.1.smithi149.stdout:3/310: chown d0/d5/d53/d34/d3a/l31 19038543 1 2024-04-24T08:24:41.640 INFO:tasks.workunit.client.1.smithi149.stdout:9/291: mknod d1/d4c/c5c 0 2024-04-24T08:24:41.642 INFO:tasks.workunit.client.1.smithi149.stdout:3/311: chown d0/d5/d53/f47 187 1 2024-04-24T08:24:41.643 INFO:tasks.workunit.client.1.smithi149.stdout:5/325: dread db/d35/f59 [0,4194304] 0 2024-04-24T08:24:41.647 INFO:tasks.workunit.client.1.smithi149.stdout:9/292: dread d1/de/f2e [0,4194304] 0 2024-04-24T08:24:41.647 INFO:tasks.workunit.client.1.smithi149.stdout:3/312: truncate d0/d5/da/d5e/f30 2657516 0 2024-04-24T08:24:41.650 INFO:tasks.workunit.client.1.smithi149.stdout:2/231: sync 2024-04-24T08:24:41.650 INFO:tasks.workunit.client.1.smithi149.stdout:6/220: sync 2024-04-24T08:24:41.652 INFO:tasks.workunit.client.1.smithi149.stdout:6/221: fdatasync d0/fa 0 2024-04-24T08:24:41.654 INFO:tasks.workunit.client.1.smithi149.stdout:2/232: chown d0/d6/l19 10359 1 2024-04-24T08:24:41.657 INFO:tasks.workunit.client.1.smithi149.stdout:3/313: getdents d0/d5/da/d19/d1b/d22/d38 0 2024-04-24T08:24:41.663 INFO:tasks.workunit.client.1.smithi149.stdout:6/222: getdents d0/d1a/d28 0 2024-04-24T08:24:41.664 INFO:tasks.workunit.client.1.smithi149.stdout:5/326: dwrite db/d1f/f30 [0,4194304] 0 2024-04-24T08:24:41.666 INFO:tasks.workunit.client.1.smithi149.stdout:6/223: fdatasync d0/f17 0 2024-04-24T08:24:41.666 INFO:tasks.workunit.client.1.smithi149.stdout:9/293: write d1/de/ff [786571,42704] 0 2024-04-24T08:24:41.680 INFO:tasks.workunit.client.1.smithi149.stdout:6/224: getdents d0/d1a/d28/d31 0 2024-04-24T08:24:41.680 INFO:tasks.workunit.client.1.smithi149.stdout:3/314: dwrite d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:41.684 INFO:tasks.workunit.client.1.smithi149.stdout:3/315: write d0/d5/d53/f47 [1551976,120461] 0 2024-04-24T08:24:41.685 INFO:tasks.workunit.client.1.smithi149.stdout:6/225: symlink d0/d1a/d28/d31/d33/l46 0 2024-04-24T08:24:41.686 INFO:tasks.workunit.client.1.smithi149.stdout:3/316: write d0/d5/da/f24 [1469692,6579] 0 2024-04-24T08:24:41.688 INFO:tasks.workunit.client.1.smithi149.stdout:5/327: truncate db/f20 248942 0 2024-04-24T08:24:41.689 INFO:tasks.workunit.client.1.smithi149.stdout:6/226: symlink d0/d1a/d28/d31/d38/l47 0 2024-04-24T08:24:41.694 INFO:tasks.workunit.client.1.smithi149.stdout:3/317: link d0/fe d0/d5/f6f 0 2024-04-24T08:24:41.695 INFO:tasks.workunit.client.1.smithi149.stdout:5/328: truncate db/d12/f5b 547317 0 2024-04-24T08:24:41.697 INFO:tasks.workunit.client.1.smithi149.stdout:3/318: stat d0/d5/da/d5e/l4b 0 2024-04-24T08:24:41.699 INFO:tasks.workunit.client.1.smithi149.stdout:3/319: mkdir d0/d5/d1a/d70 0 2024-04-24T08:24:41.704 INFO:tasks.workunit.client.1.smithi149.stdout:3/320: rename d0/d26/c60 to d0/d5/da/d5a/d63/d6a/c71 0 2024-04-24T08:24:41.706 INFO:tasks.workunit.client.1.smithi149.stdout:9/294: dread d1/d2/d34/f40 [0,4194304] 0 2024-04-24T08:24:41.708 INFO:tasks.workunit.client.1.smithi149.stdout:9/295: rmdir d1/da/d49 39 2024-04-24T08:24:41.717 INFO:tasks.workunit.client.1.smithi149.stdout:3/321: dread d0/d5/d1a/f46 [0,4194304] 0 2024-04-24T08:24:41.721 INFO:tasks.workunit.client.1.smithi149.stdout:3/322: mkdir d0/d5/da/d19/d1b/d72 0 2024-04-24T08:24:41.722 INFO:tasks.workunit.client.1.smithi149.stdout:9/296: write d1/d2/d15/d1c/f2a [1769626,63761] 0 2024-04-24T08:24:41.722 INFO:tasks.workunit.client.1.smithi149.stdout:9/297: stat d1/d4c/f54 0 2024-04-24T08:24:41.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/323: truncate d0/d5/f16 1210456 0 2024-04-24T08:24:41.725 INFO:tasks.workunit.client.1.smithi149.stdout:9/298: rmdir d1/da 39 2024-04-24T08:24:41.729 INFO:tasks.workunit.client.1.smithi149.stdout:9/299: mkdir d1/d2/d33/d5d 0 2024-04-24T08:24:41.732 INFO:tasks.workunit.client.1.smithi149.stdout:9/300: mknod d1/da/dd/c5e 0 2024-04-24T08:24:41.733 INFO:tasks.workunit.client.1.smithi149.stdout:9/301: stat d1/da/fc 0 2024-04-24T08:24:41.734 INFO:tasks.workunit.client.1.smithi149.stdout:5/329: dread db/d12/f1b [0,4194304] 0 2024-04-24T08:24:41.736 INFO:tasks.workunit.client.1.smithi149.stdout:9/302: creat d1/d2/d33/f5f x:0 0 0 2024-04-24T08:24:41.737 INFO:tasks.workunit.client.1.smithi149.stdout:5/330: symlink db/d12/l68 0 2024-04-24T08:24:41.738 INFO:tasks.workunit.client.1.smithi149.stdout:9/303: mkdir d1/d2/d33/d60 0 2024-04-24T08:24:41.740 INFO:tasks.workunit.client.1.smithi149.stdout:3/324: dwrite d0/d5/d1a/f2d [0,4194304] 0 2024-04-24T08:24:41.741 INFO:tasks.workunit.client.1.smithi149.stdout:5/331: creat db/d12/f69 x:0 0 0 2024-04-24T08:24:41.743 INFO:tasks.workunit.client.1.smithi149.stdout:3/325: chown d0/l57 3 1 2024-04-24T08:24:41.744 INFO:tasks.workunit.client.1.smithi149.stdout:5/332: chown db/d12/d2f 499 1 2024-04-24T08:24:41.746 INFO:tasks.workunit.client.1.smithi149.stdout:5/333: stat db/d12/d2f/f62 0 2024-04-24T08:24:41.747 INFO:tasks.workunit.client.1.smithi149.stdout:3/326: mkdir d0/d5/d73 0 2024-04-24T08:24:41.753 INFO:tasks.workunit.client.1.smithi149.stdout:3/327: symlink d0/d5/da/d5a/l74 0 2024-04-24T08:24:41.756 INFO:tasks.workunit.client.1.smithi149.stdout:3/328: dread - d0/d26/d3c/f65 zero size 2024-04-24T08:24:41.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/329: mknod d0/d5/da/d5e/c75 0 2024-04-24T08:24:41.763 INFO:tasks.workunit.client.1.smithi149.stdout:9/304: dwrite d1/de/f2e [0,4194304] 0 2024-04-24T08:24:41.765 INFO:tasks.workunit.client.1.smithi149.stdout:3/330: mkdir d0/d5/da/d19/d76 0 2024-04-24T08:24:41.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/331: creat d0/d5/da/d19/d1b/d61/f77 x:0 0 0 2024-04-24T08:24:41.785 INFO:tasks.workunit.client.1.smithi149.stdout:9/305: dwrite d1/de/d12/f3d [0,4194304] 0 2024-04-24T08:24:41.788 INFO:tasks.workunit.client.1.smithi149.stdout:5/334: dwrite db/d12/d60/f67 [4194304,4194304] 0 2024-04-24T08:24:41.797 INFO:tasks.workunit.client.1.smithi149.stdout:5/335: fdatasync db/d12/d22/d45/f42 0 2024-04-24T08:24:41.802 INFO:tasks.workunit.client.1.smithi149.stdout:9/306: rename d1/d4c/c5c to d1/da/d49/c61 0 2024-04-24T08:24:41.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/336: write db/d12/d1e/f4a [883260,54141] 0 2024-04-24T08:24:41.804 INFO:tasks.workunit.client.1.smithi149.stdout:3/332: dread d0/d5/f16 [0,4194304] 0 2024-04-24T08:24:41.809 INFO:tasks.workunit.client.1.smithi149.stdout:9/307: creat d1/d2/d15/d1c/f62 x:0 0 0 2024-04-24T08:24:41.810 INFO:tasks.workunit.client.1.smithi149.stdout:3/333: symlink d0/d5/da/d5a/d63/d6a/l78 0 2024-04-24T08:24:41.810 INFO:tasks.workunit.client.1.smithi149.stdout:5/337: dread db/d35/f59 [0,4194304] 0 2024-04-24T08:24:41.813 INFO:tasks.workunit.client.1.smithi149.stdout:3/334: read d0/d5/d1a/f1e [1840134,64105] 0 2024-04-24T08:24:41.817 INFO:tasks.workunit.client.1.smithi149.stdout:3/335: read - d0/d5/da/d19/d1b/d22/f41 zero size 2024-04-24T08:24:41.819 INFO:tasks.workunit.client.1.smithi149.stdout:9/308: link d1/da/d13/f45 d1/da/d49/f63 0 2024-04-24T08:24:41.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/309: creat d1/da/d13/f64 x:0 0 0 2024-04-24T08:24:41.828 INFO:tasks.workunit.client.1.smithi149.stdout:9/310: stat d1/da/c3f 0 2024-04-24T08:24:41.833 INFO:tasks.workunit.client.1.smithi149.stdout:9/311: creat d1/d2/d15/d1c/d42/f65 x:0 0 0 2024-04-24T08:24:41.838 INFO:tasks.workunit.client.1.smithi149.stdout:9/312: creat d1/d2/d33/f66 x:0 0 0 2024-04-24T08:24:41.840 INFO:tasks.workunit.client.1.smithi149.stdout:9/313: mkdir d1/da/d13/d67 0 2024-04-24T08:24:41.842 INFO:tasks.workunit.client.1.smithi149.stdout:7/277: sync 2024-04-24T08:24:41.842 INFO:tasks.workunit.client.1.smithi149.stdout:1/370: sync 2024-04-24T08:24:41.842 INFO:tasks.workunit.client.1.smithi149.stdout:8/254: sync 2024-04-24T08:24:41.842 INFO:tasks.workunit.client.1.smithi149.stdout:0/276: sync 2024-04-24T08:24:41.843 INFO:tasks.workunit.client.1.smithi149.stdout:4/325: sync 2024-04-24T08:24:41.845 INFO:tasks.workunit.client.1.smithi149.stdout:9/314: write d1/d2/d15/f58 [48840,65937] 0 2024-04-24T08:24:41.849 INFO:tasks.workunit.client.1.smithi149.stdout:0/277: chown d2/d9/d1a/f3f 9821 1 2024-04-24T08:24:41.850 INFO:tasks.workunit.client.1.smithi149.stdout:7/278: creat d9/d43/d4c/f5c x:0 0 0 2024-04-24T08:24:41.851 INFO:tasks.workunit.client.1.smithi149.stdout:3/336: dwrite d0/d5/da/fb [0,4194304] 0 2024-04-24T08:24:41.852 INFO:tasks.workunit.client.1.smithi149.stdout:8/255: chown d1/de/d18/l1e 232196 1 2024-04-24T08:24:41.856 INFO:tasks.workunit.client.1.smithi149.stdout:7/279: write d9/dc/d11/d38/f56 [2931545,69632] 0 2024-04-24T08:24:41.866 INFO:tasks.workunit.client.1.smithi149.stdout:9/315: truncate d1/da/d13/f1e 175479 0 2024-04-24T08:24:41.873 INFO:tasks.workunit.client.1.smithi149.stdout:0/278: getdents d2/d9/d1a/d36 0 2024-04-24T08:24:41.884 INFO:tasks.workunit.client.1.smithi149.stdout:9/316: getdents d1/da/d13/d67 0 2024-04-24T08:24:41.884 INFO:tasks.workunit.client.1.smithi149.stdout:1/371: dwrite d0/d5/d4f/f77 [0,4194304] 0 2024-04-24T08:24:41.889 INFO:tasks.workunit.client.1.smithi149.stdout:3/337: dread d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:41.895 INFO:tasks.workunit.client.1.smithi149.stdout:8/256: dwrite d1/d1f/d25/f42 [0,4194304] 0 2024-04-24T08:24:41.895 INFO:tasks.workunit.client.1.smithi149.stdout:7/280: dwrite d9/dc/d11/f3b [0,4194304] 0 2024-04-24T08:24:41.895 INFO:tasks.workunit.client.1.smithi149.stdout:1/372: creat d0/d5/d4f/d60/f78 x:0 0 0 2024-04-24T08:24:41.902 INFO:tasks.workunit.client.1.smithi149.stdout:4/326: dread d2/d10/d1a/d34/f37 [0,4194304] 0 2024-04-24T08:24:41.903 INFO:tasks.workunit.client.0.smithi090.stderr:++ readlink -f fsstress 2024-04-24T08:24:41.908 INFO:tasks.workunit.client.1.smithi149.stdout:4/327: write d2/d10/d1a/f36 [637888,125963] 0 2024-04-24T08:24:41.908 INFO:tasks.workunit.client.0.smithi090.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T08:24:41.908 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-04-24T08:24:41.912 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T08:24:41.912 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-04-24T08:24:41.913 INFO:tasks.workunit.client.1.smithi149.stdout:7/281: mknod d9/dc/d2e/c5d 0 2024-04-24T08:24:41.914 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-24T08:24:41.915 INFO:tasks.workunit.client.0.smithi090.stderr:++ mktemp -d -p . 2024-04-24T08:24:41.915 INFO:tasks.workunit.client.1.smithi149.stdout:3/338: write d0/d5/da/d19/d1b/d22/d38/f50 [5106459,42031] 0 2024-04-24T08:24:41.916 INFO:tasks.workunit.client.1.smithi149.stdout:9/317: dread d1/d2/f2b [0,4194304] 0 2024-04-24T08:24:41.922 INFO:tasks.workunit.client.1.smithi149.stdout:9/318: write d1/da/f50 [791354,50376] 0 2024-04-24T08:24:41.924 INFO:tasks.workunit.client.0.smithi090.stderr:+ T=./tmp.hsCCAW66Ry 2024-04-24T08:24:41.924 INFO:tasks.workunit.client.0.smithi090.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.hsCCAW66Ry -l 1 -n 1000 -p 10 -v 2024-04-24T08:24:41.925 INFO:tasks.workunit.client.1.smithi149.stdout:1/373: dwrite d0/d5/f41 [8388608,4194304] 0 2024-04-24T08:24:41.928 INFO:tasks.workunit.client.1.smithi149.stdout:4/328: link d2/d10/d1a/f6f d2/d24/d2e/d38/f71 0 2024-04-24T08:24:41.930 INFO:tasks.workunit.client.1.smithi149.stdout:7/282: mknod d9/dc/d11/d38/c5e 0 2024-04-24T08:24:41.932 INFO:tasks.workunit.client.1.smithi149.stdout:7/283: write d9/dc/de/f31 [121799,45067] 0 2024-04-24T08:24:41.936 INFO:tasks.workunit.client.1.smithi149.stdout:4/329: unlink d2/l12 0 2024-04-24T08:24:41.940 INFO:tasks.workunit.client.1.smithi149.stdout:3/339: truncate d0/d5/f6f 2838487 0 2024-04-24T08:24:41.941 INFO:tasks.workunit.client.0.smithi090.stdout:seed = 1713806699 2024-04-24T08:24:41.941 INFO:tasks.workunit.client.1.smithi149.stdout:4/330: write d2/d24/d2e/f66 [340614,19045] 0 2024-04-24T08:24:41.946 INFO:tasks.workunit.client.1.smithi149.stdout:1/374: mkdir d0/d5/db/d18/d4d/d50/d79 0 2024-04-24T08:24:41.946 INFO:tasks.workunit.client.1.smithi149.stdout:0/279: dread d2/d17/f23 [0,4194304] 0 2024-04-24T08:24:41.949 INFO:tasks.workunit.client.1.smithi149.stdout:9/319: dwrite d1/da/dd/f24 [0,4194304] 0 2024-04-24T08:24:41.950 INFO:tasks.workunit.client.1.smithi149.stdout:8/257: getdents d1/de/d18 0 2024-04-24T08:24:41.951 INFO:tasks.workunit.client.1.smithi149.stdout:7/284: symlink d9/d43/l5f 0 2024-04-24T08:24:41.954 INFO:tasks.workunit.client.1.smithi149.stdout:1/375: mknod d0/d39/d53/d61/c7a 0 2024-04-24T08:24:41.954 INFO:tasks.workunit.client.1.smithi149.stdout:9/320: readlink d1/da/d13/l1f 0 2024-04-24T08:24:41.956 INFO:tasks.workunit.client.0.smithi090.stdout:2/0: dwrite - no filename 2024-04-24T08:24:41.956 INFO:tasks.workunit.client.0.smithi090.stdout:2/1: write - no filename 2024-04-24T08:24:41.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/2: dwrite - no filename 2024-04-24T08:24:41.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/285: rmdir d9/dc/de 39 2024-04-24T08:24:41.958 INFO:tasks.workunit.client.0.smithi090.stdout:0/0: creat f0 x:0 0 0 2024-04-24T08:24:41.959 INFO:tasks.workunit.client.1.smithi149.stdout:0/280: write d2/d9/df/f16 [4999805,100115] 0 2024-04-24T08:24:41.960 INFO:tasks.workunit.client.0.smithi090.stdout:1/0: creat f0 x:0 0 0 2024-04-24T08:24:41.963 INFO:tasks.workunit.client.1.smithi149.stdout:8/258: creat d1/de/d18/d43/f55 x:0 0 0 2024-04-24T08:24:41.963 INFO:tasks.workunit.client.0.smithi090.stdout:3/0: write - no filename 2024-04-24T08:24:41.963 INFO:tasks.workunit.client.0.smithi090.stdout:3/1: write - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/2: dwrite - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/3: rename - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/4: dwrite - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/5: write - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/6: write - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/7: dread - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/8: rename - no filename 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/9: chown . 6443 1 2024-04-24T08:24:41.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/10: chown . 2528 1 2024-04-24T08:24:41.967 INFO:tasks.workunit.client.1.smithi149.stdout:1/376: chown d0/f5f 3 1 2024-04-24T08:24:41.967 INFO:tasks.workunit.client.0.smithi090.stdout:0/1: rename f0 to f1 0 2024-04-24T08:24:41.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/1: mknod c1 0 2024-04-24T08:24:41.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/259: mkdir d1/de/d18/d32/d56 0 2024-04-24T08:24:41.969 INFO:tasks.workunit.client.0.smithi090.stdout:1/2: truncate f0 293736 0 2024-04-24T08:24:41.969 INFO:tasks.workunit.client.0.smithi090.stdout:1/3: stat f0 0 2024-04-24T08:24:41.970 INFO:tasks.workunit.client.1.smithi149.stdout:3/340: dwrite d0/d5/da/d19/f48 [0,4194304] 0 2024-04-24T08:24:41.970 INFO:tasks.workunit.client.1.smithi149.stdout:9/321: symlink d1/d2/d33/d5d/l68 0 2024-04-24T08:24:41.970 INFO:tasks.workunit.client.0.smithi090.stdout:4/0: fdatasync - no filename 2024-04-24T08:24:41.970 INFO:tasks.workunit.client.0.smithi090.stdout:4/1: dread - no filename 2024-04-24T08:24:41.971 INFO:tasks.workunit.client.0.smithi090.stdout:4/2: dread - no filename 2024-04-24T08:24:41.971 INFO:tasks.workunit.client.0.smithi090.stdout:4/3: fsync - no filename 2024-04-24T08:24:41.971 INFO:tasks.workunit.client.0.smithi090.stdout:4/4: unlink - no file 2024-04-24T08:24:41.971 INFO:tasks.workunit.client.0.smithi090.stdout:1/4: write f0 [1295341,87951] 0 2024-04-24T08:24:41.972 INFO:tasks.workunit.client.1.smithi149.stdout:8/260: chown d1/d1f/d31/f35 112 1 2024-04-24T08:24:41.974 INFO:tasks.workunit.client.1.smithi149.stdout:0/281: rename d2/d17/c1e to d2/d9/df/d48/c55 0 2024-04-24T08:24:41.974 INFO:tasks.workunit.client.0.smithi090.stdout:2/3: mkdir d0 0 2024-04-24T08:24:41.976 INFO:tasks.workunit.client.1.smithi149.stdout:7/286: mknod d9/d43/d4f/c60 0 2024-04-24T08:24:41.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/2: creat f2 x:0 0 0 2024-04-24T08:24:41.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/3: write f2 [486638,52375] 0 2024-04-24T08:24:41.983 INFO:tasks.workunit.client.1.smithi149.stdout:9/322: fdatasync d1/d2/d15/f2f 0 2024-04-24T08:24:41.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/4: chown f2 23043 1 2024-04-24T08:24:41.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/5: dread - f1 zero size 2024-04-24T08:24:41.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/11: getdents . 0 2024-04-24T08:24:41.987 INFO:tasks.workunit.client.1.smithi149.stdout:3/341: unlink d0/d5/da/c3e 0 2024-04-24T08:24:41.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/0: write - no filename 2024-04-24T08:24:41.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/1: write - no filename 2024-04-24T08:24:41.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/2: rename - no filename 2024-04-24T08:24:41.990 INFO:tasks.workunit.client.0.smithi090.stdout:5/3: chown . 30856 1 2024-04-24T08:24:41.990 INFO:tasks.workunit.client.0.smithi090.stdout:5/4: dwrite - no filename 2024-04-24T08:24:41.990 INFO:tasks.workunit.client.0.smithi090.stdout:5/5: truncate - no filename 2024-04-24T08:24:41.992 INFO:tasks.workunit.client.1.smithi149.stdout:8/261: rename d1/d1f/d22/l23 to d1/d1f/d22/l57 0 2024-04-24T08:24:41.992 INFO:tasks.workunit.client.1.smithi149.stdout:8/262: fsync d1/f4f 0 2024-04-24T08:24:41.993 INFO:tasks.workunit.client.1.smithi149.stdout:8/263: chown d1/de/d18/l26 17 1 2024-04-24T08:24:41.994 INFO:tasks.workunit.client.1.smithi149.stdout:5/338: sync 2024-04-24T08:24:41.994 INFO:tasks.workunit.client.1.smithi149.stdout:2/233: sync 2024-04-24T08:24:41.994 INFO:tasks.workunit.client.1.smithi149.stdout:6/227: sync 2024-04-24T08:24:41.995 INFO:tasks.workunit.client.1.smithi149.stdout:8/264: write d1/d1f/f28 [5428356,1952] 0 2024-04-24T08:24:42.000 INFO:tasks.workunit.client.1.smithi149.stdout:7/287: creat d9/dc/d11/f61 x:0 0 0 2024-04-24T08:24:42.005 INFO:tasks.workunit.client.1.smithi149.stdout:4/331: dread d2/d18/f64 [0,4194304] 0 2024-04-24T08:24:42.011 INFO:tasks.workunit.client.0.smithi090.stdout:6/0: dread - no filename 2024-04-24T08:24:42.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/234: rename d0 to d0/d6/d49 22 2024-04-24T08:24:42.015 INFO:tasks.workunit.client.0.smithi090.stdout:0/6: unlink f1 0 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/5: getdents . 0 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/6: readlink - no filename 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/7: dread - no filename 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/8: readlink - no filename 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/9: rename - no filename 2024-04-24T08:24:42.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/228: mknod d0/d1a/d28/d31/d33/c48 0 2024-04-24T08:24:42.018 INFO:tasks.workunit.client.1.smithi149.stdout:8/265: symlink d1/d1f/d25/l58 0 2024-04-24T08:24:42.020 INFO:tasks.workunit.client.0.smithi090.stdout:2/4: mkdir d0/d1 0 2024-04-24T08:24:42.022 INFO:tasks.workunit.client.0.smithi090.stdout:3/12: creat f0 x:0 0 0 2024-04-24T08:24:42.025 INFO:tasks.workunit.client.1.smithi149.stdout:9/323: link d1/d2/d34/f40 d1/da/dd/f69 0 2024-04-24T08:24:42.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/0: readlink - no filename 2024-04-24T08:24:42.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/1: dread - no filename 2024-04-24T08:24:42.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/2: write - no filename 2024-04-24T08:24:42.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/3: dwrite - no filename 2024-04-24T08:24:42.028 INFO:tasks.workunit.client.0.smithi090.stdout:5/6: creat f0 x:0 0 0 2024-04-24T08:24:42.034 INFO:tasks.workunit.client.1.smithi149.stdout:4/332: mknod d2/d18/d1e/c72 0 2024-04-24T08:24:42.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/229: dwrite d0/f1 [4194304,4194304] 0 2024-04-24T08:24:42.036 INFO:tasks.workunit.client.1.smithi149.stdout:4/333: chown d2/d10/d1a/d34/f62 25396189 1 2024-04-24T08:24:42.038 INFO:tasks.workunit.client.0.smithi090.stdout:0/7: dwrite f2 [0,4194304] 0 2024-04-24T08:24:42.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/13: mknod c1 0 2024-04-24T08:24:42.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/14: dread - f0 zero size 2024-04-24T08:24:42.045 INFO:tasks.workunit.client.0.smithi090.stdout:3/15: dread - f0 zero size 2024-04-24T08:24:42.050 INFO:tasks.workunit.client.1.smithi149.stdout:9/324: dwrite d1/d2/f41 [4194304,4194304] 0 2024-04-24T08:24:42.058 INFO:tasks.workunit.client.1.smithi149.stdout:1/377: link d0/d5/db/d18/f29 d0/d1b/d2a/f7b 0 2024-04-24T08:24:42.060 INFO:tasks.workunit.client.1.smithi149.stdout:0/282: dwrite d2/d9/d1a/f44 [0,4194304] 0 2024-04-24T08:24:42.069 INFO:tasks.workunit.client.1.smithi149.stdout:3/342: truncate d0/d5/d53/d34/f35 3418056 0 2024-04-24T08:24:42.069 INFO:tasks.workunit.client.1.smithi149.stdout:0/283: write d2/d17/f24 [2170252,39679] 0 2024-04-24T08:24:42.074 INFO:tasks.workunit.client.1.smithi149.stdout:0/284: chown d2/d9/df/d48/f52 2016601088 1 2024-04-24T08:24:42.074 INFO:tasks.workunit.client.1.smithi149.stdout:3/343: dread - d0/d5/da/d19/d1b/d61/f77 zero size 2024-04-24T08:24:42.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/230: symlink d0/d1a/d28/d31/d33/l49 0 2024-04-24T08:24:42.078 INFO:tasks.workunit.client.1.smithi149.stdout:1/378: creat d0/d39/f7c x:0 0 0 2024-04-24T08:24:42.079 INFO:tasks.workunit.client.1.smithi149.stdout:3/344: readlink d0/d5/da/d19/l2e 0 2024-04-24T08:24:42.079 INFO:tasks.workunit.client.1.smithi149.stdout:4/334: fsync d2/d24/f39 0 2024-04-24T08:24:42.080 INFO:tasks.workunit.client.1.smithi149.stdout:2/235: creat d0/d6/db/f4a x:0 0 0 2024-04-24T08:24:42.089 INFO:tasks.workunit.client.1.smithi149.stdout:2/236: write d0/d6/db/d27/f47 [804423,49611] 0 2024-04-24T08:24:42.089 INFO:tasks.workunit.client.1.smithi149.stdout:9/325: fsync d1/da/d13/f1e 0 2024-04-24T08:24:42.090 INFO:tasks.workunit.client.1.smithi149.stdout:5/339: dwrite db/f16 [0,4194304] 0 2024-04-24T08:24:42.095 INFO:tasks.workunit.client.1.smithi149.stdout:9/326: chown d1/d2/d33/d60 640548036 1 2024-04-24T08:24:42.097 INFO:tasks.workunit.client.1.smithi149.stdout:3/345: mkdir d0/d5/da/d59/d79 0 2024-04-24T08:24:42.099 INFO:tasks.workunit.client.1.smithi149.stdout:4/335: symlink d2/d24/d2e/d38/l73 0 2024-04-24T08:24:42.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/288: write d9/dc/de/f12 [4233850,75375] 0 2024-04-24T08:24:42.103 INFO:tasks.workunit.client.1.smithi149.stdout:1/379: rename d0/d5/df/l3b to d0/d5/db/d18/d34/d51/l7d 0 2024-04-24T08:24:42.103 INFO:tasks.workunit.client.1.smithi149.stdout:7/289: chown d9/dc/f17 6 1 2024-04-24T08:24:42.105 INFO:tasks.workunit.client.1.smithi149.stdout:2/237: symlink d0/d6/db/l4b 0 2024-04-24T08:24:42.106 INFO:tasks.workunit.client.1.smithi149.stdout:0/285: dread d2/f8 [0,4194304] 0 2024-04-24T08:24:42.106 INFO:tasks.workunit.client.1.smithi149.stdout:4/336: symlink d2/d18/l74 0 2024-04-24T08:24:42.109 INFO:tasks.workunit.client.1.smithi149.stdout:1/380: dread - d0/da/f48 zero size 2024-04-24T08:24:42.114 INFO:tasks.workunit.client.1.smithi149.stdout:3/346: truncate d0/d5/da/d19/d1b/d61/f77 687348 0 2024-04-24T08:24:42.114 INFO:tasks.workunit.client.1.smithi149.stdout:8/266: truncate d1/de/d18/f48 2952712 0 2024-04-24T08:24:42.114 INFO:tasks.workunit.client.1.smithi149.stdout:4/337: dread d2/d18/d1e/f33 [0,4194304] 0 2024-04-24T08:24:42.114 INFO:tasks.workunit.client.1.smithi149.stdout:5/340: rename db/d35/c5f to db/d12/d2f/c6a 0 2024-04-24T08:24:42.115 INFO:tasks.workunit.client.1.smithi149.stdout:3/347: write d0/d5/d1a/f5d [535065,103158] 0 2024-04-24T08:24:42.117 INFO:tasks.workunit.client.1.smithi149.stdout:7/290: symlink d9/d43/l62 0 2024-04-24T08:24:42.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/341: dread - db/d12/d1e/f4f zero size 2024-04-24T08:24:42.120 INFO:tasks.workunit.client.1.smithi149.stdout:9/327: dread d1/da/fc [0,4194304] 0 2024-04-24T08:24:42.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/238: rename d0/d6/db/f17 to d0/d6/db/d39/d3d/d44/f4c 0 2024-04-24T08:24:42.125 INFO:tasks.workunit.client.1.smithi149.stdout:2/239: read d0/f8 [968439,82793] 0 2024-04-24T08:24:42.126 INFO:tasks.workunit.client.1.smithi149.stdout:6/231: write d0/f12 [1290936,88617] 0 2024-04-24T08:24:42.127 INFO:tasks.workunit.client.1.smithi149.stdout:0/286: mkdir d2/d9/df/d56 0 2024-04-24T08:24:42.127 INFO:tasks.workunit.client.1.smithi149.stdout:6/232: fsync d0/f1f 0 2024-04-24T08:24:42.130 INFO:tasks.workunit.client.1.smithi149.stdout:5/342: symlink db/d12/d60/l6b 0 2024-04-24T08:24:42.133 INFO:tasks.workunit.client.1.smithi149.stdout:7/291: dread d9/dc/f18 [0,4194304] 0 2024-04-24T08:24:42.134 INFO:tasks.workunit.client.1.smithi149.stdout:1/381: mknod d0/d1b/d5d/c7e 0 2024-04-24T08:24:42.136 INFO:tasks.workunit.client.1.smithi149.stdout:9/328: creat d1/de/f6a x:0 0 0 2024-04-24T08:24:42.138 INFO:tasks.workunit.client.1.smithi149.stdout:3/348: rename d0/d5/da/fb to d0/d26/d3c/f7a 0 2024-04-24T08:24:42.141 INFO:tasks.workunit.client.1.smithi149.stdout:3/349: read d0/d5/d53/d34/d3a/f33 [2228347,27210] 0 2024-04-24T08:24:42.144 INFO:tasks.workunit.client.1.smithi149.stdout:0/287: mkdir d2/d3c/d57 0 2024-04-24T08:24:42.145 INFO:tasks.workunit.client.1.smithi149.stdout:0/288: readlink d2/d17/l22 0 2024-04-24T08:24:42.145 INFO:tasks.workunit.client.1.smithi149.stdout:6/233: stat d0/d1a/c1d 0 2024-04-24T08:24:42.147 INFO:tasks.workunit.client.1.smithi149.stdout:7/292: symlink d9/dc/d2e/d3c/l63 0 2024-04-24T08:24:42.148 INFO:tasks.workunit.client.1.smithi149.stdout:1/382: dread d0/d5/d4f/f77 [0,4194304] 0 2024-04-24T08:24:42.150 INFO:tasks.workunit.client.1.smithi149.stdout:9/329: symlink d1/de/d12/l6b 0 2024-04-24T08:24:42.152 INFO:tasks.workunit.client.1.smithi149.stdout:1/383: write d0/d5/db/d18/d4d/f62 [917465,28730] 0 2024-04-24T08:24:42.154 INFO:tasks.workunit.client.1.smithi149.stdout:4/338: link d2/c2b d2/d10/d1a/c75 0 2024-04-24T08:24:42.154 INFO:tasks.workunit.client.1.smithi149.stdout:3/350: rename d0/d5/d1a/f46 to d0/d5/da/d5a/d63/d6a/f7b 0 2024-04-24T08:24:42.156 INFO:tasks.workunit.client.1.smithi149.stdout:8/267: getdents d1/de 0 2024-04-24T08:24:42.156 INFO:tasks.workunit.client.1.smithi149.stdout:1/384: chown d0/d1b/d5d/c7e 111713 1 2024-04-24T08:24:42.157 INFO:tasks.workunit.client.1.smithi149.stdout:4/339: write d2/d24/d2e/f51 [1938180,97483] 0 2024-04-24T08:24:42.158 INFO:tasks.workunit.client.1.smithi149.stdout:1/385: dread - d0/f25 zero size 2024-04-24T08:24:42.162 INFO:tasks.workunit.client.1.smithi149.stdout:7/293: mkdir d9/d43/d64 0 2024-04-24T08:24:42.164 INFO:tasks.workunit.client.1.smithi149.stdout:9/330: creat d1/da/dd/f6c x:0 0 0 2024-04-24T08:24:42.166 INFO:tasks.workunit.client.1.smithi149.stdout:9/331: chown d1/da/dd/f6c 843049 1 2024-04-24T08:24:42.167 INFO:tasks.workunit.client.1.smithi149.stdout:2/240: creat d0/f4d x:0 0 0 2024-04-24T08:24:42.170 INFO:tasks.workunit.client.0.smithi090.stdout:5/7: mknod c1 0 2024-04-24T08:24:42.170 INFO:tasks.workunit.client.1.smithi149.stdout:3/351: write d0/f11 [4749575,31858] 0 2024-04-24T08:24:42.174 INFO:tasks.workunit.client.1.smithi149.stdout:0/289: symlink d2/d3c/d57/l58 0 2024-04-24T08:24:42.174 INFO:tasks.workunit.client.1.smithi149.stdout:1/386: mknod d0/d1b/d2a/c7f 0 2024-04-24T08:24:42.179 INFO:tasks.workunit.client.1.smithi149.stdout:7/294: write d9/d43/d4c/f52 [285718,41727] 0 2024-04-24T08:24:42.181 INFO:tasks.workunit.client.1.smithi149.stdout:2/241: fsync d0/d6/db/f10 0 2024-04-24T08:24:42.181 INFO:tasks.workunit.client.1.smithi149.stdout:4/340: dwrite d2/d18/d1e/d3c/d48/d69/f70 [0,4194304] 0 2024-04-24T08:24:42.183 INFO:tasks.workunit.client.1.smithi149.stdout:6/234: dread d0/fe [4194304,4194304] 0 2024-04-24T08:24:42.187 INFO:tasks.workunit.client.1.smithi149.stdout:7/295: write d9/dc/d11/f37 [172766,40107] 0 2024-04-24T08:24:42.187 INFO:tasks.workunit.client.1.smithi149.stdout:5/343: read db/d12/d22/f33 [7663205,128766] 0 2024-04-24T08:24:42.188 INFO:tasks.workunit.client.1.smithi149.stdout:1/387: mknod d0/d5/db/d18/d4d/d50/c80 0 2024-04-24T08:24:42.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/341: fdatasync d2/d18/f64 0 2024-04-24T08:24:42.194 INFO:tasks.workunit.client.1.smithi149.stdout:4/342: chown d2/f6c 49591992 1 2024-04-24T08:24:42.197 INFO:tasks.workunit.client.1.smithi149.stdout:6/235: mkdir d0/d1a/d28/d4a 0 2024-04-24T08:24:42.206 INFO:tasks.workunit.client.1.smithi149.stdout:0/290: creat d2/d9/df/f59 x:0 0 0 2024-04-24T08:24:42.211 INFO:tasks.workunit.client.1.smithi149.stdout:6/236: unlink d0/f1f 0 2024-04-24T08:24:42.215 INFO:tasks.workunit.client.1.smithi149.stdout:7/296: truncate d9/dc/d2e/f39 518818 0 2024-04-24T08:24:42.218 INFO:tasks.workunit.client.1.smithi149.stdout:0/291: dread d2/d17/f23 [0,4194304] 0 2024-04-24T08:24:42.220 INFO:tasks.workunit.client.1.smithi149.stdout:4/343: mknod d2/c76 0 2024-04-24T08:24:42.223 INFO:tasks.workunit.client.1.smithi149.stdout:6/237: creat d0/d1a/d28/d31/f4b x:0 0 0 2024-04-24T08:24:42.224 INFO:tasks.workunit.client.1.smithi149.stdout:3/352: dread d0/d5/ff [0,4194304] 0 2024-04-24T08:24:42.225 INFO:tasks.workunit.client.1.smithi149.stdout:7/297: unlink d9/f2c 0 2024-04-24T08:24:42.227 INFO:tasks.workunit.client.1.smithi149.stdout:1/388: rename d0/d5/db/d18/d4d/d50/c80 to d0/c81 0 2024-04-24T08:24:42.228 INFO:tasks.workunit.client.1.smithi149.stdout:1/389: fdatasync d0/f25 0 2024-04-24T08:24:42.229 INFO:tasks.workunit.client.1.smithi149.stdout:0/292: creat d2/d9/f5a x:0 0 0 2024-04-24T08:24:42.231 INFO:tasks.workunit.client.1.smithi149.stdout:4/344: unlink d2/d18/l32 0 2024-04-24T08:24:42.232 INFO:tasks.workunit.client.1.smithi149.stdout:0/293: write d2/d17/f24 [841047,96995] 0 2024-04-24T08:24:42.233 INFO:tasks.workunit.client.1.smithi149.stdout:4/345: chown d2/d10/d1a/d34 80559 1 2024-04-24T08:24:42.234 INFO:tasks.workunit.client.1.smithi149.stdout:6/238: unlink d0/d1a/d28/d31/c44 0 2024-04-24T08:24:42.238 INFO:tasks.workunit.client.1.smithi149.stdout:4/346: creat d2/d18/d1e/d3c/d48/d69/f77 x:0 0 0 2024-04-24T08:24:42.243 INFO:tasks.workunit.client.1.smithi149.stdout:6/239: mkdir d0/d1a/d28/d31/d4c 0 2024-04-24T08:24:42.243 INFO:tasks.workunit.client.1.smithi149.stdout:7/298: dread d9/dc/de/f22 [0,4194304] 0 2024-04-24T08:24:42.245 INFO:tasks.workunit.client.1.smithi149.stdout:4/347: rename d2/d18/d1e/d3c/d48/f5f to d2/d18/d1e/d3b/f78 0 2024-04-24T08:24:42.246 INFO:tasks.workunit.client.1.smithi149.stdout:6/240: read d0/d1a/f29 [3233919,68548] 0 2024-04-24T08:24:42.246 INFO:tasks.workunit.client.1.smithi149.stdout:7/299: write d9/dc/d11/d38/f4e [1699657,72253] 0 2024-04-24T08:24:42.246 INFO:tasks.workunit.client.1.smithi149.stdout:3/353: dread d0/d5/d1a/f1e [0,4194304] 0 2024-04-24T08:24:42.247 INFO:tasks.workunit.client.1.smithi149.stdout:4/348: stat d2/d18/d1e/f15 0 2024-04-24T08:24:42.248 INFO:tasks.workunit.client.1.smithi149.stdout:3/354: chown d0/d5/d73 2918362 1 2024-04-24T08:24:42.250 INFO:tasks.workunit.client.1.smithi149.stdout:3/355: chown d0/l57 88095 1 2024-04-24T08:24:42.255 INFO:tasks.workunit.client.1.smithi149.stdout:6/241: creat d0/d1a/f4d x:0 0 0 2024-04-24T08:24:42.256 INFO:tasks.workunit.client.1.smithi149.stdout:7/300: truncate d9/dc/de/f3a 483858 0 2024-04-24T08:24:42.261 INFO:tasks.workunit.client.1.smithi149.stdout:7/301: unlink d9/c3e 0 2024-04-24T08:24:42.264 INFO:tasks.workunit.client.1.smithi149.stdout:7/302: symlink d9/dc/d11/l65 0 2024-04-24T08:24:42.265 INFO:tasks.workunit.client.1.smithi149.stdout:7/303: readlink d9/d43/l5f 0 2024-04-24T08:24:42.269 INFO:tasks.workunit.client.1.smithi149.stdout:6/242: dwrite d0/f16 [0,4194304] 0 2024-04-24T08:24:42.271 INFO:tasks.workunit.client.1.smithi149.stdout:6/243: fsync d0/f10 0 2024-04-24T08:24:42.279 INFO:tasks.workunit.client.1.smithi149.stdout:6/244: mkdir d0/d1a/d28/d31/d4c/d4e 0 2024-04-24T08:24:42.283 INFO:tasks.workunit.client.1.smithi149.stdout:7/304: dwrite d9/dc/f1b [0,4194304] 0 2024-04-24T08:24:42.284 INFO:tasks.workunit.client.1.smithi149.stdout:6/245: mknod d0/d1a/d28/d31/d4c/c4f 0 2024-04-24T08:24:42.286 INFO:tasks.workunit.client.1.smithi149.stdout:7/305: mkdir d9/d43/d4f/d66 0 2024-04-24T08:24:42.287 INFO:tasks.workunit.client.1.smithi149.stdout:6/246: symlink d0/d1a/d28/d31/d4c/l50 0 2024-04-24T08:24:42.288 INFO:tasks.workunit.client.1.smithi149.stdout:7/306: write d9/dc/d11/d26/f28 [559831,23307] 0 2024-04-24T08:24:42.288 INFO:tasks.workunit.client.1.smithi149.stdout:6/247: write d0/f10 [642250,59153] 0 2024-04-24T08:24:42.291 INFO:tasks.workunit.client.1.smithi149.stdout:7/307: mkdir d9/d43/d4c/d67 0 2024-04-24T08:24:42.293 INFO:tasks.workunit.client.1.smithi149.stdout:7/308: symlink d9/dc/d2e/d51/l68 0 2024-04-24T08:24:42.295 INFO:tasks.workunit.client.1.smithi149.stdout:7/309: truncate d9/dc/d11/f37 490099 0 2024-04-24T08:24:42.297 INFO:tasks.workunit.client.1.smithi149.stdout:7/310: write f1 [2672901,52204] 0 2024-04-24T08:24:42.300 INFO:tasks.workunit.client.1.smithi149.stdout:7/311: mkdir d9/d43/d4c/d67/d69 0 2024-04-24T08:24:42.303 INFO:tasks.workunit.client.1.smithi149.stdout:6/248: dwrite d0/d1a/d28/d31/f37 [0,4194304] 0 2024-04-24T08:24:42.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/249: fsync d0/d1a/f27 0 2024-04-24T08:24:42.317 INFO:tasks.workunit.client.1.smithi149.stdout:7/312: dwrite d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:42.319 INFO:tasks.workunit.client.1.smithi149.stdout:7/313: symlink d9/dc/d11/l6a 0 2024-04-24T08:24:42.325 INFO:tasks.workunit.client.1.smithi149.stdout:7/314: chown d9/dc/d2e/f39 5825 1 2024-04-24T08:24:42.327 INFO:tasks.workunit.client.1.smithi149.stdout:7/315: write d9/dc/d11/f42 [3658220,124919] 0 2024-04-24T08:24:42.331 INFO:tasks.workunit.client.1.smithi149.stdout:7/316: fdatasync d9/dc/d2e/f39 0 2024-04-24T08:24:42.336 INFO:tasks.workunit.client.1.smithi149.stdout:7/317: creat d9/dc/f6b x:0 0 0 2024-04-24T08:24:42.341 INFO:tasks.workunit.client.1.smithi149.stdout:5/344: dread db/d12/f5b [0,4194304] 0 2024-04-24T08:24:42.344 INFO:tasks.workunit.client.0.smithi090.stdout:6/1: creat f0 x:0 0 0 2024-04-24T08:24:42.344 INFO:tasks.workunit.client.0.smithi090.stdout:3/16: mkdir d2 0 2024-04-24T08:24:42.344 INFO:tasks.workunit.client.1.smithi149.stdout:5/345: fdatasync db/d12/d2f/f46 0 2024-04-24T08:24:42.345 INFO:tasks.workunit.client.1.smithi149.stdout:5/346: dread - db/d12/f69 zero size 2024-04-24T08:24:42.347 INFO:tasks.workunit.client.0.smithi090.stdout:5/8: truncate f0 933267 0 2024-04-24T08:24:42.347 INFO:tasks.workunit.client.0.smithi090.stdout:1/5: getdents . 0 2024-04-24T08:24:42.348 INFO:tasks.workunit.client.0.smithi090.stdout:3/17: chown f0 1702 1 2024-04-24T08:24:42.356 INFO:tasks.workunit.client.0.smithi090.stdout:7/4: creat f0 x:0 0 0 2024-04-24T08:24:42.363 INFO:tasks.workunit.client.1.smithi149.stdout:8/268: write d1/de/f38 [566,56074] 0 2024-04-24T08:24:42.365 INFO:tasks.workunit.client.0.smithi090.stdout:2/5: rmdir d0/d1 0 2024-04-24T08:24:42.367 INFO:tasks.workunit.client.1.smithi149.stdout:5/347: dwrite db/f16 [0,4194304] 0 2024-04-24T08:24:42.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/5: truncate f0 837745 0 2024-04-24T08:24:42.369 INFO:tasks.workunit.client.0.smithi090.stdout:6/2: mknod c1 0 2024-04-24T08:24:42.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/0: rmdir - no directory 2024-04-24T08:24:42.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/1: rename - no filename 2024-04-24T08:24:42.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/2: stat - no entries 2024-04-24T08:24:42.372 INFO:tasks.workunit.client.0.smithi090.stdout:9/3: fdatasync - no filename 2024-04-24T08:24:42.372 INFO:tasks.workunit.client.0.smithi090.stdout:8/0: symlink l0 0 2024-04-24T08:24:42.374 INFO:tasks.workunit.client.1.smithi149.stdout:5/348: mkdir db/d12/d22/d6c 0 2024-04-24T08:24:42.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/390: dread d0/fe [0,4194304] 0 2024-04-24T08:24:42.383 INFO:tasks.workunit.client.0.smithi090.stdout:7/6: rename f0 to f1 0 2024-04-24T08:24:42.383 INFO:tasks.workunit.client.0.smithi090.stdout:2/6: creat d0/f2 x:0 0 0 2024-04-24T08:24:42.385 INFO:tasks.workunit.client.1.smithi149.stdout:2/242: truncate d0/d6/db/f1a 2128839 0 2024-04-24T08:24:42.388 INFO:tasks.workunit.client.0.smithi090.stdout:8/1: symlink l1 0 2024-04-24T08:24:42.388 INFO:tasks.workunit.client.0.smithi090.stdout:8/2: dread - no filename 2024-04-24T08:24:42.388 INFO:tasks.workunit.client.0.smithi090.stdout:7/7: stat f1 0 2024-04-24T08:24:42.388 INFO:tasks.workunit.client.0.smithi090.stdout:1/6: dread f0 [0,4194304] 0 2024-04-24T08:24:42.390 INFO:tasks.workunit.client.0.smithi090.stdout:6/3: rename c1 to c2 0 2024-04-24T08:24:42.392 INFO:tasks.workunit.client.1.smithi149.stdout:1/391: rename d0/d5/db/d18/d4d/d50/d79 to d0/d5/db/d82 0 2024-04-24T08:24:42.393 INFO:tasks.workunit.client.1.smithi149.stdout:9/332: dread d1/d4c/f54 [0,4194304] 0 2024-04-24T08:24:42.394 INFO:tasks.workunit.client.1.smithi149.stdout:1/392: chown d0/f16 144955 1 2024-04-24T08:24:42.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/8: dwrite f2 [0,4194304] 0 2024-04-24T08:24:42.395 INFO:tasks.workunit.client.1.smithi149.stdout:2/243: rmdir d0/d6/db/d39/d3d 39 2024-04-24T08:24:42.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/8: symlink l2 0 2024-04-24T08:24:42.396 INFO:tasks.workunit.client.0.smithi090.stdout:2/7: write d0/f2 [318232,30240] 0 2024-04-24T08:24:42.400 INFO:tasks.workunit.client.0.smithi090.stdout:8/3: mkdir d2 0 2024-04-24T08:24:42.401 INFO:tasks.workunit.client.1.smithi149.stdout:7/318: dread d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:42.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/269: dread d1/de/d18/f48 [0,4194304] 0 2024-04-24T08:24:42.401 INFO:tasks.workunit.client.1.smithi149.stdout:9/333: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:42.402 INFO:tasks.workunit.client.1.smithi149.stdout:8/270: fsync d1/d1f/f52 0 2024-04-24T08:24:42.406 INFO:tasks.workunit.client.1.smithi149.stdout:1/393: read d0/d5/db/f1e [1242079,47709] 0 2024-04-24T08:24:42.406 INFO:tasks.workunit.client.0.smithi090.stdout:1/7: mknod c2 0 2024-04-24T08:24:42.407 INFO:tasks.workunit.client.0.smithi090.stdout:3/18: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.408 INFO:tasks.workunit.client.0.smithi090.stdout:7/9: creat f3 x:0 0 0 2024-04-24T08:24:42.409 INFO:tasks.workunit.client.0.smithi090.stdout:5/9: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.413 INFO:tasks.workunit.client.0.smithi090.stdout:7/10: chown f1 1210321 1 2024-04-24T08:24:42.416 INFO:tasks.workunit.client.1.smithi149.stdout:9/334: mkdir d1/d4c/d6d 0 2024-04-24T08:24:42.417 INFO:tasks.workunit.client.0.smithi090.stdout:2/8: creat d0/f3 x:0 0 0 2024-04-24T08:24:42.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/11: write f3 [725353,39438] 0 2024-04-24T08:24:42.428 INFO:tasks.workunit.client.0.smithi090.stdout:6/4: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.431 INFO:tasks.workunit.client.1.smithi149.stdout:1/394: truncate d0/d1b/d2a/f7b 1588184 0 2024-04-24T08:24:42.432 INFO:tasks.workunit.client.0.smithi090.stdout:0/9: dwrite f2 [0,4194304] 0 2024-04-24T08:24:42.433 INFO:tasks.workunit.client.0.smithi090.stdout:5/10: creat f2 x:0 0 0 2024-04-24T08:24:42.433 INFO:tasks.workunit.client.1.smithi149.stdout:1/395: readlink d0/d5/df/l15 0 2024-04-24T08:24:42.435 INFO:tasks.workunit.client.1.smithi149.stdout:9/335: symlink d1/d2/d15/l6e 0 2024-04-24T08:24:42.439 INFO:tasks.workunit.client.1.smithi149.stdout:0/294: truncate d2/d9/df/f21 3446080 0 2024-04-24T08:24:42.439 INFO:tasks.workunit.client.0.smithi090.stdout:5/11: chown c1 101113703 1 2024-04-24T08:24:42.439 INFO:tasks.workunit.client.1.smithi149.stdout:2/244: creat d0/d6/db/d39/f4e x:0 0 0 2024-04-24T08:24:42.442 INFO:tasks.workunit.client.0.smithi090.stdout:8/4: creat d2/f3 x:0 0 0 2024-04-24T08:24:42.442 INFO:tasks.workunit.client.1.smithi149.stdout:2/245: stat d0/d6/db/d28/l33 0 2024-04-24T08:24:42.443 INFO:tasks.workunit.client.0.smithi090.stdout:5/12: chown c1 12283 1 2024-04-24T08:24:42.446 INFO:tasks.workunit.client.1.smithi149.stdout:1/396: rmdir d0/da 39 2024-04-24T08:24:42.447 INFO:tasks.workunit.client.0.smithi090.stdout:6/5: creat f3 x:0 0 0 2024-04-24T08:24:42.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/13: truncate f2 560143 0 2024-04-24T08:24:42.451 INFO:tasks.workunit.client.0.smithi090.stdout:0/10: rename f2 to f3 0 2024-04-24T08:24:42.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/19: fdatasync f0 0 2024-04-24T08:24:42.457 INFO:tasks.workunit.client.1.smithi149.stdout:9/336: rmdir d1/da/d13 39 2024-04-24T08:24:42.457 INFO:tasks.workunit.client.0.smithi090.stdout:0/11: read f3 [3586598,6155] 0 2024-04-24T08:24:42.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/246: mkdir d0/d6/db/d27/d4f 0 2024-04-24T08:24:42.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/9: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:42.459 INFO:tasks.workunit.client.0.smithi090.stdout:5/14: write f2 [865691,17178] 0 2024-04-24T08:24:42.459 INFO:tasks.workunit.client.1.smithi149.stdout:0/295: dwrite d2/d9/df/d48/f4c [0,4194304] 0 2024-04-24T08:24:42.461 INFO:tasks.workunit.client.1.smithi149.stdout:6/250: getdents d0/d1a 0 2024-04-24T08:24:42.462 INFO:tasks.workunit.client.0.smithi090.stdout:7/12: dwrite f3 [0,4194304] 0 2024-04-24T08:24:42.464 INFO:tasks.workunit.client.0.smithi090.stdout:7/13: readlink l2 0 2024-04-24T08:24:42.465 INFO:tasks.workunit.client.1.smithi149.stdout:0/296: truncate d2/d9/df/d2f/f43 435941 0 2024-04-24T08:24:42.468 INFO:tasks.workunit.client.1.smithi149.stdout:9/337: creat d1/d4c/f6f x:0 0 0 2024-04-24T08:24:42.473 INFO:tasks.workunit.client.1.smithi149.stdout:3/356: write d0/d5/da/d5a/d63/d6a/f7b [1675115,109513] 0 2024-04-24T08:24:42.477 INFO:tasks.workunit.client.1.smithi149.stdout:3/357: write d0/d5/da/f6e [427246,101991] 0 2024-04-24T08:24:42.479 INFO:tasks.workunit.client.1.smithi149.stdout:3/358: dread - d0/d5/d53/f3f zero size 2024-04-24T08:24:42.480 INFO:tasks.workunit.client.0.smithi090.stdout:0/12: mknod c4 0 2024-04-24T08:24:42.482 INFO:tasks.workunit.client.0.smithi090.stdout:8/5: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:42.485 INFO:tasks.workunit.client.1.smithi149.stdout:0/297: mknod d2/d9/df/d40/c5b 0 2024-04-24T08:24:42.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/15: dread f0 [0,4194304] 0 2024-04-24T08:24:42.489 INFO:tasks.workunit.client.1.smithi149.stdout:9/338: rename d1/d4c/f6f to d1/d2/d34/f70 0 2024-04-24T08:24:42.489 INFO:tasks.workunit.client.1.smithi149.stdout:6/251: mknod d0/c51 0 2024-04-24T08:24:42.491 INFO:tasks.workunit.client.0.smithi090.stdout:7/14: creat f4 x:0 0 0 2024-04-24T08:24:42.493 INFO:tasks.workunit.client.1.smithi149.stdout:6/252: read d0/f3f [37908,57881] 0 2024-04-24T08:24:42.497 INFO:tasks.workunit.client.1.smithi149.stdout:0/298: mknod d2/d9/df/d40/c5c 0 2024-04-24T08:24:42.498 INFO:tasks.workunit.client.0.smithi090.stdout:3/20: creat d2/f3 x:0 0 0 2024-04-24T08:24:42.498 INFO:tasks.workunit.client.0.smithi090.stdout:8/6: symlink d2/l4 0 2024-04-24T08:24:42.499 INFO:tasks.workunit.client.0.smithi090.stdout:2/10: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:42.499 INFO:tasks.workunit.client.1.smithi149.stdout:9/339: fdatasync d1/d2/f2b 0 2024-04-24T08:24:42.501 INFO:tasks.workunit.client.1.smithi149.stdout:9/340: chown d1/d2/d15/l6e 63 1 2024-04-24T08:24:42.501 INFO:tasks.workunit.client.1.smithi149.stdout:4/349: sync 2024-04-24T08:24:42.502 INFO:tasks.workunit.client.0.smithi090.stdout:0/13: dread f3 [0,4194304] 0 2024-04-24T08:24:42.502 INFO:tasks.workunit.client.0.smithi090.stdout:2/11: write d0/f2 [661209,117492] 0 2024-04-24T08:24:42.503 INFO:tasks.workunit.client.1.smithi149.stdout:4/350: fdatasync d2/d18/d1e/d3c/f43 0 2024-04-24T08:24:42.504 INFO:tasks.workunit.client.0.smithi090.stdout:5/16: creat f3 x:0 0 0 2024-04-24T08:24:42.504 INFO:tasks.workunit.client.0.smithi090.stdout:2/12: stat d0/f3 0 2024-04-24T08:24:42.506 INFO:tasks.workunit.client.0.smithi090.stdout:7/15: symlink l5 0 2024-04-24T08:24:42.510 INFO:tasks.workunit.client.1.smithi149.stdout:6/253: mknod d0/d1a/c52 0 2024-04-24T08:24:42.510 INFO:tasks.workunit.client.1.smithi149.stdout:6/254: stat d0/fa 0 2024-04-24T08:24:42.515 INFO:tasks.workunit.client.0.smithi090.stdout:0/14: rename c4 to c5 0 2024-04-24T08:24:42.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/341: creat d1/de/f71 x:0 0 0 2024-04-24T08:24:42.522 INFO:tasks.workunit.client.0.smithi090.stdout:7/16: creat f6 x:0 0 0 2024-04-24T08:24:42.522 INFO:tasks.workunit.client.0.smithi090.stdout:7/17: stat f3 0 2024-04-24T08:24:42.522 INFO:tasks.workunit.client.0.smithi090.stdout:7/18: dread - f4 zero size 2024-04-24T08:24:42.526 INFO:tasks.workunit.client.0.smithi090.stdout:5/17: link f3 f4 0 2024-04-24T08:24:42.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:42 smithi149 ceph-mon[31282]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:24:42.527 INFO:tasks.workunit.client.1.smithi149.stdout:0/299: creat d2/d9/df/f5d x:0 0 0 2024-04-24T08:24:42.527 INFO:tasks.workunit.client.0.smithi090.stdout:5/18: chown f0 91 1 2024-04-24T08:24:42.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/19: dread - f4 zero size 2024-04-24T08:24:42.528 INFO:tasks.workunit.client.1.smithi149.stdout:0/300: chown d2/d9/df/d40/d4a 39632088 1 2024-04-24T08:24:42.529 INFO:tasks.workunit.client.1.smithi149.stdout:9/342: mkdir d1/da/dd/d55/d72 0 2024-04-24T08:24:42.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/19: fdatasync f1 0 2024-04-24T08:24:42.531 INFO:tasks.workunit.client.1.smithi149.stdout:9/343: write d1/d2/d33/f4e [569635,84770] 0 2024-04-24T08:24:42.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/20: mknod c5 0 2024-04-24T08:24:42.535 INFO:tasks.workunit.client.1.smithi149.stdout:9/344: mkdir d1/d2/d15/d73 0 2024-04-24T08:24:42.540 INFO:tasks.workunit.client.0.smithi090.stdout:5/21: unlink c1 0 2024-04-24T08:24:42.542 INFO:tasks.workunit.client.1.smithi149.stdout:7/319: getdents d9/d43/d4c 0 2024-04-24T08:24:42.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:42 smithi090 ceph-mon[25016]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-24T08:24:42.543 INFO:tasks.workunit.client.1.smithi149.stdout:4/351: dread d2/f9 [0,4194304] 0 2024-04-24T08:24:42.544 INFO:tasks.workunit.client.1.smithi149.stdout:0/301: dwrite d2/d9/df/d48/f4f [0,4194304] 0 2024-04-24T08:24:42.544 INFO:tasks.workunit.client.1.smithi149.stdout:7/320: chown d9/dc/de/l45 0 1 2024-04-24T08:24:42.547 INFO:tasks.workunit.client.1.smithi149.stdout:0/302: dread - d2/d9/d1a/f3f zero size 2024-04-24T08:24:42.548 INFO:tasks.workunit.client.0.smithi090.stdout:7/20: dwrite f4 [0,4194304] 0 2024-04-24T08:24:42.549 INFO:tasks.workunit.client.1.smithi149.stdout:4/352: dread d2/d24/d2e/d3f/f40 [0,4194304] 0 2024-04-24T08:24:42.549 INFO:tasks.workunit.client.1.smithi149.stdout:7/321: truncate d9/dc/de/f31 622091 0 2024-04-24T08:24:42.550 INFO:tasks.workunit.client.1.smithi149.stdout:0/303: fdatasync d2/d9/df/d2f/f30 0 2024-04-24T08:24:42.553 INFO:tasks.workunit.client.0.smithi090.stdout:7/21: creat f7 x:0 0 0 2024-04-24T08:24:42.563 INFO:tasks.workunit.client.1.smithi149.stdout:5/349: write db/d12/f5b [996705,18393] 0 2024-04-24T08:24:42.565 INFO:tasks.workunit.client.1.smithi149.stdout:5/350: chown db/c15 15362139 1 2024-04-24T08:24:42.570 INFO:tasks.workunit.client.1.smithi149.stdout:5/351: write db/d12/d22/d45/d5d/f63 [299332,57634] 0 2024-04-24T08:24:42.571 INFO:tasks.workunit.client.0.smithi090.stdout:1/8: getdents . 0 2024-04-24T08:24:42.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/353: dwrite d2/d24/d2e/d3f/f4b [4194304,4194304] 0 2024-04-24T08:24:42.576 INFO:tasks.workunit.client.0.smithi090.stdout:1/9: mkdir d3 0 2024-04-24T08:24:42.577 INFO:tasks.workunit.client.0.smithi090.stdout:5/22: dwrite f3 [0,4194304] 0 2024-04-24T08:24:42.579 INFO:tasks.workunit.client.0.smithi090.stdout:5/23: chown f0 19495968 1 2024-04-24T08:24:42.584 INFO:tasks.workunit.client.1.smithi149.stdout:8/271: dwrite d1/d1f/f50 [0,4194304] 0 2024-04-24T08:24:42.589 INFO:tasks.workunit.client.0.smithi090.stdout:7/22: dwrite f1 [0,4194304] 0 2024-04-24T08:24:42.590 INFO:tasks.workunit.client.0.smithi090.stdout:6/6: fsync f3 0 2024-04-24T08:24:42.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/15: rename f3 to f6 0 2024-04-24T08:24:42.597 INFO:tasks.workunit.client.0.smithi090.stdout:1/10: creat d3/f4 x:0 0 0 2024-04-24T08:24:42.598 INFO:tasks.workunit.client.1.smithi149.stdout:2/247: rmdir d0 39 2024-04-24T08:24:42.601 INFO:tasks.workunit.client.0.smithi090.stdout:5/24: rename f0 to f6 0 2024-04-24T08:24:42.603 INFO:tasks.workunit.client.0.smithi090.stdout:5/25: truncate f6 4711887 0 2024-04-24T08:24:42.604 INFO:tasks.workunit.client.1.smithi149.stdout:0/304: dread d2/d17/f24 [0,4194304] 0 2024-04-24T08:24:42.605 INFO:tasks.workunit.client.0.smithi090.stdout:7/23: unlink f1 0 2024-04-24T08:24:42.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/352: creat db/d12/d22/d6c/f6d x:0 0 0 2024-04-24T08:24:42.609 INFO:tasks.workunit.client.0.smithi090.stdout:6/7: unlink f3 0 2024-04-24T08:24:42.611 INFO:tasks.workunit.client.1.smithi149.stdout:8/272: dwrite d1/de/f1a [0,4194304] 0 2024-04-24T08:24:42.616 INFO:tasks.workunit.client.0.smithi090.stdout:5/26: dread f6 [0,4194304] 0 2024-04-24T08:24:42.617 INFO:tasks.workunit.client.0.smithi090.stdout:1/11: mkdir d3/d5 0 2024-04-24T08:24:42.622 INFO:tasks.workunit.client.1.smithi149.stdout:0/305: rmdir d2/d9/df/d2f 39 2024-04-24T08:24:42.625 INFO:tasks.workunit.client.1.smithi149.stdout:5/353: creat db/d12/d22/d6c/f6e x:0 0 0 2024-04-24T08:24:42.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/354: read - db/d35/f4c zero size 2024-04-24T08:24:42.627 INFO:tasks.workunit.client.1.smithi149.stdout:3/359: write d0/d5/da/d5e/f30 [1776236,98981] 0 2024-04-24T08:24:42.628 INFO:tasks.workunit.client.0.smithi090.stdout:7/24: symlink l8 0 2024-04-24T08:24:42.630 INFO:tasks.workunit.client.0.smithi090.stdout:6/8: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.630 INFO:tasks.workunit.client.0.smithi090.stdout:6/9: rmdir - no directory 2024-04-24T08:24:42.631 INFO:tasks.workunit.client.1.smithi149.stdout:3/360: truncate d0/d5/d1a/f5d 939008 0 2024-04-24T08:24:42.637 INFO:tasks.workunit.client.1.smithi149.stdout:4/354: getdents d2 0 2024-04-24T08:24:42.639 INFO:tasks.workunit.client.0.smithi090.stdout:8/7: truncate d2/f3 786438 0 2024-04-24T08:24:42.640 INFO:tasks.workunit.client.1.smithi149.stdout:4/355: chown d2/d10/d1a/d34/f37 11 1 2024-04-24T08:24:42.640 INFO:tasks.workunit.client.0.smithi090.stdout:5/27: creat f7 x:0 0 0 2024-04-24T08:24:42.641 INFO:tasks.workunit.client.1.smithi149.stdout:4/356: fdatasync d2/d10/f23 0 2024-04-24T08:24:42.642 INFO:tasks.workunit.client.0.smithi090.stdout:2/13: truncate d0/f3 830035 0 2024-04-24T08:24:42.642 INFO:tasks.workunit.client.0.smithi090.stdout:7/25: creat f9 x:0 0 0 2024-04-24T08:24:42.644 INFO:tasks.workunit.client.0.smithi090.stdout:6/10: creat f4 x:0 0 0 2024-04-24T08:24:42.645 INFO:tasks.workunit.client.1.smithi149.stdout:4/357: fdatasync d2/d10/d1a/f36 0 2024-04-24T08:24:42.646 INFO:tasks.workunit.client.0.smithi090.stdout:1/12: symlink d3/d5/l6 0 2024-04-24T08:24:42.648 INFO:tasks.workunit.client.0.smithi090.stdout:5/28: mkdir d8 0 2024-04-24T08:24:42.655 INFO:tasks.workunit.client.0.smithi090.stdout:5/29: dread f2 [0,4194304] 0 2024-04-24T08:24:42.656 INFO:tasks.workunit.client.1.smithi149.stdout:4/358: rename d2/d18/d1e/f15 to d2/d18/d1e/d3b/f79 0 2024-04-24T08:24:42.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/30: dread - f7 zero size 2024-04-24T08:24:42.658 INFO:tasks.workunit.client.0.smithi090.stdout:6/11: fsync f0 0 2024-04-24T08:24:42.659 INFO:tasks.workunit.client.1.smithi149.stdout:7/322: truncate d9/dc/d11/d26/f28 503458 0 2024-04-24T08:24:42.660 INFO:tasks.workunit.client.1.smithi149.stdout:2/248: link d0/c14 d0/d6/db/d27/c50 0 2024-04-24T08:24:42.661 INFO:tasks.workunit.client.1.smithi149.stdout:9/345: write d1/d2/f48 [2747,81246] 0 2024-04-24T08:24:42.663 INFO:tasks.workunit.client.1.smithi149.stdout:4/359: creat d2/d24/d2e/d38/f7a x:0 0 0 2024-04-24T08:24:42.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/346: write d1/d4c/f54 [1235520,58045] 0 2024-04-24T08:24:42.665 INFO:tasks.workunit.client.0.smithi090.stdout:7/26: dwrite f7 [0,4194304] 0 2024-04-24T08:24:42.666 INFO:tasks.workunit.client.1.smithi149.stdout:7/323: write d9/f5b [4796320,92951] 0 2024-04-24T08:24:42.668 INFO:tasks.workunit.client.0.smithi090.stdout:7/27: write f6 [617538,43720] 0 2024-04-24T08:24:42.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/16: rename f6 to f7 0 2024-04-24T08:24:42.672 INFO:tasks.workunit.client.0.smithi090.stdout:6/12: rename f4 to f5 0 2024-04-24T08:24:42.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/31: link f7 d8/f9 0 2024-04-24T08:24:42.677 INFO:tasks.workunit.client.0.smithi090.stdout:1/13: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.678 INFO:tasks.workunit.client.1.smithi149.stdout:1/397: dwrite d0/d5/db/d18/f29 [0,4194304] 0 2024-04-24T08:24:42.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/14: rename d3/d5 to d3/d5/d7 22 2024-04-24T08:24:42.681 INFO:tasks.workunit.client.1.smithi149.stdout:1/398: fsync d0/d5/d4f/d60/f78 0 2024-04-24T08:24:42.686 INFO:tasks.workunit.client.1.smithi149.stdout:9/347: write d1/da/f27 [2414478,77703] 0 2024-04-24T08:24:42.688 INFO:tasks.workunit.client.0.smithi090.stdout:7/28: creat fa x:0 0 0 2024-04-24T08:24:42.689 INFO:tasks.workunit.client.0.smithi090.stdout:7/29: dread - f9 zero size 2024-04-24T08:24:42.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/324: dread d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:42.690 INFO:tasks.workunit.client.0.smithi090.stdout:7/30: stat f4 0 2024-04-24T08:24:42.691 INFO:tasks.workunit.client.0.smithi090.stdout:0/17: write f7 [994202,113393] 0 2024-04-24T08:24:42.693 INFO:tasks.workunit.client.1.smithi149.stdout:4/360: dwrite d2/d24/d2e/d38/f5e [0,4194304] 0 2024-04-24T08:24:42.695 INFO:tasks.workunit.client.1.smithi149.stdout:7/325: write d9/dc/de/f31 [1585387,11213] 0 2024-04-24T08:24:42.697 INFO:tasks.workunit.client.1.smithi149.stdout:8/273: write d1/de/d18/f45 [1907436,33457] 0 2024-04-24T08:24:42.699 INFO:tasks.workunit.client.0.smithi090.stdout:6/13: write f5 [529104,47226] 0 2024-04-24T08:24:42.699 INFO:tasks.workunit.client.1.smithi149.stdout:1/399: creat d0/d5/db/d63/f83 x:0 0 0 2024-04-24T08:24:42.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/348: rmdir d1/d2 39 2024-04-24T08:24:42.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/8: write d2/f3 [1430948,73564] 0 2024-04-24T08:24:42.716 INFO:tasks.workunit.client.1.smithi149.stdout:5/355: dwrite db/d12/d22/d45/f42 [0,4194304] 0 2024-04-24T08:24:42.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/349: fdatasync d1/d4c/f54 0 2024-04-24T08:24:42.720 INFO:tasks.workunit.client.0.smithi090.stdout:2/14: rename d0/f3 to d0/f4 0 2024-04-24T08:24:42.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/31: mkdir db 0 2024-04-24T08:24:42.723 INFO:tasks.workunit.client.1.smithi149.stdout:3/361: write d0/d5/f6f [1852422,129465] 0 2024-04-24T08:24:42.723 INFO:tasks.workunit.client.0.smithi090.stdout:0/18: dwrite f7 [0,4194304] 0 2024-04-24T08:24:42.726 INFO:tasks.workunit.client.1.smithi149.stdout:6/255: sync 2024-04-24T08:24:42.726 INFO:tasks.workunit.client.1.smithi149.stdout:2/249: dread d0/d6/db/d28/f2c [0,4194304] 0 2024-04-24T08:24:42.727 INFO:tasks.workunit.client.0.smithi090.stdout:1/15: link d3/f4 d3/f8 0 2024-04-24T08:24:42.727 INFO:tasks.workunit.client.1.smithi149.stdout:7/326: creat d9/dc/d11/f6c x:0 0 0 2024-04-24T08:24:42.727 INFO:tasks.workunit.client.1.smithi149.stdout:8/274: symlink d1/de/d18/l59 0 2024-04-24T08:24:42.728 INFO:tasks.workunit.client.1.smithi149.stdout:5/356: write db/d12/d2f/f46 [1612651,19807] 0 2024-04-24T08:24:42.728 INFO:tasks.workunit.client.1.smithi149.stdout:6/256: write d0/fa [1071124,32402] 0 2024-04-24T08:24:42.730 INFO:tasks.workunit.client.0.smithi090.stdout:1/16: write f0 [3014861,80044] 0 2024-04-24T08:24:42.730 INFO:tasks.workunit.client.0.smithi090.stdout:6/14: dwrite f0 [0,4194304] 0 2024-04-24T08:24:42.730 INFO:tasks.workunit.client.1.smithi149.stdout:4/361: dwrite d2/d10/f31 [0,4194304] 0 2024-04-24T08:24:42.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/275: fsync d1/d1f/f2c 0 2024-04-24T08:24:42.742 INFO:tasks.workunit.client.1.smithi149.stdout:9/350: mkdir d1/da/d13/d74 0 2024-04-24T08:24:42.746 INFO:tasks.workunit.client.0.smithi090.stdout:0/19: rename f7 to f8 0 2024-04-24T08:24:42.751 INFO:tasks.workunit.client.0.smithi090.stdout:1/17: dread f0 [0,4194304] 0 2024-04-24T08:24:42.753 INFO:tasks.workunit.client.0.smithi090.stdout:2/15: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:42.759 INFO:tasks.workunit.client.1.smithi149.stdout:9/351: dread d1/de/f2e [0,4194304] 0 2024-04-24T08:24:42.762 INFO:tasks.workunit.client.1.smithi149.stdout:4/362: creat d2/d18/d1e/d3b/f7b x:0 0 0 2024-04-24T08:24:42.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/15: rename f5 to f6 0 2024-04-24T08:24:42.764 INFO:tasks.workunit.client.1.smithi149.stdout:0/306: dread d2/f8 [0,4194304] 0 2024-04-24T08:24:42.764 INFO:tasks.workunit.client.1.smithi149.stdout:2/250: dwrite d0/d6/db/f15 [0,4194304] 0 2024-04-24T08:24:42.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/362: rename d0/d26/d3c/f7a to d0/d5/da/d5a/d63/d6a/f7c 0 2024-04-24T08:24:42.771 INFO:tasks.workunit.client.1.smithi149.stdout:8/276: rename d1/d1f/d31/l51 to d1/d1f/d31/l5a 0 2024-04-24T08:24:42.773 INFO:tasks.workunit.client.0.smithi090.stdout:7/32: creat db/fc x:0 0 0 2024-04-24T08:24:42.775 INFO:tasks.workunit.client.0.smithi090.stdout:7/33: stat db/fc 0 2024-04-24T08:24:42.776 INFO:tasks.workunit.client.0.smithi090.stdout:0/20: symlink l9 0 2024-04-24T08:24:42.777 INFO:tasks.workunit.client.1.smithi149.stdout:0/307: dread d2/d3c/f51 [0,4194304] 0 2024-04-24T08:24:42.778 INFO:tasks.workunit.client.0.smithi090.stdout:1/18: mknod d3/c9 0 2024-04-24T08:24:42.779 INFO:tasks.workunit.client.1.smithi149.stdout:7/327: getdents d9/d43/d4f/d66 0 2024-04-24T08:24:42.780 INFO:tasks.workunit.client.0.smithi090.stdout:0/21: chown f8 218 1 2024-04-24T08:24:42.782 INFO:tasks.workunit.client.1.smithi149.stdout:9/352: rmdir d1/da/dd/d55 39 2024-04-24T08:24:42.782 INFO:tasks.workunit.client.0.smithi090.stdout:5/32: dwrite f6 [0,4194304] 0 2024-04-24T08:24:42.789 INFO:tasks.workunit.client.0.smithi090.stdout:6/16: chown c2 5 1 2024-04-24T08:24:42.789 INFO:tasks.workunit.client.0.smithi090.stdout:6/17: rmdir - no directory 2024-04-24T08:24:42.789 INFO:tasks.workunit.client.1.smithi149.stdout:2/251: fsync d0/f2d 0 2024-04-24T08:24:42.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/19: mkdir d3/d5/da 0 2024-04-24T08:24:42.794 INFO:tasks.workunit.client.1.smithi149.stdout:3/363: mknod d0/d26/c7d 0 2024-04-24T08:24:42.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/22: dread f8 [0,4194304] 0 2024-04-24T08:24:42.795 INFO:tasks.workunit.client.0.smithi090.stdout:7/34: dread f7 [0,4194304] 0 2024-04-24T08:24:42.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/252: dread d0/f8 [0,4194304] 0 2024-04-24T08:24:42.800 INFO:tasks.workunit.client.1.smithi149.stdout:0/308: dwrite d2/d9/df/d48/f4c [0,4194304] 0 2024-04-24T08:24:42.801 INFO:tasks.workunit.client.0.smithi090.stdout:6/18: dread f0 [0,4194304] 0 2024-04-24T08:24:42.813 INFO:tasks.workunit.client.0.smithi090.stdout:1/20: fsync d3/f8 0 2024-04-24T08:24:42.815 INFO:tasks.workunit.client.0.smithi090.stdout:0/23: chown c5 0 1 2024-04-24T08:24:42.817 INFO:tasks.workunit.client.0.smithi090.stdout:5/33: mkdir d8/da 0 2024-04-24T08:24:42.818 INFO:tasks.workunit.client.0.smithi090.stdout:5/34: chown d8 110 1 2024-04-24T08:24:42.819 INFO:tasks.workunit.client.1.smithi149.stdout:4/363: write d2/d18/d1e/f33 [110867,46087] 0 2024-04-24T08:24:42.819 INFO:tasks.workunit.client.1.smithi149.stdout:2/253: dwrite d0/d6/db/d28/f38 [4194304,4194304] 0 2024-04-24T08:24:42.821 INFO:tasks.workunit.client.0.smithi090.stdout:6/19: symlink l7 0 2024-04-24T08:24:42.824 INFO:tasks.workunit.client.1.smithi149.stdout:2/254: write d0/d6/db/f15 [4746205,29102] 0 2024-04-24T08:24:42.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/309: dwrite d2/d9/f2d [0,4194304] 0 2024-04-24T08:24:42.831 INFO:tasks.workunit.client.1.smithi149.stdout:6/257: truncate d0/f16 3913632 0 2024-04-24T08:24:42.831 INFO:tasks.workunit.client.1.smithi149.stdout:3/364: creat d0/d5/da/d19/d1b/d22/d38/f7e x:0 0 0 2024-04-24T08:24:42.836 INFO:tasks.workunit.client.1.smithi149.stdout:3/365: readlink d0/d5/d53/d34/d3a/l31 0 2024-04-24T08:24:42.837 INFO:tasks.workunit.client.1.smithi149.stdout:9/353: mknod d1/da/dd/c75 0 2024-04-24T08:24:42.839 INFO:tasks.workunit.client.0.smithi090.stdout:5/35: symlink d8/lb 0 2024-04-24T08:24:42.840 INFO:tasks.workunit.client.1.smithi149.stdout:3/366: chown d0/d5/da/d19/d1b/d22/d38/f50 48769376 1 2024-04-24T08:24:42.845 INFO:tasks.workunit.client.1.smithi149.stdout:8/277: rmdir d1/de/d18/d32/d56 0 2024-04-24T08:24:42.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/20: mkdir d8 0 2024-04-24T08:24:42.847 INFO:tasks.workunit.client.0.smithi090.stdout:2/16: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:42.848 INFO:tasks.workunit.client.1.smithi149.stdout:8/278: read - d1/f4f zero size 2024-04-24T08:24:42.848 INFO:tasks.workunit.client.1.smithi149.stdout:4/364: dwrite d2/d18/f1b [4194304,4194304] 0 2024-04-24T08:24:42.850 INFO:tasks.workunit.client.1.smithi149.stdout:0/310: unlink d2/d3c/f49 0 2024-04-24T08:24:42.851 INFO:tasks.workunit.client.0.smithi090.stdout:2/17: stat d0/f4 0 2024-04-24T08:24:42.852 INFO:tasks.workunit.client.1.smithi149.stdout:4/365: write d2/d24/d2e/d3f/f4b [4297751,24235] 0 2024-04-24T08:24:42.852 INFO:tasks.workunit.client.1.smithi149.stdout:0/311: readlink d2/l7 0 2024-04-24T08:24:42.852 INFO:tasks.workunit.client.1.smithi149.stdout:2/255: dwrite d0/f2d [0,4194304] 0 2024-04-24T08:24:42.857 INFO:tasks.workunit.client.1.smithi149.stdout:0/312: readlink d2/d9/d1a/d36/l3b 0 2024-04-24T08:24:42.860 INFO:tasks.workunit.client.0.smithi090.stdout:2/18: write d0/f2 [1427060,16539] 0 2024-04-24T08:24:42.863 INFO:tasks.workunit.client.0.smithi090.stdout:6/21: unlink c2 0 2024-04-24T08:24:42.867 INFO:tasks.workunit.client.1.smithi149.stdout:3/367: mknod d0/d26/d3c/c7f 0 2024-04-24T08:24:42.870 INFO:tasks.workunit.client.0.smithi090.stdout:1/21: dwrite d3/f4 [0,4194304] 0 2024-04-24T08:24:42.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/24: dwrite f8 [0,4194304] 0 2024-04-24T08:24:42.875 INFO:tasks.workunit.client.0.smithi090.stdout:7/35: getdents db 0 2024-04-24T08:24:42.875 INFO:tasks.workunit.client.0.smithi090.stdout:5/36: dread f6 [0,4194304] 0 2024-04-24T08:24:42.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/279: symlink d1/de/l5b 0 2024-04-24T08:24:42.879 INFO:tasks.workunit.client.0.smithi090.stdout:5/37: chown f3 25345 1 2024-04-24T08:24:42.880 INFO:tasks.workunit.client.0.smithi090.stdout:2/19: unlink d0/f2 0 2024-04-24T08:24:42.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/22: write f6 [376121,102841] 0 2024-04-24T08:24:42.883 INFO:tasks.workunit.client.1.smithi149.stdout:7/328: getdents d9/dc/de/d44 0 2024-04-24T08:24:42.886 INFO:tasks.workunit.client.1.smithi149.stdout:1/400: sync 2024-04-24T08:24:42.889 INFO:tasks.workunit.client.0.smithi090.stdout:1/22: dread d3/f4 [0,4194304] 0 2024-04-24T08:24:42.889 INFO:tasks.workunit.client.0.smithi090.stdout:7/36: rename f6 to db/fd 0 2024-04-24T08:24:42.897 INFO:tasks.workunit.client.1.smithi149.stdout:9/354: rename d1/d2/d15/d73 to d1/d2/d33/d76 0 2024-04-24T08:24:42.898 INFO:tasks.workunit.client.1.smithi149.stdout:3/368: mkdir d0/d5/da/d19/d80 0 2024-04-24T08:24:42.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/20: rename d0/f4 to d0/f5 0 2024-04-24T08:24:42.904 INFO:tasks.workunit.client.0.smithi090.stdout:7/37: dwrite f3 [0,4194304] 0 2024-04-24T08:24:42.906 INFO:tasks.workunit.client.1.smithi149.stdout:4/366: link d2/d18/d1e/d3c/f52 d2/d18/d1e/d3b/f7c 0 2024-04-24T08:24:42.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/38: rename db to db/de 22 2024-04-24T08:24:42.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/39: readlink l2 0 2024-04-24T08:24:42.907 INFO:tasks.workunit.client.1.smithi149.stdout:4/367: read - d2/d10/d1a/d34/f62 zero size 2024-04-24T08:24:42.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/40: dread - db/fc zero size 2024-04-24T08:24:42.908 INFO:tasks.workunit.client.1.smithi149.stdout:7/329: mknod d9/dc/d2e/d3c/c6d 0 2024-04-24T08:24:42.911 INFO:tasks.workunit.client.1.smithi149.stdout:6/258: mknod d0/d1a/d28/d31/d4c/d4e/c53 0 2024-04-24T08:24:42.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/38: creat d8/da/fc x:0 0 0 2024-04-24T08:24:42.912 INFO:tasks.workunit.client.1.smithi149.stdout:6/259: chown d0/fc 3 1 2024-04-24T08:24:42.912 INFO:tasks.workunit.client.0.smithi090.stdout:5/39: chown c5 29 1 2024-04-24T08:24:42.914 INFO:tasks.workunit.client.1.smithi149.stdout:6/260: write d0/fa [8753150,59400] 0 2024-04-24T08:24:42.915 INFO:tasks.workunit.client.0.smithi090.stdout:6/23: mknod d8/c9 0 2024-04-24T08:24:42.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/25: fdatasync f8 0 2024-04-24T08:24:42.920 INFO:tasks.workunit.client.0.smithi090.stdout:7/41: creat db/ff x:0 0 0 2024-04-24T08:24:42.920 INFO:tasks.workunit.client.1.smithi149.stdout:3/369: creat d0/d5/d53/d34/f81 x:0 0 0 2024-04-24T08:24:42.921 INFO:tasks.workunit.client.0.smithi090.stdout:6/24: fsync f0 0 2024-04-24T08:24:42.922 INFO:tasks.workunit.client.1.smithi149.stdout:0/313: dwrite d2/d3c/f51 [0,4194304] 0 2024-04-24T08:24:42.925 INFO:tasks.workunit.client.0.smithi090.stdout:7/42: creat db/f10 x:0 0 0 2024-04-24T08:24:42.927 INFO:tasks.workunit.client.0.smithi090.stdout:1/23: dwrite d3/f4 [0,4194304] 0 2024-04-24T08:24:42.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/26: link c5 ca 0 2024-04-24T08:24:42.934 INFO:tasks.workunit.client.0.smithi090.stdout:5/40: write f7 [753931,112534] 0 2024-04-24T08:24:42.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/25: dread f6 [0,4194304] 0 2024-04-24T08:24:42.937 INFO:tasks.workunit.client.0.smithi090.stdout:5/41: stat d8/f9 0 2024-04-24T08:24:42.939 INFO:tasks.workunit.client.0.smithi090.stdout:6/26: readlink l7 0 2024-04-24T08:24:42.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/21: dwrite d0/f5 [4194304,4194304] 0 2024-04-24T08:24:42.947 INFO:tasks.workunit.client.0.smithi090.stdout:0/27: getdents . 0 2024-04-24T08:24:42.952 INFO:tasks.workunit.client.0.smithi090.stdout:6/27: dread f6 [0,4194304] 0 2024-04-24T08:24:42.952 INFO:tasks.workunit.client.0.smithi090.stdout:7/43: dread f4 [0,4194304] 0 2024-04-24T08:24:42.954 INFO:tasks.workunit.client.0.smithi090.stdout:1/24: truncate d3/f4 470269 0 2024-04-24T08:24:42.954 INFO:tasks.workunit.client.0.smithi090.stdout:0/28: rename l9 to lb 0 2024-04-24T08:24:42.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/28: creat d8/fa x:0 0 0 2024-04-24T08:24:42.957 INFO:tasks.workunit.client.0.smithi090.stdout:6/29: chown f6 10 1 2024-04-24T08:24:42.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/44: creat db/f11 x:0 0 0 2024-04-24T08:24:42.960 INFO:tasks.workunit.client.0.smithi090.stdout:6/30: truncate f6 1406062 0 2024-04-24T08:24:42.962 INFO:tasks.workunit.client.0.smithi090.stdout:7/45: creat db/f12 x:0 0 0 2024-04-24T08:24:42.964 INFO:tasks.workunit.client.1.smithi149.stdout:2/256: rmdir d0/d6/db/d27/d4f 0 2024-04-24T08:24:42.966 INFO:tasks.workunit.client.1.smithi149.stdout:7/330: rmdir d9/d43 39 2024-04-24T08:24:42.966 INFO:tasks.workunit.client.0.smithi090.stdout:1/25: getdents d3 0 2024-04-24T08:24:42.967 INFO:tasks.workunit.client.1.smithi149.stdout:8/280: link d1/l10 d1/d1f/d22/l5c 0 2024-04-24T08:24:42.967 INFO:tasks.workunit.client.0.smithi090.stdout:0/29: link c5 cc 0 2024-04-24T08:24:42.968 INFO:tasks.workunit.client.1.smithi149.stdout:9/355: link d1/de/d12/f3d d1/d2/d33/d60/f77 0 2024-04-24T08:24:42.968 INFO:tasks.workunit.client.1.smithi149.stdout:6/261: write d0/d1a/f2e [533540,120647] 0 2024-04-24T08:24:42.969 INFO:tasks.workunit.client.0.smithi090.stdout:5/42: dwrite f7 [0,4194304] 0 2024-04-24T08:24:42.974 INFO:tasks.workunit.client.0.smithi090.stdout:6/31: dread f6 [0,4194304] 0 2024-04-24T08:24:42.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/26: rmdir d3/d5 39 2024-04-24T08:24:42.979 INFO:tasks.workunit.client.0.smithi090.stdout:2/22: dread d0/f5 [4194304,4194304] 0 2024-04-24T08:24:42.981 INFO:tasks.workunit.client.1.smithi149.stdout:3/370: mknod d0/d5/d1a/d70/c82 0 2024-04-24T08:24:42.983 INFO:tasks.workunit.client.1.smithi149.stdout:9/356: creat d1/d2/d34/f78 x:0 0 0 2024-04-24T08:24:42.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/43: creat d8/da/fd x:0 0 0 2024-04-24T08:24:42.989 INFO:tasks.workunit.client.1.smithi149.stdout:1/401: getdents d0/d39/d53/d61 0 2024-04-24T08:24:42.991 INFO:tasks.workunit.client.1.smithi149.stdout:7/331: rename d9/f5b to d9/d43/d4f/d66/f6e 0 2024-04-24T08:24:42.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/257: creat d0/d6/db/d39/d3d/f51 x:0 0 0 2024-04-24T08:24:42.993 INFO:tasks.workunit.client.0.smithi090.stdout:2/23: fdatasync d0/f5 0 2024-04-24T08:24:42.996 INFO:tasks.workunit.client.1.smithi149.stdout:1/402: write d0/d5/db/d63/f83 [95876,27853] 0 2024-04-24T08:24:42.996 INFO:tasks.workunit.client.1.smithi149.stdout:6/262: write d0/d1a/f32 [1042189,89321] 0 2024-04-24T08:24:42.997 INFO:tasks.workunit.client.1.smithi149.stdout:8/281: write d1/de/f38 [38850,128813] 0 2024-04-24T08:24:43.001 INFO:tasks.workunit.client.1.smithi149.stdout:3/371: rename d0/d26/d5c to d0/d26/d3c/d83 0 2024-04-24T08:24:43.001 INFO:tasks.workunit.client.1.smithi149.stdout:6/263: fsync d0/d1a/d28/d31/d38/f3a 0 2024-04-24T08:24:43.003 INFO:tasks.workunit.client.1.smithi149.stdout:3/372: chown d0/d5/da/d59/d79 1043796 1 2024-04-24T08:24:43.003 INFO:tasks.workunit.client.0.smithi090.stdout:0/30: dread f8 [0,4194304] 0 2024-04-24T08:24:43.003 INFO:tasks.workunit.client.0.smithi090.stdout:2/24: rmdir d0 39 2024-04-24T08:24:43.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/46: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:43.004 INFO:tasks.workunit.client.0.smithi090.stdout:1/27: rename d3/f8 to d3/fb 0 2024-04-24T08:24:43.004 INFO:tasks.workunit.client.1.smithi149.stdout:8/282: write d1/d1f/f50 [1127869,37042] 0 2024-04-24T08:24:43.006 INFO:tasks.workunit.client.0.smithi090.stdout:6/32: getdents d8 0 2024-04-24T08:24:43.007 INFO:tasks.workunit.client.0.smithi090.stdout:7/47: dread - db/f11 zero size 2024-04-24T08:24:43.008 INFO:tasks.workunit.client.0.smithi090.stdout:0/31: write f8 [792748,7521] 0 2024-04-24T08:24:43.010 INFO:tasks.workunit.client.1.smithi149.stdout:3/373: write d0/d5/da/d19/d1b/d61/f77 [473293,74322] 0 2024-04-24T08:24:43.010 INFO:tasks.workunit.client.1.smithi149.stdout:9/357: dwrite d1/d2/d33/f66 [0,4194304] 0 2024-04-24T08:24:43.013 INFO:tasks.workunit.client.1.smithi149.stdout:6/264: mkdir d0/d1a/d28/d31/d38/d54 0 2024-04-24T08:24:43.015 INFO:tasks.workunit.client.0.smithi090.stdout:5/44: write f2 [1395071,13092] 0 2024-04-24T08:24:43.018 INFO:tasks.workunit.client.0.smithi090.stdout:6/33: fdatasync f6 0 2024-04-24T08:24:43.022 INFO:tasks.workunit.client.0.smithi090.stdout:1/28: fsync d3/fb 0 2024-04-24T08:24:43.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/45: write f3 [2047164,62836] 0 2024-04-24T08:24:43.023 INFO:tasks.workunit.client.1.smithi149.stdout:7/332: dwrite d9/d43/d4c/f52 [0,4194304] 0 2024-04-24T08:24:43.025 INFO:tasks.workunit.client.1.smithi149.stdout:8/283: fdatasync d1/d1f/d22/f33 0 2024-04-24T08:24:43.025 INFO:tasks.workunit.client.1.smithi149.stdout:7/333: readlink d9/d43/l62 0 2024-04-24T08:24:43.027 INFO:tasks.workunit.client.1.smithi149.stdout:2/258: rename d0/l1b to d0/d6/db/d39/d3d/l52 0 2024-04-24T08:24:43.027 INFO:tasks.workunit.client.0.smithi090.stdout:6/34: fdatasync d8/fa 0 2024-04-24T08:24:43.027 INFO:tasks.workunit.client.0.smithi090.stdout:7/48: rename l2 to db/l13 0 2024-04-24T08:24:43.027 INFO:tasks.workunit.client.0.smithi090.stdout:6/35: write f6 [537412,56848] 0 2024-04-24T08:24:43.029 INFO:tasks.workunit.client.1.smithi149.stdout:1/403: creat d0/f84 x:0 0 0 2024-04-24T08:24:43.034 INFO:tasks.workunit.client.0.smithi090.stdout:1/29: write f0 [1469618,9241] 0 2024-04-24T08:24:43.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/374: symlink d0/d5/d53/d4c/d5f/l84 0 2024-04-24T08:24:43.038 INFO:tasks.workunit.client.1.smithi149.stdout:2/259: creat d0/d6/db/d39/d3d/d44/f53 x:0 0 0 2024-04-24T08:24:43.039 INFO:tasks.workunit.client.1.smithi149.stdout:7/334: rename d9/dc/de/d44/f49 to d9/dc/d2e/f6f 0 2024-04-24T08:24:43.041 INFO:tasks.workunit.client.1.smithi149.stdout:1/404: creat d0/d1b/d5d/f85 x:0 0 0 2024-04-24T08:24:43.043 INFO:tasks.workunit.client.1.smithi149.stdout:1/405: write d0/d1b/d5d/f85 [103512,4185] 0 2024-04-24T08:24:43.043 INFO:tasks.workunit.client.1.smithi149.stdout:8/284: mkdir d1/d5d 0 2024-04-24T08:24:43.049 INFO:tasks.workunit.client.1.smithi149.stdout:1/406: dread d0/d1b/f64 [0,4194304] 0 2024-04-24T08:24:43.050 INFO:tasks.workunit.client.1.smithi149.stdout:3/375: readlink d0/d5/da/d19/d1b/d22/d38/l4a 0 2024-04-24T08:24:43.050 INFO:tasks.workunit.client.1.smithi149.stdout:6/265: dwrite d0/d1a/f4d [0,4194304] 0 2024-04-24T08:24:43.052 INFO:tasks.workunit.client.0.smithi090.stdout:0/32: link lb ld 0 2024-04-24T08:24:43.054 INFO:tasks.workunit.client.0.smithi090.stdout:2/25: getdents d0 0 2024-04-24T08:24:43.055 INFO:tasks.workunit.client.0.smithi090.stdout:5/46: dwrite f2 [0,4194304] 0 2024-04-24T08:24:43.056 INFO:tasks.workunit.client.0.smithi090.stdout:6/36: link l7 d8/lb 0 2024-04-24T08:24:43.056 INFO:tasks.workunit.client.1.smithi149.stdout:2/260: stat d0/c14 0 2024-04-24T08:24:43.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/26: fdatasync d0/f5 0 2024-04-24T08:24:43.062 INFO:tasks.workunit.client.1.smithi149.stdout:7/335: fdatasync d9/d43/d4c/f52 0 2024-04-24T08:24:43.063 INFO:tasks.workunit.client.0.smithi090.stdout:6/37: truncate d8/fa 994491 0 2024-04-24T08:24:43.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/33: symlink le 0 2024-04-24T08:24:43.066 INFO:tasks.workunit.client.1.smithi149.stdout:8/285: rmdir d1/d1f/d31 39 2024-04-24T08:24:43.066 INFO:tasks.workunit.client.1.smithi149.stdout:5/357: sync 2024-04-24T08:24:43.068 INFO:tasks.workunit.client.0.smithi090.stdout:1/30: dwrite d3/fb [0,4194304] 0 2024-04-24T08:24:43.072 INFO:tasks.workunit.client.1.smithi149.stdout:2/261: dread d0/f2d [0,4194304] 0 2024-04-24T08:24:43.072 INFO:tasks.workunit.client.0.smithi090.stdout:7/49: truncate db/f10 2300405 0 2024-04-24T08:24:43.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/38: mkdir d8/dc 0 2024-04-24T08:24:43.075 INFO:tasks.workunit.client.1.smithi149.stdout:3/376: rmdir d0/d5 39 2024-04-24T08:24:43.078 INFO:tasks.workunit.client.0.smithi090.stdout:5/47: dread d8/f9 [0,4194304] 0 2024-04-24T08:24:43.079 INFO:tasks.workunit.client.1.smithi149.stdout:1/407: rmdir d0/d5/d4f/d60/d19/d5c 39 2024-04-24T08:24:43.080 INFO:tasks.workunit.client.0.smithi090.stdout:2/27: dread d0/f5 [0,4194304] 0 2024-04-24T08:24:43.082 INFO:tasks.workunit.client.1.smithi149.stdout:9/358: getdents d1/d2/d15 0 2024-04-24T08:24:43.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/28: chown d0 14530 1 2024-04-24T08:24:43.084 INFO:tasks.workunit.client.0.smithi090.stdout:5/48: truncate d8/da/fd 776713 0 2024-04-24T08:24:43.085 INFO:tasks.workunit.client.1.smithi149.stdout:7/336: symlink d9/d43/l70 0 2024-04-24T08:24:43.086 INFO:tasks.workunit.client.0.smithi090.stdout:2/29: write d0/f5 [8365936,126547] 0 2024-04-24T08:24:43.088 INFO:tasks.workunit.client.1.smithi149.stdout:5/358: dwrite db/d12/d1e/f4a [0,4194304] 0 2024-04-24T08:24:43.089 INFO:tasks.workunit.client.0.smithi090.stdout:0/34: dwrite f8 [0,4194304] 0 2024-04-24T08:24:43.093 INFO:tasks.workunit.client.0.smithi090.stdout:1/31: getdents d3 0 2024-04-24T08:24:43.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/39: dread f0 [0,4194304] 0 2024-04-24T08:24:43.094 INFO:tasks.workunit.client.1.smithi149.stdout:2/262: mknod d0/d6/db/d39/d3d/c54 0 2024-04-24T08:24:43.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/35: creat ff x:0 0 0 2024-04-24T08:24:43.097 INFO:tasks.workunit.client.0.smithi090.stdout:7/50: dread db/f10 [0,4194304] 0 2024-04-24T08:24:43.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/408: creat d0/d5/db/d63/f86 x:0 0 0 2024-04-24T08:24:43.098 INFO:tasks.workunit.client.1.smithi149.stdout:6/266: symlink d0/l55 0 2024-04-24T08:24:43.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/40: rename d8/c9 to d8/cd 0 2024-04-24T08:24:43.101 INFO:tasks.workunit.client.0.smithi090.stdout:7/51: stat db/ff 0 2024-04-24T08:24:43.104 INFO:tasks.workunit.client.0.smithi090.stdout:7/52: write fa [294708,10182] 0 2024-04-24T08:24:43.106 INFO:tasks.workunit.client.0.smithi090.stdout:7/53: chown fa 63 1 2024-04-24T08:24:43.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/49: dwrite f4 [0,4194304] 0 2024-04-24T08:24:43.110 INFO:tasks.workunit.client.1.smithi149.stdout:3/377: dread d0/d5/da/f6e [0,4194304] 0 2024-04-24T08:24:43.110 INFO:tasks.workunit.client.1.smithi149.stdout:8/286: dwrite d1/de/d18/d43/f55 [0,4194304] 0 2024-04-24T08:24:43.111 INFO:tasks.workunit.client.1.smithi149.stdout:7/337: write d9/dc/d2e/f6f [1579727,57117] 0 2024-04-24T08:24:43.111 INFO:tasks.workunit.client.0.smithi090.stdout:6/41: mknod d8/ce 0 2024-04-24T08:24:43.111 INFO:tasks.workunit.client.0.smithi090.stdout:1/32: dread f0 [0,4194304] 0 2024-04-24T08:24:43.113 INFO:tasks.workunit.client.0.smithi090.stdout:6/42: write d8/fa [2041727,37387] 0 2024-04-24T08:24:43.114 INFO:tasks.workunit.client.0.smithi090.stdout:5/50: dread d8/da/fd [0,4194304] 0 2024-04-24T08:24:43.118 INFO:tasks.workunit.client.0.smithi090.stdout:7/54: mknod db/c14 0 2024-04-24T08:24:43.121 INFO:tasks.workunit.client.1.smithi149.stdout:5/359: symlink db/d12/d1e/d65/l6f 0 2024-04-24T08:24:43.121 INFO:tasks.workunit.client.1.smithi149.stdout:8/287: write d1/de/d18/f45 [728261,88808] 0 2024-04-24T08:24:43.125 INFO:tasks.workunit.client.1.smithi149.stdout:7/338: rmdir d9/dc/d2e 39 2024-04-24T08:24:43.125 INFO:tasks.workunit.client.1.smithi149.stdout:8/288: chown d1/d1f/d22/f33 64206946 1 2024-04-24T08:24:43.128 INFO:tasks.workunit.client.0.smithi090.stdout:6/43: chown d8/lb 63564989 1 2024-04-24T08:24:43.129 INFO:tasks.workunit.client.0.smithi090.stdout:5/51: symlink d8/le 0 2024-04-24T08:24:43.131 INFO:tasks.workunit.client.1.smithi149.stdout:9/359: rename d1/da/d49 to d1/da/dd/d79 0 2024-04-24T08:24:43.131 INFO:tasks.workunit.client.0.smithi090.stdout:5/52: read d8/f9 [3672320,37766] 0 2024-04-24T08:24:43.131 INFO:tasks.workunit.client.0.smithi090.stdout:1/33: dwrite d3/fb [4194304,4194304] 0 2024-04-24T08:24:43.134 INFO:tasks.workunit.client.1.smithi149.stdout:5/360: symlink db/d12/d22/l70 0 2024-04-24T08:24:43.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/53: write f3 [4782143,66291] 0 2024-04-24T08:24:43.136 INFO:tasks.workunit.client.0.smithi090.stdout:5/54: truncate f3 5441002 0 2024-04-24T08:24:43.139 INFO:tasks.workunit.client.1.smithi149.stdout:9/360: chown d1/d2/d33/d76 188267 1 2024-04-24T08:24:43.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/36: getdents . 0 2024-04-24T08:24:43.142 INFO:tasks.workunit.client.1.smithi149.stdout:5/361: read - db/d12/d1e/f64 zero size 2024-04-24T08:24:43.144 INFO:tasks.workunit.client.1.smithi149.stdout:9/361: dread - d1/d2/d34/f78 zero size 2024-04-24T08:24:43.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/362: truncate db/d23/f44 127261 0 2024-04-24T08:24:43.148 INFO:tasks.workunit.client.0.smithi090.stdout:7/55: dwrite f7 [0,4194304] 0 2024-04-24T08:24:43.149 INFO:tasks.workunit.client.0.smithi090.stdout:7/56: write f7 [524396,30039] 0 2024-04-24T08:24:43.150 INFO:tasks.workunit.client.1.smithi149.stdout:5/363: write db/d12/f5b [579253,59177] 0 2024-04-24T08:24:43.150 INFO:tasks.workunit.client.0.smithi090.stdout:7/57: chown l5 3915 1 2024-04-24T08:24:43.153 INFO:tasks.workunit.client.1.smithi149.stdout:7/339: dwrite d9/dc/d11/f42 [0,4194304] 0 2024-04-24T08:24:43.153 INFO:tasks.workunit.client.0.smithi090.stdout:6/44: symlink d8/dc/lf 0 2024-04-24T08:24:43.157 INFO:tasks.workunit.client.0.smithi090.stdout:5/55: mknod d8/cf 0 2024-04-24T08:24:43.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/340: truncate d9/d43/d4c/f5c 360968 0 2024-04-24T08:24:43.160 INFO:tasks.workunit.client.0.smithi090.stdout:0/37: rename le to l10 0 2024-04-24T08:24:43.162 INFO:tasks.workunit.client.0.smithi090.stdout:6/45: dread d8/fa [0,4194304] 0 2024-04-24T08:24:43.167 INFO:tasks.workunit.client.0.smithi090.stdout:1/34: mkdir d3/d5/da/dc 0 2024-04-24T08:24:43.167 INFO:tasks.workunit.client.1.smithi149.stdout:3/378: getdents d0/d26 0 2024-04-24T08:24:43.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/379: read d0/d5/d1a/f5d [201064,33281] 0 2024-04-24T08:24:43.175 INFO:tasks.workunit.client.0.smithi090.stdout:1/35: write d3/fb [5413457,1311] 0 2024-04-24T08:24:43.176 INFO:tasks.workunit.client.0.smithi090.stdout:1/36: chown f0 37796669 1 2024-04-24T08:24:43.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/58: creat db/f15 x:0 0 0 2024-04-24T08:24:43.178 INFO:tasks.workunit.client.0.smithi090.stdout:7/59: write f9 [582469,61777] 0 2024-04-24T08:24:43.179 INFO:tasks.workunit.client.1.smithi149.stdout:5/364: creat db/d12/d22/d45/d5d/f71 x:0 0 0 2024-04-24T08:24:43.180 INFO:tasks.workunit.client.0.smithi090.stdout:5/56: creat d8/f10 x:0 0 0 2024-04-24T08:24:43.183 INFO:tasks.workunit.client.1.smithi149.stdout:7/341: write d9/dc/d2e/f6f [2192870,121521] 0 2024-04-24T08:24:43.185 INFO:tasks.workunit.client.0.smithi090.stdout:7/60: fsync f9 0 2024-04-24T08:24:43.188 INFO:tasks.workunit.client.1.smithi149.stdout:2/263: rename d0/d6/db/f1a to d0/f55 0 2024-04-24T08:24:43.193 INFO:tasks.workunit.client.1.smithi149.stdout:5/365: dread db/d12/d22/d45/f42 [0,4194304] 0 2024-04-24T08:24:43.196 INFO:tasks.workunit.client.1.smithi149.stdout:9/362: mknod d1/da/dd/d55/d72/c7a 0 2024-04-24T08:24:43.197 INFO:tasks.workunit.client.1.smithi149.stdout:9/363: chown d1/d2/d33/f4e 370737890 1 2024-04-24T08:24:43.199 INFO:tasks.workunit.client.1.smithi149.stdout:7/342: read d9/dc/d11/d26/f28 [346827,42167] 0 2024-04-24T08:24:43.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/409: rename d0/d5/db/d18/d4d/f62 to d0/d5/db/f87 0 2024-04-24T08:24:43.204 INFO:tasks.workunit.client.1.smithi149.stdout:1/410: fsync d0/d5/d4f/d60/d19/f57 0 2024-04-24T08:24:43.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/364: unlink d1/de/f6a 0 2024-04-24T08:24:43.211 INFO:tasks.workunit.client.1.smithi149.stdout:2/264: mknod d0/c56 0 2024-04-24T08:24:43.211 INFO:tasks.workunit.client.1.smithi149.stdout:3/380: truncate d0/d5/da/d19/d1b/d61/f77 441140 0 2024-04-24T08:24:43.217 INFO:tasks.workunit.client.1.smithi149.stdout:1/411: stat d0/da 0 2024-04-24T08:24:43.222 INFO:tasks.workunit.client.1.smithi149.stdout:9/365: dwrite d1/d2/f48 [0,4194304] 0 2024-04-24T08:24:43.227 INFO:tasks.workunit.client.1.smithi149.stdout:3/381: rmdir d0/d5/da/d19 39 2024-04-24T08:24:43.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/382: dread - d0/d5/d53/d34/d3a/f36 zero size 2024-04-24T08:24:43.233 INFO:tasks.workunit.client.1.smithi149.stdout:5/366: link db/d12/d1e/f41 db/d12/f72 0 2024-04-24T08:24:43.240 INFO:tasks.workunit.client.1.smithi149.stdout:8/289: rename d1/d5d to d1/d1f/d5e 0 2024-04-24T08:24:43.244 INFO:tasks.workunit.client.1.smithi149.stdout:5/367: mkdir db/d12/d2f/d73 0 2024-04-24T08:24:43.247 INFO:tasks.workunit.client.1.smithi149.stdout:1/412: symlink d0/d1b/l88 0 2024-04-24T08:24:43.248 INFO:tasks.workunit.client.1.smithi149.stdout:5/368: dread - db/d12/d1e/f64 zero size 2024-04-24T08:24:43.251 INFO:tasks.workunit.client.1.smithi149.stdout:8/290: creat d1/d1f/d22/f5f x:0 0 0 2024-04-24T08:24:43.252 INFO:tasks.workunit.client.1.smithi149.stdout:5/369: chown db/d12/d22/f33 184476 1 2024-04-24T08:24:43.252 INFO:tasks.workunit.client.1.smithi149.stdout:5/370: chown db/d23 2121503 1 2024-04-24T08:24:43.253 INFO:tasks.workunit.client.1.smithi149.stdout:3/383: dread d0/d5/da/d19/d1b/d22/d38/f64 [0,4194304] 0 2024-04-24T08:24:43.254 INFO:tasks.workunit.client.1.smithi149.stdout:9/366: dwrite d1/de/f22 [0,4194304] 0 2024-04-24T08:24:43.263 INFO:tasks.workunit.client.1.smithi149.stdout:8/291: rename d1/de/d18/l2d to d1/de/d18/l60 0 2024-04-24T08:24:43.264 INFO:tasks.workunit.client.1.smithi149.stdout:4/368: sync 2024-04-24T08:24:43.265 INFO:tasks.workunit.client.1.smithi149.stdout:3/384: symlink d0/d5/da/d5a/l85 0 2024-04-24T08:24:43.267 INFO:tasks.workunit.client.1.smithi149.stdout:1/413: dwrite d0/da/f24 [0,4194304] 0 2024-04-24T08:24:43.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/367: creat d1/da/dd/d55/d72/f7b x:0 0 0 2024-04-24T08:24:43.272 INFO:tasks.workunit.client.1.smithi149.stdout:5/371: dwrite db/d12/d22/d6c/f6e [0,4194304] 0 2024-04-24T08:24:43.272 INFO:tasks.workunit.client.1.smithi149.stdout:1/414: readlink d0/d5/d4f/l6a 0 2024-04-24T08:24:43.273 INFO:tasks.workunit.client.1.smithi149.stdout:8/292: write d1/de/f2f [765590,69117] 0 2024-04-24T08:24:43.275 INFO:tasks.workunit.client.1.smithi149.stdout:1/415: stat d0/da 0 2024-04-24T08:24:43.284 INFO:tasks.workunit.client.1.smithi149.stdout:3/385: rename d0/d26/d3c/d83 to d0/d5/d53/d4c/d5f/d86 0 2024-04-24T08:24:43.288 INFO:tasks.workunit.client.1.smithi149.stdout:2/265: dwrite d0/d6/db/d39/d3d/d44/f4c [0,4194304] 0 2024-04-24T08:24:43.290 INFO:tasks.workunit.client.1.smithi149.stdout:9/368: creat d1/d2/d33/d76/f7c x:0 0 0 2024-04-24T08:24:43.299 INFO:tasks.workunit.client.1.smithi149.stdout:5/372: mkdir db/d12/d2f/d74 0 2024-04-24T08:24:43.299 INFO:tasks.workunit.client.1.smithi149.stdout:5/373: chown db/d12/d2f/f62 20336333 1 2024-04-24T08:24:43.311 INFO:tasks.workunit.client.1.smithi149.stdout:8/293: dwrite d1/de/f2b [4194304,4194304] 0 2024-04-24T08:24:43.315 INFO:tasks.workunit.client.1.smithi149.stdout:2/266: mkdir d0/d6/db/d39/d3d/d44/d57 0 2024-04-24T08:24:43.321 INFO:tasks.workunit.client.1.smithi149.stdout:5/374: read db/d12/f3d [144868,92134] 0 2024-04-24T08:24:43.321 INFO:tasks.workunit.client.1.smithi149.stdout:3/386: mknod d0/d5/da/d59/d79/c87 0 2024-04-24T08:24:43.323 INFO:tasks.workunit.client.1.smithi149.stdout:2/267: readlink d0/d6/l34 0 2024-04-24T08:24:43.324 INFO:tasks.workunit.client.1.smithi149.stdout:3/387: chown d0/d5/d53/d34/d3a/f33 18963 1 2024-04-24T08:24:43.325 INFO:tasks.workunit.client.1.smithi149.stdout:2/268: dread - d0/d6/f41 zero size 2024-04-24T08:24:43.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/294: dwrite d1/d1f/d22/f33 [0,4194304] 0 2024-04-24T08:24:43.329 INFO:tasks.workunit.client.1.smithi149.stdout:9/369: link d1/d2/d15/d1c/d42/c46 d1/da/dd/d55/c7d 0 2024-04-24T08:24:43.333 INFO:tasks.workunit.client.1.smithi149.stdout:1/416: getdents d0/d5/db/d18/d4d 0 2024-04-24T08:24:43.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/269: dread d0/d6/db/f42 [0,4194304] 0 2024-04-24T08:24:43.346 INFO:tasks.workunit.client.1.smithi149.stdout:2/270: stat d0/f4d 0 2024-04-24T08:24:43.347 INFO:tasks.workunit.client.1.smithi149.stdout:8/295: dwrite d1/d1f/f50 [0,4194304] 0 2024-04-24T08:24:43.348 INFO:tasks.workunit.client.1.smithi149.stdout:1/417: dread d0/d5/d4f/f77 [0,4194304] 0 2024-04-24T08:24:43.350 INFO:tasks.workunit.client.1.smithi149.stdout:2/271: write d0/d6/db/d39/f4e [752607,97326] 0 2024-04-24T08:24:43.351 INFO:tasks.workunit.client.1.smithi149.stdout:8/296: fdatasync d1/de/d18/d43/f55 0 2024-04-24T08:24:43.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/370: mknod d1/da/dd/c7e 0 2024-04-24T08:24:43.355 INFO:tasks.workunit.client.1.smithi149.stdout:1/418: dread d0/d39/d53/d61/f71 [0,4194304] 0 2024-04-24T08:24:43.358 INFO:tasks.workunit.client.1.smithi149.stdout:1/419: write d0/d1b/d4b/f58 [1034637,31577] 0 2024-04-24T08:24:43.362 INFO:tasks.workunit.client.1.smithi149.stdout:5/375: link db/l19 db/d12/l75 0 2024-04-24T08:24:43.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/272: write d0/f22 [2273479,87226] 0 2024-04-24T08:24:43.370 INFO:tasks.workunit.client.1.smithi149.stdout:3/388: getdents d0/d5/d53/d34 0 2024-04-24T08:24:43.372 INFO:tasks.workunit.client.1.smithi149.stdout:3/389: chown d0/d5/d53/d6d 969376525 1 2024-04-24T08:24:43.374 INFO:tasks.workunit.client.1.smithi149.stdout:1/420: mknod d0/d1b/d4b/c89 0 2024-04-24T08:24:43.379 INFO:tasks.workunit.client.1.smithi149.stdout:2/273: mkdir d0/d6/db/d39/d3d/d58 0 2024-04-24T08:24:43.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/371: link d1/da/dd/d55/d72/f7b d1/d2/d15/d1c/d42/f7f 0 2024-04-24T08:24:43.387 INFO:tasks.workunit.client.1.smithi149.stdout:9/372: truncate d1/d2/d15/f58 1159046 0 2024-04-24T08:24:43.387 INFO:tasks.workunit.client.1.smithi149.stdout:5/376: rename c3 to db/d12/d1e/d65/c76 0 2024-04-24T08:24:43.389 INFO:tasks.workunit.client.1.smithi149.stdout:5/377: chown db/d12/d22/l70 32064 1 2024-04-24T08:24:43.390 INFO:tasks.workunit.client.1.smithi149.stdout:8/297: creat d1/f61 x:0 0 0 2024-04-24T08:24:43.391 INFO:tasks.workunit.client.1.smithi149.stdout:1/421: dwrite d0/d5/db/d18/d34/f56 [0,4194304] 0 2024-04-24T08:24:43.391 INFO:tasks.workunit.client.1.smithi149.stdout:3/390: getdents d0/d5/d53/d6d 0 2024-04-24T08:24:43.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/422: fsync d0/d5/db/d18/f2c 0 2024-04-24T08:24:43.397 INFO:tasks.workunit.client.1.smithi149.stdout:8/298: fsync d1/d1f/d31/f35 0 2024-04-24T08:24:43.398 INFO:tasks.workunit.client.1.smithi149.stdout:5/378: fsync db/d12/f72 0 2024-04-24T08:24:43.399 INFO:tasks.workunit.client.1.smithi149.stdout:8/299: write d1/f4f [52605,128632] 0 2024-04-24T08:24:43.402 INFO:tasks.workunit.client.1.smithi149.stdout:1/423: stat d0/d5/c17 0 2024-04-24T08:24:43.404 INFO:tasks.workunit.client.1.smithi149.stdout:3/391: mkdir d0/d88 0 2024-04-24T08:24:43.407 INFO:tasks.workunit.client.1.smithi149.stdout:8/300: symlink d1/d1f/d5e/l62 0 2024-04-24T08:24:43.407 INFO:tasks.workunit.client.1.smithi149.stdout:3/392: fdatasync d0/d5/da/d19/d1b/d22/d38/f7e 0 2024-04-24T08:24:43.412 INFO:tasks.workunit.client.1.smithi149.stdout:5/379: rmdir db/d12/d2f/d73 0 2024-04-24T08:24:43.412 INFO:tasks.workunit.client.1.smithi149.stdout:3/393: chown d0/d5/da/d19/d1b/d22/d38/l4a 124108110 1 2024-04-24T08:24:43.415 INFO:tasks.workunit.client.0.smithi090.stdout:5/57: symlink d8/da/l11 0 2024-04-24T08:24:43.417 INFO:tasks.workunit.client.1.smithi149.stdout:3/394: mknod d0/d5/da/d59/d79/c89 0 2024-04-24T08:24:43.419 INFO:tasks.workunit.client.1.smithi149.stdout:8/301: truncate d1/d1f/f2c 571782 0 2024-04-24T08:24:43.419 INFO:tasks.workunit.client.1.smithi149.stdout:3/395: readlink d0/d5/d53/d34/l6c 0 2024-04-24T08:24:43.421 INFO:tasks.workunit.client.0.smithi090.stdout:7/61: symlink db/l16 0 2024-04-24T08:24:43.426 INFO:tasks.workunit.client.1.smithi149.stdout:8/302: mkdir d1/de/d18/d32/d63 0 2024-04-24T08:24:43.428 INFO:tasks.workunit.client.1.smithi149.stdout:5/380: dwrite db/d12/d2f/f57 [0,4194304] 0 2024-04-24T08:24:43.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/62: mkdir db/d17 0 2024-04-24T08:24:43.430 INFO:tasks.workunit.client.1.smithi149.stdout:1/424: dread d0/d5/db/d63/f6b [4194304,4194304] 0 2024-04-24T08:24:43.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/303: rmdir d1/de/d18 39 2024-04-24T08:24:43.433 INFO:tasks.workunit.client.0.smithi090.stdout:5/58: dread f2 [0,4194304] 0 2024-04-24T08:24:43.433 INFO:tasks.workunit.client.1.smithi149.stdout:1/425: chown d0/d5/db/d18/d34/f56 1 1 2024-04-24T08:24:43.437 INFO:tasks.workunit.client.0.smithi090.stdout:6/46: getdents d8 0 2024-04-24T08:24:43.441 INFO:tasks.workunit.client.0.smithi090.stdout:5/59: mknod d8/c12 0 2024-04-24T08:24:43.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/47: dread f6 [0,4194304] 0 2024-04-24T08:24:43.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/63: link db/ff db/d17/f18 0 2024-04-24T08:24:43.447 INFO:tasks.workunit.client.1.smithi149.stdout:3/396: link d0/d5/d53/d34/f81 d0/d5/d53/d39/f8a 0 2024-04-24T08:24:43.449 INFO:tasks.workunit.client.1.smithi149.stdout:5/381: mknod db/d12/d22/d45/c77 0 2024-04-24T08:24:43.450 INFO:tasks.workunit.client.1.smithi149.stdout:5/382: readlink db/d12/l17 0 2024-04-24T08:24:43.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/64: mkdir db/d19 0 2024-04-24T08:24:43.454 INFO:tasks.workunit.client.1.smithi149.stdout:1/426: unlink d0/da/fc 0 2024-04-24T08:24:43.455 INFO:tasks.workunit.client.1.smithi149.stdout:3/397: creat d0/d5/d53/d4c/d5f/f8b x:0 0 0 2024-04-24T08:24:43.456 INFO:tasks.workunit.client.1.smithi149.stdout:9/373: dwrite d1/d2/d15/d1c/f20 [0,4194304] 0 2024-04-24T08:24:43.457 INFO:tasks.workunit.client.1.smithi149.stdout:3/398: dread - d0/d5/d53/d34/f81 zero size 2024-04-24T08:24:43.458 INFO:tasks.workunit.client.0.smithi090.stdout:5/60: dwrite d8/da/fc [0,4194304] 0 2024-04-24T08:24:43.460 INFO:tasks.workunit.client.1.smithi149.stdout:3/399: chown d0/d5/d53/d34/d3a/f36 227740 1 2024-04-24T08:24:43.463 INFO:tasks.workunit.client.1.smithi149.stdout:5/383: dread db/d12/d1e/f4a [0,4194304] 0 2024-04-24T08:24:43.470 INFO:tasks.workunit.client.0.smithi090.stdout:6/48: dwrite d8/fa [0,4194304] 0 2024-04-24T08:24:43.482 INFO:tasks.workunit.client.1.smithi149.stdout:8/304: rename d1/d1f/f40 to d1/f64 0 2024-04-24T08:24:43.482 INFO:tasks.workunit.client.1.smithi149.stdout:0/314: sync 2024-04-24T08:24:43.483 INFO:tasks.workunit.client.0.smithi090.stdout:6/49: dread f0 [0,4194304] 0 2024-04-24T08:24:43.485 INFO:tasks.workunit.client.1.smithi149.stdout:9/374: dread d1/da/dd/d79/f63 [0,4194304] 0 2024-04-24T08:24:43.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/61: mkdir d8/da/d13 0 2024-04-24T08:24:43.489 INFO:tasks.workunit.client.1.smithi149.stdout:1/427: creat d0/d5/db/d18/f8a x:0 0 0 2024-04-24T08:24:43.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/375: dread d1/d2/d33/f66 [0,4194304] 0 2024-04-24T08:24:43.497 INFO:tasks.workunit.client.0.smithi090.stdout:5/62: mkdir d8/da/d14 0 2024-04-24T08:24:43.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/65: rmdir db/d19 0 2024-04-24T08:24:43.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/66: dread - db/d17/f18 zero size 2024-04-24T08:24:43.503 INFO:tasks.workunit.client.0.smithi090.stdout:0/38: truncate f8 586658 0 2024-04-24T08:24:43.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/63: rename f3 to d8/f15 0 2024-04-24T08:24:43.508 INFO:tasks.workunit.client.1.smithi149.stdout:0/315: rmdir d2/d9/d1a/d36 39 2024-04-24T08:24:43.510 INFO:tasks.workunit.client.0.smithi090.stdout:1/37: truncate d3/f4 1957991 0 2024-04-24T08:24:43.510 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:43 smithi090 ceph-mon[25016]: pgmap v14: 65 pgs: 65 active+clean; 843 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 106 MiB/s wr, 274 op/s 2024-04-24T08:24:43.513 INFO:tasks.workunit.client.1.smithi149.stdout:1/428: rmdir d0/d1b/d5d 39 2024-04-24T08:24:43.515 INFO:tasks.workunit.client.1.smithi149.stdout:1/429: chown d0/d5/f8 449522 1 2024-04-24T08:24:43.515 INFO:tasks.workunit.client.0.smithi090.stdout:7/67: dread f3 [0,4194304] 0 2024-04-24T08:24:43.516 INFO:tasks.workunit.client.1.smithi149.stdout:3/400: creat d0/d5/da/d19/d80/f8c x:0 0 0 2024-04-24T08:24:43.517 INFO:tasks.workunit.client.1.smithi149.stdout:9/376: creat d1/d2/d33/f80 x:0 0 0 2024-04-24T08:24:43.518 INFO:tasks.workunit.client.1.smithi149.stdout:3/401: dread - d0/d5/da/d19/d1b/d22/d38/f7e zero size 2024-04-24T08:24:43.521 INFO:tasks.workunit.client.1.smithi149.stdout:3/402: read d0/d5/d53/d34/f3b [1614624,125593] 0 2024-04-24T08:24:43.522 INFO:tasks.workunit.client.1.smithi149.stdout:8/305: mkdir d1/de/d18/d32/d65 0 2024-04-24T08:24:43.524 INFO:tasks.workunit.client.1.smithi149.stdout:3/403: write d0/d5/d53/d4c/d5f/f8b [883815,52457] 0 2024-04-24T08:24:43.525 INFO:tasks.workunit.client.0.smithi090.stdout:5/64: symlink d8/da/d14/l16 0 2024-04-24T08:24:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:43 smithi149 ceph-mon[31282]: pgmap v14: 65 pgs: 65 active+clean; 843 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 106 MiB/s wr, 274 op/s 2024-04-24T08:24:43.527 INFO:tasks.workunit.client.1.smithi149.stdout:0/316: write d2/d9/df/d2f/f30 [538751,11025] 0 2024-04-24T08:24:43.528 INFO:tasks.workunit.client.0.smithi090.stdout:7/68: creat db/f1a x:0 0 0 2024-04-24T08:24:43.530 INFO:tasks.workunit.client.1.smithi149.stdout:0/317: write d2/d9/df/f59 [190934,19006] 0 2024-04-24T08:24:43.532 INFO:tasks.workunit.client.0.smithi090.stdout:5/65: creat d8/da/d14/f17 x:0 0 0 2024-04-24T08:24:43.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/66: stat d8/da/fc 0 2024-04-24T08:24:43.533 INFO:tasks.workunit.client.1.smithi149.stdout:9/377: creat d1/d2/d15/f81 x:0 0 0 2024-04-24T08:24:43.535 INFO:tasks.workunit.client.0.smithi090.stdout:7/69: write f3 [2603626,13812] 0 2024-04-24T08:24:43.538 INFO:tasks.workunit.client.1.smithi149.stdout:3/404: rename d0/l57 to d0/d88/l8d 0 2024-04-24T08:24:43.542 INFO:tasks.workunit.client.1.smithi149.stdout:5/384: write db/d23/f44 [548344,98968] 0 2024-04-24T08:24:43.543 INFO:tasks.workunit.client.0.smithi090.stdout:6/50: dwrite d8/fa [4194304,4194304] 0 2024-04-24T08:24:43.544 INFO:tasks.workunit.client.0.smithi090.stdout:0/39: write f8 [993818,105809] 0 2024-04-24T08:24:43.544 INFO:tasks.workunit.client.0.smithi090.stdout:7/70: mkdir db/d17/d1b 0 2024-04-24T08:24:43.546 INFO:tasks.workunit.client.0.smithi090.stdout:5/67: dread d8/f15 [4194304,4194304] 0 2024-04-24T08:24:43.546 INFO:tasks.workunit.client.1.smithi149.stdout:8/306: creat d1/de/d18/d32/d65/f66 x:0 0 0 2024-04-24T08:24:43.550 INFO:tasks.workunit.client.0.smithi090.stdout:1/38: fsync d3/f4 0 2024-04-24T08:24:43.552 INFO:tasks.workunit.client.0.smithi090.stdout:0/40: mknod c11 0 2024-04-24T08:24:43.552 INFO:tasks.workunit.client.1.smithi149.stdout:1/430: creat d0/d5/d4f/d60/d19/f8b x:0 0 0 2024-04-24T08:24:43.554 INFO:tasks.workunit.client.1.smithi149.stdout:1/431: write d0/f25 [638828,10663] 0 2024-04-24T08:24:43.554 INFO:tasks.workunit.client.1.smithi149.stdout:3/405: truncate d0/d5/da/f6e 1082540 0 2024-04-24T08:24:43.556 INFO:tasks.workunit.client.0.smithi090.stdout:5/68: mkdir d8/da/d18 0 2024-04-24T08:24:43.556 INFO:tasks.workunit.client.0.smithi090.stdout:6/51: dread f6 [0,4194304] 0 2024-04-24T08:24:43.557 INFO:tasks.workunit.client.1.smithi149.stdout:5/385: mkdir db/d12/d2f/d78 0 2024-04-24T08:24:43.559 INFO:tasks.workunit.client.1.smithi149.stdout:9/378: unlink d1/d2/d15/d1c/d42/c46 0 2024-04-24T08:24:43.561 INFO:tasks.workunit.client.1.smithi149.stdout:9/379: write d1/d2/f2b [2408379,71112] 0 2024-04-24T08:24:43.562 INFO:tasks.workunit.client.0.smithi090.stdout:1/39: dread d3/f4 [0,4194304] 0 2024-04-24T08:24:43.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/41: creat f12 x:0 0 0 2024-04-24T08:24:43.563 INFO:tasks.workunit.client.1.smithi149.stdout:8/307: creat d1/d1f/d31/f67 x:0 0 0 2024-04-24T08:24:43.564 INFO:tasks.workunit.client.0.smithi090.stdout:1/40: readlink d3/d5/l6 0 2024-04-24T08:24:43.565 INFO:tasks.workunit.client.0.smithi090.stdout:7/71: dwrite fa [0,4194304] 0 2024-04-24T08:24:43.566 INFO:tasks.workunit.client.1.smithi149.stdout:0/318: creat d2/d17/f5e x:0 0 0 2024-04-24T08:24:43.566 INFO:tasks.workunit.client.0.smithi090.stdout:6/52: truncate f6 1615794 0 2024-04-24T08:24:43.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/432: write d0/d5/f8 [836333,14392] 0 2024-04-24T08:24:43.569 INFO:tasks.workunit.client.1.smithi149.stdout:0/319: write d2/d9/d1a/f25 [289103,32210] 0 2024-04-24T08:24:43.570 INFO:tasks.workunit.client.1.smithi149.stdout:3/406: creat d0/d88/f8e x:0 0 0 2024-04-24T08:24:43.571 INFO:tasks.workunit.client.0.smithi090.stdout:5/69: dread f2 [0,4194304] 0 2024-04-24T08:24:43.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/386: dread db/f16 [0,4194304] 0 2024-04-24T08:24:43.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/72: dread db/fd [0,4194304] 0 2024-04-24T08:24:43.577 INFO:tasks.workunit.client.0.smithi090.stdout:6/53: creat d8/dc/f10 x:0 0 0 2024-04-24T08:24:43.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/73: readlink l8 0 2024-04-24T08:24:43.579 INFO:tasks.workunit.client.0.smithi090.stdout:5/70: creat d8/da/f19 x:0 0 0 2024-04-24T08:24:43.579 INFO:tasks.workunit.client.1.smithi149.stdout:1/433: creat d0/d5/db/f8c x:0 0 0 2024-04-24T08:24:43.580 INFO:tasks.workunit.client.1.smithi149.stdout:8/308: dread d1/d1f/d22/f33 [0,4194304] 0 2024-04-24T08:24:43.581 INFO:tasks.workunit.client.1.smithi149.stdout:0/320: readlink d2/d9/l14 0 2024-04-24T08:24:43.582 INFO:tasks.workunit.client.0.smithi090.stdout:6/54: creat d8/dc/f11 x:0 0 0 2024-04-24T08:24:43.585 INFO:tasks.workunit.client.0.smithi090.stdout:7/74: symlink db/d17/l1c 0 2024-04-24T08:24:43.586 INFO:tasks.workunit.client.1.smithi149.stdout:9/380: dwrite d1/d2/d33/f80 [0,4194304] 0 2024-04-24T08:24:43.586 INFO:tasks.workunit.client.0.smithi090.stdout:0/42: dwrite f12 [0,4194304] 0 2024-04-24T08:24:43.587 INFO:tasks.workunit.client.0.smithi090.stdout:0/43: dread - ff zero size 2024-04-24T08:24:43.591 INFO:tasks.workunit.client.1.smithi149.stdout:3/407: dread d0/d5/da/d19/d1b/d22/d38/f64 [0,4194304] 0 2024-04-24T08:24:43.594 INFO:tasks.workunit.client.0.smithi090.stdout:0/44: creat f13 x:0 0 0 2024-04-24T08:24:43.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/41: truncate f0 522185 0 2024-04-24T08:24:43.596 INFO:tasks.workunit.client.0.smithi090.stdout:7/75: symlink db/d17/d1b/l1d 0 2024-04-24T08:24:43.597 INFO:tasks.workunit.client.1.smithi149.stdout:6/267: sync 2024-04-24T08:24:43.597 INFO:tasks.workunit.client.1.smithi149.stdout:7/343: sync 2024-04-24T08:24:43.598 INFO:tasks.workunit.client.0.smithi090.stdout:0/45: write f13 [348642,98575] 0 2024-04-24T08:24:43.603 INFO:tasks.workunit.client.0.smithi090.stdout:0/46: creat f14 x:0 0 0 2024-04-24T08:24:43.605 INFO:tasks.workunit.client.0.smithi090.stdout:7/76: symlink db/d17/d1b/l1e 0 2024-04-24T08:24:43.607 INFO:tasks.workunit.client.0.smithi090.stdout:5/71: dwrite f7 [0,4194304] 0 2024-04-24T08:24:43.608 INFO:tasks.workunit.client.0.smithi090.stdout:6/55: dread f6 [0,4194304] 0 2024-04-24T08:24:43.613 INFO:tasks.workunit.client.0.smithi090.stdout:5/72: symlink d8/l1a 0 2024-04-24T08:24:43.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/77: mknod db/d17/c1f 0 2024-04-24T08:24:43.615 INFO:tasks.workunit.client.1.smithi149.stdout:3/408: fsync d0/d5/da/f24 0 2024-04-24T08:24:43.615 INFO:tasks.workunit.client.1.smithi149.stdout:9/381: rename d1/de/c5b to d1/d2/d15/d1c/d42/c82 0 2024-04-24T08:24:43.615 INFO:tasks.workunit.client.0.smithi090.stdout:5/73: dread - d8/da/f19 zero size 2024-04-24T08:24:43.617 INFO:tasks.workunit.client.0.smithi090.stdout:0/47: dread f8 [0,4194304] 0 2024-04-24T08:24:43.617 INFO:tasks.workunit.client.1.smithi149.stdout:0/321: creat d2/d9/df/d56/f5f x:0 0 0 2024-04-24T08:24:43.618 INFO:tasks.workunit.client.0.smithi090.stdout:0/48: write f12 [1557202,104472] 0 2024-04-24T08:24:43.620 INFO:tasks.workunit.client.1.smithi149.stdout:5/387: write db/d12/d1e/f64 [699754,20213] 0 2024-04-24T08:24:43.620 INFO:tasks.workunit.client.0.smithi090.stdout:0/49: write ff [585851,72388] 0 2024-04-24T08:24:43.622 INFO:tasks.workunit.client.1.smithi149.stdout:1/434: dwrite d0/d39/f7c [0,4194304] 0 2024-04-24T08:24:43.624 INFO:tasks.workunit.client.1.smithi149.stdout:3/409: chown d0/d5/da/d19/d1b/c69 19408785 1 2024-04-24T08:24:43.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/388: symlink db/d1f/l79 0 2024-04-24T08:24:43.627 INFO:tasks.workunit.client.0.smithi090.stdout:1/42: truncate f0 1384306 0 2024-04-24T08:24:43.629 INFO:tasks.workunit.client.1.smithi149.stdout:9/382: readlink d1/d2/d15/d1c/l53 0 2024-04-24T08:24:43.630 INFO:tasks.workunit.client.1.smithi149.stdout:3/410: write d0/d26/d3c/f65 [1040964,126221] 0 2024-04-24T08:24:43.635 INFO:tasks.workunit.client.1.smithi149.stdout:3/411: chown d0/d5/d53/l4f 1 1 2024-04-24T08:24:43.637 INFO:tasks.workunit.client.1.smithi149.stdout:5/389: chown db/d12/d2f/c6a 9261164 1 2024-04-24T08:24:43.638 INFO:tasks.workunit.client.0.smithi090.stdout:7/78: dwrite db/f12 [0,4194304] 0 2024-04-24T08:24:43.639 INFO:tasks.workunit.client.1.smithi149.stdout:5/390: readlink db/d12/d22/l2e 0 2024-04-24T08:24:43.640 INFO:tasks.workunit.client.1.smithi149.stdout:1/435: mknod d0/d5/db/d18/d4d/c8d 0 2024-04-24T08:24:43.644 INFO:tasks.workunit.client.0.smithi090.stdout:6/56: write f0 [2214378,5458] 0 2024-04-24T08:24:43.647 INFO:tasks.workunit.client.0.smithi090.stdout:6/57: write f0 [148871,116955] 0 2024-04-24T08:24:43.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/58: dread - d8/dc/f10 zero size 2024-04-24T08:24:43.649 INFO:tasks.workunit.client.1.smithi149.stdout:3/412: creat d0/d5/d1a/f8f x:0 0 0 2024-04-24T08:24:43.649 INFO:tasks.workunit.client.1.smithi149.stdout:9/383: dread d1/d2/f48 [0,4194304] 0 2024-04-24T08:24:43.649 INFO:tasks.workunit.client.0.smithi090.stdout:6/59: chown d8 0 1 2024-04-24T08:24:43.650 INFO:tasks.workunit.client.1.smithi149.stdout:1/436: fdatasync d0/f5f 0 2024-04-24T08:24:43.651 INFO:tasks.workunit.client.0.smithi090.stdout:6/60: truncate d8/fa 8490052 0 2024-04-24T08:24:43.655 INFO:tasks.workunit.client.1.smithi149.stdout:9/384: mkdir d1/d2/d33/d5d/d83 0 2024-04-24T08:24:43.657 INFO:tasks.workunit.client.1.smithi149.stdout:0/322: getdents d2 0 2024-04-24T08:24:43.658 INFO:tasks.workunit.client.1.smithi149.stdout:5/391: dread db/d12/d1e/f64 [0,4194304] 0 2024-04-24T08:24:43.659 INFO:tasks.workunit.client.0.smithi090.stdout:6/61: mkdir d8/dc/d12 0 2024-04-24T08:24:43.660 INFO:tasks.workunit.client.1.smithi149.stdout:3/413: read d0/d5/da/d5e/f30 [1285324,78451] 0 2024-04-24T08:24:43.661 INFO:tasks.workunit.client.0.smithi090.stdout:0/50: truncate f12 161991 0 2024-04-24T08:24:43.665 INFO:tasks.workunit.client.0.smithi090.stdout:0/51: dread f8 [0,4194304] 0 2024-04-24T08:24:43.666 INFO:tasks.workunit.client.1.smithi149.stdout:6/268: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:43.666 INFO:tasks.workunit.client.0.smithi090.stdout:0/52: chown f14 101 1 2024-04-24T08:24:43.669 INFO:tasks.workunit.client.1.smithi149.stdout:9/385: creat d1/d2/d33/d76/f84 x:0 0 0 2024-04-24T08:24:43.672 INFO:tasks.workunit.client.1.smithi149.stdout:7/344: dread d9/dc/f35 [0,4194304] 0 2024-04-24T08:24:43.677 INFO:tasks.workunit.client.1.smithi149.stdout:0/323: unlink d2/d3c/f51 0 2024-04-24T08:24:43.679 INFO:tasks.workunit.client.0.smithi090.stdout:6/62: dwrite d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:43.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/43: link f0 d3/d5/fd 0 2024-04-24T08:24:43.690 INFO:tasks.workunit.client.1.smithi149.stdout:5/392: getdents db/d12/d2f/d74 0 2024-04-24T08:24:43.692 INFO:tasks.workunit.client.1.smithi149.stdout:9/386: unlink d1/d2/d15/l37 0 2024-04-24T08:24:43.694 INFO:tasks.workunit.client.1.smithi149.stdout:3/414: dread d0/d5/ff [0,4194304] 0 2024-04-24T08:24:43.696 INFO:tasks.workunit.client.1.smithi149.stdout:9/387: write d1/d2/d15/d1c/f20 [1358269,37232] 0 2024-04-24T08:24:43.696 INFO:tasks.workunit.client.0.smithi090.stdout:1/44: creat d3/d5/da/dc/fe x:0 0 0 2024-04-24T08:24:43.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/45: chown d3/d5/da/dc 3264 1 2024-04-24T08:24:43.700 INFO:tasks.workunit.client.0.smithi090.stdout:0/53: dwrite f8 [0,4194304] 0 2024-04-24T08:24:43.702 INFO:tasks.workunit.client.1.smithi149.stdout:1/437: truncate d0/d39/f7c 4150793 0 2024-04-24T08:24:43.703 INFO:tasks.workunit.client.1.smithi149.stdout:7/345: getdents d9/d43/d4c 0 2024-04-24T08:24:43.704 INFO:tasks.workunit.client.0.smithi090.stdout:1/46: creat d3/d5/ff x:0 0 0 2024-04-24T08:24:43.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/269: link d0/l19 d0/d1a/d28/d4a/l56 0 2024-04-24T08:24:43.706 INFO:tasks.workunit.client.0.smithi090.stdout:0/54: creat f15 x:0 0 0 2024-04-24T08:24:43.706 INFO:tasks.workunit.client.1.smithi149.stdout:1/438: chown d0/d5/d4f/d60/d19/l3e 508 1 2024-04-24T08:24:43.707 INFO:tasks.workunit.client.0.smithi090.stdout:6/63: dwrite d8/dc/f11 [0,4194304] 0 2024-04-24T08:24:43.707 INFO:tasks.workunit.client.1.smithi149.stdout:3/415: symlink d0/d5/d53/d6d/l90 0 2024-04-24T08:24:43.708 INFO:tasks.workunit.client.0.smithi090.stdout:6/64: fsync d8/fa 0 2024-04-24T08:24:43.709 INFO:tasks.workunit.client.1.smithi149.stdout:7/346: creat d9/d43/d4c/f71 x:0 0 0 2024-04-24T08:24:43.714 INFO:tasks.workunit.client.0.smithi090.stdout:0/55: mknod c16 0 2024-04-24T08:24:43.721 INFO:tasks.workunit.client.0.smithi090.stdout:6/65: dwrite d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:43.726 INFO:tasks.workunit.client.1.smithi149.stdout:1/439: symlink d0/d39/d53/d61/l8e 0 2024-04-24T08:24:43.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/388: dwrite d1/da/f3e [0,4194304] 0 2024-04-24T08:24:43.734 INFO:tasks.workunit.client.1.smithi149.stdout:5/393: dwrite db/d35/f59 [0,4194304] 0 2024-04-24T08:24:43.739 INFO:tasks.workunit.client.1.smithi149.stdout:6/270: symlink d0/d1a/l57 0 2024-04-24T08:24:43.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/324: dwrite d2/fa [0,4194304] 0 2024-04-24T08:24:43.743 INFO:tasks.workunit.client.1.smithi149.stdout:1/440: creat d0/d5/d4f/d60/f8f x:0 0 0 2024-04-24T08:24:43.744 INFO:tasks.workunit.client.0.smithi090.stdout:1/47: link c1 d3/c10 0 2024-04-24T08:24:43.746 INFO:tasks.workunit.client.1.smithi149.stdout:6/271: stat d0/d1a/d28/d31/d4c/l50 0 2024-04-24T08:24:43.750 INFO:tasks.workunit.client.1.smithi149.stdout:7/347: mkdir d9/dc/d72 0 2024-04-24T08:24:43.751 INFO:tasks.workunit.client.1.smithi149.stdout:3/416: dwrite d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:43.751 INFO:tasks.workunit.client.1.smithi149.stdout:5/394: creat db/d12/d1e/d65/f7a x:0 0 0 2024-04-24T08:24:43.755 INFO:tasks.workunit.client.1.smithi149.stdout:9/389: mknod d1/da/dd/c85 0 2024-04-24T08:24:43.756 INFO:tasks.workunit.client.1.smithi149.stdout:9/390: chown d1/de/f2e 2 1 2024-04-24T08:24:43.759 INFO:tasks.workunit.client.1.smithi149.stdout:9/391: chown d1/d2/f41 0 1 2024-04-24T08:24:43.759 INFO:tasks.workunit.client.0.smithi090.stdout:0/56: link cc c17 0 2024-04-24T08:24:43.761 INFO:tasks.workunit.client.0.smithi090.stdout:0/57: write f14 [987386,82092] 0 2024-04-24T08:24:43.761 INFO:tasks.workunit.client.1.smithi149.stdout:9/392: write d1/d4c/f54 [1974024,109883] 0 2024-04-24T08:24:43.762 INFO:tasks.workunit.client.1.smithi149.stdout:4/369: sync 2024-04-24T08:24:43.762 INFO:tasks.workunit.client.1.smithi149.stdout:2/274: sync 2024-04-24T08:24:43.764 INFO:tasks.workunit.client.1.smithi149.stdout:1/441: creat d0/d5/db/d18/d4d/f90 x:0 0 0 2024-04-24T08:24:43.765 INFO:tasks.workunit.client.0.smithi090.stdout:1/48: unlink d3/d5/fd 0 2024-04-24T08:24:43.766 INFO:tasks.workunit.client.1.smithi149.stdout:4/370: write d2/d24/d2e/d3f/f4d [578538,18863] 0 2024-04-24T08:24:43.766 INFO:tasks.workunit.client.1.smithi149.stdout:7/348: rmdir d9/dc/d11/d26 39 2024-04-24T08:24:43.768 INFO:tasks.workunit.client.0.smithi090.stdout:6/66: write f6 [1104703,109910] 0 2024-04-24T08:24:43.768 INFO:tasks.workunit.client.1.smithi149.stdout:9/393: write d1/d2/d33/f80 [1036860,4685] 0 2024-04-24T08:24:43.770 INFO:tasks.workunit.client.1.smithi149.stdout:7/349: fsync d9/dc/de/f3a 0 2024-04-24T08:24:43.771 INFO:tasks.workunit.client.1.smithi149.stdout:4/371: stat d2/d24/d2e/f4a 0 2024-04-24T08:24:43.775 INFO:tasks.workunit.client.1.smithi149.stdout:3/417: creat d0/d5/d1a/f91 x:0 0 0 2024-04-24T08:24:43.775 INFO:tasks.workunit.client.0.smithi090.stdout:0/58: unlink c5 0 2024-04-24T08:24:43.778 INFO:tasks.workunit.client.1.smithi149.stdout:5/395: dread db/f20 [0,4194304] 0 2024-04-24T08:24:43.778 INFO:tasks.workunit.client.1.smithi149.stdout:0/325: symlink d2/d9/d1a/l60 0 2024-04-24T08:24:43.778 INFO:tasks.workunit.client.0.smithi090.stdout:0/59: write f8 [2855279,126284] 0 2024-04-24T08:24:43.782 INFO:tasks.workunit.client.1.smithi149.stdout:1/442: rmdir d0/d39/d53 39 2024-04-24T08:24:43.783 INFO:tasks.workunit.client.0.smithi090.stdout:6/67: dread d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:43.787 INFO:tasks.workunit.client.0.smithi090.stdout:6/68: symlink d8/l13 0 2024-04-24T08:24:43.788 INFO:tasks.workunit.client.0.smithi090.stdout:6/69: fsync d8/fa 0 2024-04-24T08:24:43.789 INFO:tasks.workunit.client.1.smithi149.stdout:1/443: fdatasync d0/d5/d4f/d60/d19/f57 0 2024-04-24T08:24:43.790 INFO:tasks.workunit.client.0.smithi090.stdout:6/70: fsync d8/dc/f11 0 2024-04-24T08:24:43.793 INFO:tasks.workunit.client.1.smithi149.stdout:0/326: rename d2/d9/df/d2f/f50 to d2/d3c/f61 0 2024-04-24T08:24:43.793 INFO:tasks.workunit.client.0.smithi090.stdout:6/71: write f6 [691820,22495] 0 2024-04-24T08:24:43.795 INFO:tasks.workunit.client.0.smithi090.stdout:6/72: chown d8/dc/lf 5617 1 2024-04-24T08:24:43.798 INFO:tasks.workunit.client.1.smithi149.stdout:3/418: dread d0/d5/da/d5a/d63/d6a/f7b [0,4194304] 0 2024-04-24T08:24:43.798 INFO:tasks.workunit.client.1.smithi149.stdout:2/275: dwrite d0/d6/db/d27/f37 [0,4194304] 0 2024-04-24T08:24:43.808 INFO:tasks.workunit.client.0.smithi090.stdout:6/73: dread d8/fa [0,4194304] 0 2024-04-24T08:24:43.811 INFO:tasks.workunit.client.1.smithi149.stdout:0/327: mknod d2/d3c/c62 0 2024-04-24T08:24:43.811 INFO:tasks.workunit.client.1.smithi149.stdout:6/272: truncate d0/d1a/f25 244766 0 2024-04-24T08:24:43.812 INFO:tasks.workunit.client.1.smithi149.stdout:9/394: link d1/da/d13/l43 d1/d4c/l86 0 2024-04-24T08:24:43.814 INFO:tasks.workunit.client.0.smithi090.stdout:6/74: symlink d8/l14 0 2024-04-24T08:24:43.818 INFO:tasks.workunit.client.1.smithi149.stdout:7/350: dwrite d9/dc/de/f16 [0,4194304] 0 2024-04-24T08:24:43.818 INFO:tasks.workunit.client.0.smithi090.stdout:1/49: dwrite f0 [0,4194304] 0 2024-04-24T08:24:43.821 INFO:tasks.workunit.client.0.smithi090.stdout:6/75: creat d8/f15 x:0 0 0 2024-04-24T08:24:43.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/76: read d8/fa [7024339,117700] 0 2024-04-24T08:24:43.823 INFO:tasks.workunit.client.0.smithi090.stdout:1/50: symlink d3/l11 0 2024-04-24T08:24:43.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/60: getdents . 0 2024-04-24T08:24:43.824 INFO:tasks.workunit.client.1.smithi149.stdout:5/396: dwrite db/d12/f1b [0,4194304] 0 2024-04-24T08:24:43.825 INFO:tasks.workunit.client.0.smithi090.stdout:6/77: chown d8/f15 0 1 2024-04-24T08:24:43.826 INFO:tasks.workunit.client.1.smithi149.stdout:5/397: write db/d12/d22/d45/d5d/f71 [792031,96820] 0 2024-04-24T08:24:43.827 INFO:tasks.workunit.client.1.smithi149.stdout:0/328: dread d2/f8 [0,4194304] 0 2024-04-24T08:24:43.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/78: write f0 [2043813,35208] 0 2024-04-24T08:24:43.828 INFO:tasks.workunit.client.1.smithi149.stdout:5/398: write db/d12/f1b [1020290,81759] 0 2024-04-24T08:24:43.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/51: mkdir d3/d12 0 2024-04-24T08:24:43.839 INFO:tasks.workunit.client.1.smithi149.stdout:2/276: symlink d0/d6/db/l59 0 2024-04-24T08:24:43.848 INFO:tasks.workunit.client.0.smithi090.stdout:6/79: link d8/fa d8/f16 0 2024-04-24T08:24:43.849 INFO:tasks.workunit.client.1.smithi149.stdout:0/329: mknod d2/d9/df/d40/c63 0 2024-04-24T08:24:43.849 INFO:tasks.workunit.client.0.smithi090.stdout:0/61: creat f18 x:0 0 0 2024-04-24T08:24:43.852 INFO:tasks.workunit.client.1.smithi149.stdout:5/399: symlink db/d12/d22/l7b 0 2024-04-24T08:24:43.854 INFO:tasks.workunit.client.1.smithi149.stdout:5/400: fdatasync db/d1f/f5c 0 2024-04-24T08:24:43.854 INFO:tasks.workunit.client.1.smithi149.stdout:1/444: truncate d0/d5/d4f/d60/d19/f2b 3046989 0 2024-04-24T08:24:43.855 INFO:tasks.workunit.client.1.smithi149.stdout:9/395: truncate d1/da/dd/d55/d72/f7b 220780 0 2024-04-24T08:24:43.856 INFO:tasks.workunit.client.1.smithi149.stdout:5/401: chown db/d12/f69 7 1 2024-04-24T08:24:43.857 INFO:tasks.workunit.client.1.smithi149.stdout:7/351: creat d9/d43/d64/f73 x:0 0 0 2024-04-24T08:24:43.858 INFO:tasks.workunit.client.1.smithi149.stdout:5/402: dread - db/d12/d1e/f4f zero size 2024-04-24T08:24:43.858 INFO:tasks.workunit.client.1.smithi149.stdout:7/352: readlink d9/dc/d2e/l30 0 2024-04-24T08:24:43.860 INFO:tasks.workunit.client.1.smithi149.stdout:2/277: write d0/d6/db/f42 [464267,130881] 0 2024-04-24T08:24:43.863 INFO:tasks.workunit.client.1.smithi149.stdout:0/330: rename d2/d9/d1a/f44 to d2/d9/df/d40/d4a/f64 0 2024-04-24T08:24:43.864 INFO:tasks.workunit.client.0.smithi090.stdout:6/80: link d8/dc/lf d8/dc/l17 0 2024-04-24T08:24:43.868 INFO:tasks.workunit.client.1.smithi149.stdout:0/331: write d2/d9/f2d [3769039,84670] 0 2024-04-24T08:24:43.868 INFO:tasks.workunit.client.1.smithi149.stdout:3/419: link d0/d88/l8d d0/d26/l92 0 2024-04-24T08:24:43.873 INFO:tasks.workunit.client.1.smithi149.stdout:5/403: creat db/d35/f7c x:0 0 0 2024-04-24T08:24:43.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/62: mknod c19 0 2024-04-24T08:24:43.877 INFO:tasks.workunit.client.1.smithi149.stdout:6/273: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:43.879 INFO:tasks.workunit.client.0.smithi090.stdout:6/81: dwrite f0 [0,4194304] 0 2024-04-24T08:24:43.884 INFO:tasks.workunit.client.1.smithi149.stdout:7/353: rename d9/dc/d11/f3b to d9/dc/de/d44/f74 0 2024-04-24T08:24:43.886 INFO:tasks.workunit.client.0.smithi090.stdout:0/63: write f12 [204266,117738] 0 2024-04-24T08:24:43.888 INFO:tasks.workunit.client.0.smithi090.stdout:6/82: mknod d8/dc/c18 0 2024-04-24T08:24:43.889 INFO:tasks.workunit.client.0.smithi090.stdout:6/83: chown d8/ce 6 1 2024-04-24T08:24:43.892 INFO:tasks.workunit.client.1.smithi149.stdout:9/396: symlink d1/da/l87 0 2024-04-24T08:24:43.893 INFO:tasks.workunit.client.1.smithi149.stdout:5/404: fdatasync db/d12/d22/f33 0 2024-04-24T08:24:43.895 INFO:tasks.workunit.client.1.smithi149.stdout:5/405: chown db/d12/d22/d6c 108 1 2024-04-24T08:24:43.896 INFO:tasks.workunit.client.0.smithi090.stdout:6/84: mkdir d8/dc/d19 0 2024-04-24T08:24:43.898 INFO:tasks.workunit.client.0.smithi090.stdout:0/64: mkdir d1a 0 2024-04-24T08:24:43.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/274: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:43.900 INFO:tasks.workunit.client.1.smithi149.stdout:6/275: write d0/f1 [8299949,6567] 0 2024-04-24T08:24:43.901 INFO:tasks.workunit.client.1.smithi149.stdout:2/278: creat d0/d6/db/d39/d3d/d44/d57/f5a x:0 0 0 2024-04-24T08:24:43.902 INFO:tasks.workunit.client.0.smithi090.stdout:6/85: creat d8/dc/f1a x:0 0 0 2024-04-24T08:24:43.903 INFO:tasks.workunit.client.1.smithi149.stdout:0/332: symlink d2/d9/d1a/l65 0 2024-04-24T08:24:43.904 INFO:tasks.workunit.client.0.smithi090.stdout:6/86: truncate d8/f15 687115 0 2024-04-24T08:24:43.905 INFO:tasks.workunit.client.1.smithi149.stdout:3/420: rename d0/d5/da/d19/d1b/d61 to d0/d5/d53/d34/d3a/d93 0 2024-04-24T08:24:43.906 INFO:tasks.workunit.client.1.smithi149.stdout:8/309: sync 2024-04-24T08:24:43.906 INFO:tasks.workunit.client.1.smithi149.stdout:7/354: mknod d9/d43/d4f/d66/c75 0 2024-04-24T08:24:43.908 INFO:tasks.workunit.client.1.smithi149.stdout:0/333: truncate d2/d9/d1a/f3f 770454 0 2024-04-24T08:24:43.909 INFO:tasks.workunit.client.1.smithi149.stdout:1/445: creat d0/d1b/f91 x:0 0 0 2024-04-24T08:24:43.909 INFO:tasks.workunit.client.1.smithi149.stdout:3/421: write d0/d5/f6f [2601192,125762] 0 2024-04-24T08:24:43.911 INFO:tasks.workunit.client.1.smithi149.stdout:1/446: chown d0/d5/db/d18/d34/d51 175494 1 2024-04-24T08:24:43.913 INFO:tasks.workunit.client.1.smithi149.stdout:3/422: read d0/d5/f16 [804218,119352] 0 2024-04-24T08:24:43.917 INFO:tasks.workunit.client.1.smithi149.stdout:2/279: symlink d0/d6/db/d28/l5b 0 2024-04-24T08:24:43.923 INFO:tasks.workunit.client.1.smithi149.stdout:7/355: dwrite d9/dc/f1b [0,4194304] 0 2024-04-24T08:24:43.924 INFO:tasks.workunit.client.1.smithi149.stdout:1/447: creat d0/d5/db/f92 x:0 0 0 2024-04-24T08:24:43.927 INFO:tasks.workunit.client.1.smithi149.stdout:3/423: rmdir d0/d5/da/d5a 39 2024-04-24T08:24:43.928 INFO:tasks.workunit.client.0.smithi090.stdout:0/65: truncate ff 592296 0 2024-04-24T08:24:43.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/280: fsync d0/d6/f12 0 2024-04-24T08:24:43.930 INFO:tasks.workunit.client.1.smithi149.stdout:9/397: creat d1/f88 x:0 0 0 2024-04-24T08:24:43.931 INFO:tasks.workunit.client.1.smithi149.stdout:5/406: link db/d12/d22/l70 db/d12/d22/l7d 0 2024-04-24T08:24:43.934 INFO:tasks.workunit.client.1.smithi149.stdout:7/356: truncate d9/dc/de/f22 30336 0 2024-04-24T08:24:43.935 INFO:tasks.workunit.client.1.smithi149.stdout:5/407: chown db/d12/d1e/f4a 57 1 2024-04-24T08:24:43.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/66: creat d1a/f1b x:0 0 0 2024-04-24T08:24:43.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/87: dwrite d8/f16 [4194304,4194304] 0 2024-04-24T08:24:43.938 INFO:tasks.workunit.client.1.smithi149.stdout:8/310: creat d1/d1f/f68 x:0 0 0 2024-04-24T08:24:43.940 INFO:tasks.workunit.client.0.smithi090.stdout:0/67: dread - f15 zero size 2024-04-24T08:24:43.947 INFO:tasks.workunit.client.1.smithi149.stdout:6/276: truncate d0/f12 1585361 0 2024-04-24T08:24:43.953 INFO:tasks.workunit.client.1.smithi149.stdout:3/424: symlink d0/d5/da/d5a/d63/d6a/l94 0 2024-04-24T08:24:43.954 INFO:tasks.workunit.client.1.smithi149.stdout:9/398: dwrite d1/d2/d34/f78 [0,4194304] 0 2024-04-24T08:24:43.961 INFO:tasks.workunit.client.1.smithi149.stdout:2/281: write d0/d6/db/f1c [985729,84138] 0 2024-04-24T08:24:43.963 INFO:tasks.workunit.client.1.smithi149.stdout:1/448: rename d0/d5/d4f/d60/d19/l1a to d0/d1b/l93 0 2024-04-24T08:24:43.963 INFO:tasks.workunit.client.1.smithi149.stdout:2/282: fdatasync d0/f2 0 2024-04-24T08:24:43.965 INFO:tasks.workunit.client.1.smithi149.stdout:8/311: dwrite d1/de/f2b [4194304,4194304] 0 2024-04-24T08:24:43.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/283: chown d0/d6/db/d39/l3f 14713377 1 2024-04-24T08:24:43.969 INFO:tasks.workunit.client.1.smithi149.stdout:5/408: symlink db/d12/d2f/d74/l7e 0 2024-04-24T08:24:43.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/284: readlink d0/l40 0 2024-04-24T08:24:43.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/399: creat d1/da/d13/f89 x:0 0 0 2024-04-24T08:24:43.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/425: creat d0/d5/d53/d34/f95 x:0 0 0 2024-04-24T08:24:43.975 INFO:tasks.workunit.client.1.smithi149.stdout:9/400: readlink d1/d2/d15/d1c/l53 0 2024-04-24T08:24:43.975 INFO:tasks.workunit.client.1.smithi149.stdout:3/426: chown d0/d5/d73 3136 1 2024-04-24T08:24:43.975 INFO:tasks.workunit.client.1.smithi149.stdout:7/357: dwrite d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:43.979 INFO:tasks.workunit.client.1.smithi149.stdout:6/277: symlink d0/d1a/l58 0 2024-04-24T08:24:43.980 INFO:tasks.workunit.client.1.smithi149.stdout:8/312: readlink d1/l12 0 2024-04-24T08:24:43.983 INFO:tasks.workunit.client.1.smithi149.stdout:5/409: mkdir db/d1f/d7f 0 2024-04-24T08:24:43.984 INFO:tasks.workunit.client.1.smithi149.stdout:9/401: truncate d1/d2/d33/f5f 420659 0 2024-04-24T08:24:43.984 INFO:tasks.workunit.client.1.smithi149.stdout:6/278: mkdir d0/d1a/d28/d59 0 2024-04-24T08:24:43.985 INFO:tasks.workunit.client.1.smithi149.stdout:2/285: mknod d0/d6/db/d39/d3d/d58/c5c 0 2024-04-24T08:24:43.986 INFO:tasks.workunit.client.1.smithi149.stdout:9/402: chown d1/da/d13/f64 1480 1 2024-04-24T08:24:43.988 INFO:tasks.workunit.client.1.smithi149.stdout:1/449: link d0/d5/db/d18/f29 d0/d5/df/f94 0 2024-04-24T08:24:43.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/286: read d0/f22 [98369,1483] 0 2024-04-24T08:24:43.992 INFO:tasks.workunit.client.1.smithi149.stdout:8/313: creat d1/d1f/f69 x:0 0 0 2024-04-24T08:24:43.994 INFO:tasks.workunit.client.1.smithi149.stdout:5/410: write db/d12/d22/d45/f21 [8271050,8754] 0 2024-04-24T08:24:43.994 INFO:tasks.workunit.client.1.smithi149.stdout:9/403: symlink d1/d2/d33/d76/l8a 0 2024-04-24T08:24:43.996 INFO:tasks.workunit.client.1.smithi149.stdout:2/287: write d0/f4d [811544,92746] 0 2024-04-24T08:24:43.997 INFO:tasks.workunit.client.1.smithi149.stdout:8/314: read d1/de/d18/f48 [210003,121572] 0 2024-04-24T08:24:43.998 INFO:tasks.workunit.client.0.smithi090.stdout:0/68: dwrite ff [0,4194304] 0 2024-04-24T08:24:43.999 INFO:tasks.workunit.client.1.smithi149.stdout:9/404: write d1/d2/d15/f30 [4210463,72160] 0 2024-04-24T08:24:44.003 INFO:tasks.workunit.client.1.smithi149.stdout:1/450: creat d0/d5/db/d63/f95 x:0 0 0 2024-04-24T08:24:44.004 INFO:tasks.workunit.client.0.smithi090.stdout:0/69: mknod d1a/c1c 0 2024-04-24T08:24:44.008 INFO:tasks.workunit.client.0.smithi090.stdout:0/70: chown c17 33923 1 2024-04-24T08:24:44.014 INFO:tasks.workunit.client.1.smithi149.stdout:1/451: dread d0/d1b/d4b/f58 [0,4194304] 0 2024-04-24T08:24:44.014 INFO:tasks.workunit.client.0.smithi090.stdout:0/71: creat d1a/f1d x:0 0 0 2024-04-24T08:24:44.015 INFO:tasks.workunit.client.1.smithi149.stdout:7/358: rename d9/dc/de/f22 to d9/dc/d72/f76 0 2024-04-24T08:24:44.017 INFO:tasks.workunit.client.1.smithi149.stdout:5/411: dread db/d35/f59 [0,4194304] 0 2024-04-24T08:24:44.017 INFO:tasks.workunit.client.1.smithi149.stdout:7/359: stat d9/dc/de/f12 0 2024-04-24T08:24:44.018 INFO:tasks.workunit.client.0.smithi090.stdout:0/72: fsync f14 0 2024-04-24T08:24:44.018 INFO:tasks.workunit.client.1.smithi149.stdout:5/412: chown db/d12/d2f/d78 2320323 1 2024-04-24T08:24:44.020 INFO:tasks.workunit.client.1.smithi149.stdout:5/413: chown db/d12/d22/f33 13 1 2024-04-24T08:24:44.022 INFO:tasks.workunit.client.1.smithi149.stdout:6/279: truncate d0/d1a/f4d 3773503 0 2024-04-24T08:24:44.024 INFO:tasks.workunit.client.1.smithi149.stdout:3/427: dread d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:44.027 INFO:tasks.workunit.client.1.smithi149.stdout:9/405: symlink d1/da/dd/d55/l8b 0 2024-04-24T08:24:44.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/406: stat d1/d2/f48 0 2024-04-24T08:24:44.029 INFO:tasks.workunit.client.1.smithi149.stdout:1/452: creat d0/d5/d4f/d60/d19/d5c/f96 x:0 0 0 2024-04-24T08:24:44.031 INFO:tasks.workunit.client.1.smithi149.stdout:5/414: creat db/d23/f80 x:0 0 0 2024-04-24T08:24:44.032 INFO:tasks.workunit.client.1.smithi149.stdout:3/428: mknod d0/d5/da/d19/d1b/d22/c96 0 2024-04-24T08:24:44.032 INFO:tasks.workunit.client.1.smithi149.stdout:5/415: stat db/d23 0 2024-04-24T08:24:44.033 INFO:tasks.workunit.client.1.smithi149.stdout:7/360: creat d9/d43/d4c/d67/d69/f77 x:0 0 0 2024-04-24T08:24:44.034 INFO:tasks.workunit.client.1.smithi149.stdout:1/453: chown d0/d5/db/d63/f95 3 1 2024-04-24T08:24:44.035 INFO:tasks.workunit.client.1.smithi149.stdout:9/407: creat d1/d2/d15/f8c x:0 0 0 2024-04-24T08:24:44.035 INFO:tasks.workunit.client.1.smithi149.stdout:2/288: dwrite d0/f22 [0,4194304] 0 2024-04-24T08:24:44.037 INFO:tasks.workunit.client.1.smithi149.stdout:5/416: chown db/d12/d2f/f46 921131982 1 2024-04-24T08:24:44.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/289: write d0/f22 [2596577,88846] 0 2024-04-24T08:24:44.039 INFO:tasks.workunit.client.1.smithi149.stdout:5/417: write db/d12/d60/f67 [5954477,46912] 0 2024-04-24T08:24:44.040 INFO:tasks.workunit.client.0.smithi090.stdout:0/73: dwrite d1a/f1d [0,4194304] 0 2024-04-24T08:24:44.040 INFO:tasks.workunit.client.1.smithi149.stdout:7/361: creat d9/dc/de/d44/f78 x:0 0 0 2024-04-24T08:24:44.043 INFO:tasks.workunit.client.1.smithi149.stdout:5/418: chown db/d23/f43 19922 1 2024-04-24T08:24:44.044 INFO:tasks.workunit.client.1.smithi149.stdout:1/454: chown d0/d5/db/d18/d2f/c67 382 1 2024-04-24T08:24:44.050 INFO:tasks.workunit.client.1.smithi149.stdout:9/408: mkdir d1/da/dd/d8d 0 2024-04-24T08:24:44.050 INFO:tasks.workunit.client.0.smithi090.stdout:0/74: creat d1a/f1e x:0 0 0 2024-04-24T08:24:44.052 INFO:tasks.workunit.client.1.smithi149.stdout:9/409: dread - d1/d2/d15/d1c/f62 zero size 2024-04-24T08:24:44.053 INFO:tasks.workunit.client.0.smithi090.stdout:0/75: write f12 [1107250,79278] 0 2024-04-24T08:24:44.054 INFO:tasks.workunit.client.1.smithi149.stdout:9/410: chown d1/d2/d15/f2f 801 1 2024-04-24T08:24:44.056 INFO:tasks.workunit.client.1.smithi149.stdout:6/280: truncate d0/fe 2491881 0 2024-04-24T08:24:44.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/76: symlink d1a/l1f 0 2024-04-24T08:24:44.064 INFO:tasks.workunit.client.1.smithi149.stdout:5/419: symlink db/d12/d1e/l81 0 2024-04-24T08:24:44.065 INFO:tasks.workunit.client.0.smithi090.stdout:0/77: write f12 [1030712,29672] 0 2024-04-24T08:24:44.066 INFO:tasks.workunit.client.0.smithi090.stdout:0/78: chown f14 330527 1 2024-04-24T08:24:44.068 INFO:tasks.workunit.client.1.smithi149.stdout:1/455: mknod d0/c97 0 2024-04-24T08:24:44.072 INFO:tasks.workunit.client.0.smithi090.stdout:0/79: creat d1a/f20 x:0 0 0 2024-04-24T08:24:44.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/80: write f18 [407136,104806] 0 2024-04-24T08:24:44.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/281: dwrite d0/f11 [0,4194304] 0 2024-04-24T08:24:44.077 INFO:tasks.workunit.client.1.smithi149.stdout:2/290: dread d0/d6/db/d28/f38 [0,4194304] 0 2024-04-24T08:24:44.081 INFO:tasks.workunit.client.1.smithi149.stdout:1/456: rmdir d0/da 39 2024-04-24T08:24:44.081 INFO:tasks.workunit.client.1.smithi149.stdout:5/420: chown db/d12/d2f/c32 102474826 1 2024-04-24T08:24:44.087 INFO:tasks.workunit.client.1.smithi149.stdout:3/429: truncate d0/d5/d53/d4c/d5f/f8b 513339 0 2024-04-24T08:24:44.088 INFO:tasks.workunit.client.1.smithi149.stdout:2/291: dread d0/f4d [0,4194304] 0 2024-04-24T08:24:44.090 INFO:tasks.workunit.client.0.smithi090.stdout:0/81: truncate f14 712135 0 2024-04-24T08:24:44.090 INFO:tasks.workunit.client.1.smithi149.stdout:7/362: truncate d9/dc/de/d44/f74 1010724 0 2024-04-24T08:24:44.094 INFO:tasks.workunit.client.0.smithi090.stdout:0/82: mknod d1a/c21 0 2024-04-24T08:24:44.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/83: stat lb 0 2024-04-24T08:24:44.096 INFO:tasks.workunit.client.0.smithi090.stdout:0/84: fsync ff 0 2024-04-24T08:24:44.096 INFO:tasks.workunit.client.1.smithi149.stdout:9/411: rename d1/da/d13/l43 to d1/da/l8e 0 2024-04-24T08:24:44.097 INFO:tasks.workunit.client.0.smithi090.stdout:0/85: read f12 [17021,8530] 0 2024-04-24T08:24:44.099 INFO:tasks.workunit.client.1.smithi149.stdout:6/282: link d0/d1a/d28/d31/d38/l43 d0/d1a/d28/d59/l5a 0 2024-04-24T08:24:44.100 INFO:tasks.workunit.client.0.smithi090.stdout:0/86: symlink d1a/l22 0 2024-04-24T08:24:44.103 INFO:tasks.workunit.client.1.smithi149.stdout:9/412: dread d1/da/f50 [0,4194304] 0 2024-04-24T08:24:44.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/87: symlink d1a/l23 0 2024-04-24T08:24:44.109 INFO:tasks.workunit.client.1.smithi149.stdout:1/457: rename d0/d5/db/d18/d34/d51 to d0/d5/db/d63/d98 0 2024-04-24T08:24:44.113 INFO:tasks.workunit.client.1.smithi149.stdout:7/363: dwrite d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:44.113 INFO:tasks.workunit.client.1.smithi149.stdout:6/283: mknod d0/d1a/d28/d31/c5b 0 2024-04-24T08:24:44.116 INFO:tasks.workunit.client.1.smithi149.stdout:1/458: truncate d0/d5/db/d18/d4d/f90 349751 0 2024-04-24T08:24:44.117 INFO:tasks.workunit.client.1.smithi149.stdout:7/364: write d9/d43/d4c/f71 [139151,90561] 0 2024-04-24T08:24:44.119 INFO:tasks.workunit.client.1.smithi149.stdout:1/459: chown d0/d5/d4f/d60/d19/d5c/f5e 94 1 2024-04-24T08:24:44.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/372: sync 2024-04-24T08:24:44.120 INFO:tasks.workunit.client.1.smithi149.stdout:6/284: creat d0/d1a/d28/d31/f5c x:0 0 0 2024-04-24T08:24:44.121 INFO:tasks.workunit.client.1.smithi149.stdout:3/430: getdents d0/d5/d53/d34/d3a/d93 0 2024-04-24T08:24:44.123 INFO:tasks.workunit.client.1.smithi149.stdout:2/292: getdents d0/d6/db/d39 0 2024-04-24T08:24:44.125 INFO:tasks.workunit.client.1.smithi149.stdout:9/413: dwrite d1/da/f50 [0,4194304] 0 2024-04-24T08:24:44.127 INFO:tasks.workunit.client.1.smithi149.stdout:2/293: dread - d0/d6/db/d39/d3d/f51 zero size 2024-04-24T08:24:44.128 INFO:tasks.workunit.client.0.smithi090.stdout:0/88: dwrite d1a/f1b [0,4194304] 0 2024-04-24T08:24:44.133 INFO:tasks.workunit.client.1.smithi149.stdout:5/421: rename db/d35/f61 to db/d12/f82 0 2024-04-24T08:24:44.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/89: fsync f8 0 2024-04-24T08:24:44.135 INFO:tasks.workunit.client.1.smithi149.stdout:6/285: symlink d0/d1a/d28/d31/d38/l5d 0 2024-04-24T08:24:44.136 INFO:tasks.workunit.client.1.smithi149.stdout:4/373: mkdir d2/d18/d1e/d3c/d48/d69/d7d 0 2024-04-24T08:24:44.137 INFO:tasks.workunit.client.1.smithi149.stdout:9/414: write d1/d2/d15/f2f [976811,3107] 0 2024-04-24T08:24:44.140 INFO:tasks.workunit.client.1.smithi149.stdout:1/460: getdents d0/d5/db/d82 0 2024-04-24T08:24:44.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/90: creat d1a/f24 x:0 0 0 2024-04-24T08:24:44.141 INFO:tasks.workunit.client.1.smithi149.stdout:6/286: chown d0/d1a/d28/d31/c5b 1 1 2024-04-24T08:24:44.144 INFO:tasks.workunit.client.0.smithi090.stdout:0/91: mkdir d1a/d25 0 2024-04-24T08:24:44.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/461: stat d0/d5/db/d18/d34/f4e 0 2024-04-24T08:24:44.149 INFO:tasks.workunit.client.1.smithi149.stdout:4/374: symlink d2/l7e 0 2024-04-24T08:24:44.149 INFO:tasks.workunit.client.1.smithi149.stdout:1/462: stat d0/d5/f41 0 2024-04-24T08:24:44.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/365: dwrite d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:44.151 INFO:tasks.workunit.client.1.smithi149.stdout:6/287: write d0/d1a/d28/d31/f37 [1759020,130998] 0 2024-04-24T08:24:44.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/431: rename d0/d5/d1a/f91 to d0/d5/da/f97 0 2024-04-24T08:24:44.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/92: rename cc to d1a/d25/c26 0 2024-04-24T08:24:44.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/93: write d1a/f1b [5015808,59152] 0 2024-04-24T08:24:44.168 INFO:tasks.workunit.client.1.smithi149.stdout:3/432: link d0/d5/f16 d0/d5/da/d59/d79/f98 0 2024-04-24T08:24:44.170 INFO:tasks.workunit.client.1.smithi149.stdout:2/294: rename d0/l18 to d0/d6/db/d39/d3d/d44/l5d 0 2024-04-24T08:24:44.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/433: chown d0/d26/d3c/l62 7319470 1 2024-04-24T08:24:44.173 INFO:tasks.workunit.client.1.smithi149.stdout:7/366: getdents d9/dc/d11 0 2024-04-24T08:24:44.179 INFO:tasks.workunit.client.1.smithi149.stdout:7/367: truncate d9/dc/d11/f42 6184957 0 2024-04-24T08:24:44.179 INFO:tasks.workunit.client.1.smithi149.stdout:1/463: dwrite d0/d5/db/f1c [0,4194304] 0 2024-04-24T08:24:44.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/422: getdents db/d12 0 2024-04-24T08:24:44.185 INFO:tasks.workunit.client.1.smithi149.stdout:7/368: mknod d9/dc/de/d44/c79 0 2024-04-24T08:24:44.193 INFO:tasks.workunit.client.0.smithi090.stdout:0/94: dwrite f18 [0,4194304] 0 2024-04-24T08:24:44.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/95: dread - d1a/f24 zero size 2024-04-24T08:24:44.194 INFO:tasks.workunit.client.1.smithi149.stdout:3/434: dwrite d0/d5/da/d19/d80/f8c [0,4194304] 0 2024-04-24T08:24:44.195 INFO:tasks.workunit.client.1.smithi149.stdout:2/295: dwrite d0/f4d [0,4194304] 0 2024-04-24T08:24:44.195 INFO:tasks.workunit.client.1.smithi149.stdout:5/423: mkdir db/d12/d22/d6c/d83 0 2024-04-24T08:24:44.201 INFO:tasks.workunit.client.1.smithi149.stdout:7/369: rmdir d9/d43/d4f/d66 39 2024-04-24T08:24:44.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/296: chown d0/d6/db/d39/d3d/d44/d57 19711 1 2024-04-24T08:24:44.206 INFO:tasks.workunit.client.0.smithi090.stdout:0/96: dread f8 [0,4194304] 0 2024-04-24T08:24:44.218 INFO:tasks.workunit.client.1.smithi149.stdout:6/288: dwrite d0/d1a/d28/d31/f37 [4194304,4194304] 0 2024-04-24T08:24:44.220 INFO:tasks.workunit.client.1.smithi149.stdout:5/424: rename db/d35/f59 to db/d12/f84 0 2024-04-24T08:24:44.246 INFO:tasks.workunit.client.1.smithi149.stdout:2/297: write d0/d6/db/d27/f3b [109337,13077] 0 2024-04-24T08:24:44.250 INFO:tasks.workunit.client.1.smithi149.stdout:3/435: write d0/d5/da/f6e [996543,44867] 0 2024-04-24T08:24:44.252 INFO:tasks.workunit.client.1.smithi149.stdout:5/425: mkdir db/d12/d1e/d85 0 2024-04-24T08:24:44.254 INFO:tasks.workunit.client.1.smithi149.stdout:4/375: dwrite d2/d10/d1a/d34/f37 [0,4194304] 0 2024-04-24T08:24:44.255 INFO:tasks.workunit.client.1.smithi149.stdout:7/370: link d9/dc/de/d44/c79 d9/dc/d2e/c7a 0 2024-04-24T08:24:44.257 INFO:tasks.workunit.client.1.smithi149.stdout:1/464: link d0/d39/d53/d61/l8e d0/d5/df/l99 0 2024-04-24T08:24:44.258 INFO:tasks.workunit.client.1.smithi149.stdout:7/371: write d9/dc/d11/d26/f4a [4699985,121276] 0 2024-04-24T08:24:44.260 INFO:tasks.workunit.client.1.smithi149.stdout:2/298: stat d0/d6/db/d39/d3d/d44/l5d 0 2024-04-24T08:24:44.260 INFO:tasks.workunit.client.1.smithi149.stdout:6/289: dread d0/f1 [0,4194304] 0 2024-04-24T08:24:44.260 INFO:tasks.workunit.client.1.smithi149.stdout:3/436: mkdir d0/d26/d3c/d99 0 2024-04-24T08:24:44.261 INFO:tasks.workunit.client.1.smithi149.stdout:1/465: write d0/d5/db/f1c [186626,51044] 0 2024-04-24T08:24:44.262 INFO:tasks.workunit.client.1.smithi149.stdout:0/334: sync 2024-04-24T08:24:44.265 INFO:tasks.workunit.client.1.smithi149.stdout:4/376: read - d2/d24/d2e/d38/f5c zero size 2024-04-24T08:24:44.273 INFO:tasks.workunit.client.1.smithi149.stdout:5/426: rmdir db/d12/d22/d6c 39 2024-04-24T08:24:44.276 INFO:tasks.workunit.client.1.smithi149.stdout:2/299: creat d0/d6/db/d39/d3d/d58/f5e x:0 0 0 2024-04-24T08:24:44.281 INFO:tasks.workunit.client.1.smithi149.stdout:7/372: dread d9/dc/d2e/f41 [0,4194304] 0 2024-04-24T08:24:44.281 INFO:tasks.workunit.client.1.smithi149.stdout:0/335: creat d2/d9/df/d48/f66 x:0 0 0 2024-04-24T08:24:44.287 INFO:tasks.workunit.client.1.smithi149.stdout:4/377: mkdir d2/d24/d7f 0 2024-04-24T08:24:44.288 INFO:tasks.workunit.client.1.smithi149.stdout:0/336: read - d2/d9/df/d56/f5f zero size 2024-04-24T08:24:44.291 INFO:tasks.workunit.client.1.smithi149.stdout:4/378: fdatasync d2/d10/d1a/d34/f37 0 2024-04-24T08:24:44.291 INFO:tasks.workunit.client.1.smithi149.stdout:0/337: write d2/d9/df/d56/f5f [930842,75688] 0 2024-04-24T08:24:44.292 INFO:tasks.workunit.client.1.smithi149.stdout:3/437: rename d0/d5/da/d5a/d63/d6a/f7b to d0/d5/d53/d6d/f9a 0 2024-04-24T08:24:44.294 INFO:tasks.workunit.client.1.smithi149.stdout:6/290: dwrite d0/d1a/d28/d31/d38/f3a [0,4194304] 0 2024-04-24T08:24:44.311 INFO:tasks.workunit.client.1.smithi149.stdout:4/379: dwrite d2/d10/f23 [0,4194304] 0 2024-04-24T08:24:44.319 INFO:tasks.workunit.client.1.smithi149.stdout:6/291: dwrite d0/d1a/d28/d31/d38/f3a [0,4194304] 0 2024-04-24T08:24:44.319 INFO:tasks.workunit.client.1.smithi149.stdout:3/438: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:44.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/439: write d0/d5/d53/f3f [385478,18368] 0 2024-04-24T08:24:44.323 INFO:tasks.workunit.client.1.smithi149.stdout:0/338: read d2/d9/df/f1d [1071070,15971] 0 2024-04-24T08:24:44.325 INFO:tasks.workunit.client.1.smithi149.stdout:5/427: mknod db/d12/d22/c86 0 2024-04-24T08:24:44.327 INFO:tasks.workunit.client.1.smithi149.stdout:1/466: dread d0/d1b/d2a/f7b [0,4194304] 0 2024-04-24T08:24:44.332 INFO:tasks.workunit.client.1.smithi149.stdout:7/373: mkdir d9/dc/d7b 0 2024-04-24T08:24:44.337 INFO:tasks.workunit.client.1.smithi149.stdout:6/292: symlink d0/d1a/d28/d31/d33/l5e 0 2024-04-24T08:24:44.340 INFO:tasks.workunit.client.1.smithi149.stdout:0/339: unlink d2/d9/df/d2f/f3d 0 2024-04-24T08:24:44.347 INFO:tasks.workunit.client.1.smithi149.stdout:7/374: rename d9/dc/f1b to d9/dc/d2e/d51/f7c 0 2024-04-24T08:24:44.350 INFO:tasks.workunit.client.1.smithi149.stdout:7/375: write d9/d43/d4c/f71 [200701,90261] 0 2024-04-24T08:24:44.353 INFO:tasks.workunit.client.1.smithi149.stdout:0/340: creat d2/d9/df/d40/f67 x:0 0 0 2024-04-24T08:24:44.354 INFO:tasks.workunit.client.1.smithi149.stdout:3/440: dread d0/d5/d53/d34/f3b [0,4194304] 0 2024-04-24T08:24:44.358 INFO:tasks.workunit.client.1.smithi149.stdout:4/380: rename d2/d24/d2e/d3f/f4b to d2/d18/d1e/d3c/f80 0 2024-04-24T08:24:44.360 INFO:tasks.workunit.client.1.smithi149.stdout:4/381: dread - d2/d18/d1e/d3c/f43 zero size 2024-04-24T08:24:44.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/97: getdents d1a/d25 0 2024-04-24T08:24:44.364 INFO:tasks.workunit.client.1.smithi149.stdout:5/428: creat db/d12/f87 x:0 0 0 2024-04-24T08:24:44.365 INFO:tasks.workunit.client.1.smithi149.stdout:3/441: unlink d0/d5/da/d19/d1b/d22/f41 0 2024-04-24T08:24:44.368 INFO:tasks.workunit.client.1.smithi149.stdout:1/467: write d0/f16 [8202751,101479] 0 2024-04-24T08:24:44.373 INFO:tasks.workunit.client.1.smithi149.stdout:5/429: unlink db/d1f/l79 0 2024-04-24T08:24:44.375 INFO:tasks.workunit.client.1.smithi149.stdout:5/430: fsync db/d1f/f5c 0 2024-04-24T08:24:44.378 INFO:tasks.workunit.client.1.smithi149.stdout:3/442: creat d0/d5/da/d19/d76/f9b x:0 0 0 2024-04-24T08:24:44.378 INFO:tasks.workunit.client.1.smithi149.stdout:5/431: write db/d12/d22/f33 [8518426,127624] 0 2024-04-24T08:24:44.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/468: unlink d0/d1b/l93 0 2024-04-24T08:24:44.384 INFO:tasks.workunit.client.1.smithi149.stdout:5/432: stat db/d12/d2f/l50 0 2024-04-24T08:24:44.390 INFO:tasks.workunit.client.1.smithi149.stdout:1/469: unlink d0/d1b/l88 0 2024-04-24T08:24:44.391 INFO:tasks.workunit.client.1.smithi149.stdout:6/293: dwrite d0/d1a/f2b [0,4194304] 0 2024-04-24T08:24:44.391 INFO:tasks.workunit.client.1.smithi149.stdout:5/433: mknod db/d12/d2f/d74/c88 0 2024-04-24T08:24:44.398 INFO:tasks.workunit.client.1.smithi149.stdout:4/382: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:44.399 INFO:tasks.workunit.client.1.smithi149.stdout:3/443: dwrite d0/fe [0,4194304] 0 2024-04-24T08:24:44.402 INFO:tasks.workunit.client.1.smithi149.stdout:3/444: fsync d0/d5/d53/d34/d3a/f33 0 2024-04-24T08:24:44.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/376: dwrite d9/dc/d72/f76 [0,4194304] 0 2024-04-24T08:24:44.407 INFO:tasks.workunit.client.1.smithi149.stdout:6/294: rename d0/d1a/d28/d31/l35 to d0/d1a/d28/d31/d38/l5f 0 2024-04-24T08:24:44.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/98: dwrite d1a/f1e [0,4194304] 0 2024-04-24T08:24:44.409 INFO:tasks.workunit.client.1.smithi149.stdout:7/377: chown d9/dc/d2e/d51 5 1 2024-04-24T08:24:44.412 INFO:tasks.workunit.client.0.smithi090.stdout:0/99: truncate d1a/f1e 4963958 0 2024-04-24T08:24:44.417 INFO:tasks.workunit.client.1.smithi149.stdout:9/415: sync 2024-04-24T08:24:44.417 INFO:tasks.workunit.client.1.smithi149.stdout:8/315: sync 2024-04-24T08:24:44.423 INFO:tasks.workunit.client.1.smithi149.stdout:1/470: dwrite d0/d5/d4f/d60/d19/f8b [0,4194304] 0 2024-04-24T08:24:44.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/434: dread db/f1a [0,4194304] 0 2024-04-24T08:24:44.425 INFO:tasks.workunit.client.1.smithi149.stdout:4/383: rename d2/d18/d1e/d3c/f52 to d2/d10/f81 0 2024-04-24T08:24:44.427 INFO:tasks.workunit.client.1.smithi149.stdout:1/471: chown d0/d1b 0 1 2024-04-24T08:24:44.429 INFO:tasks.workunit.client.1.smithi149.stdout:5/435: stat db/d12/d1e/f4f 0 2024-04-24T08:24:44.431 INFO:tasks.workunit.client.1.smithi149.stdout:9/416: unlink d1/da/dd/l16 0 2024-04-24T08:24:44.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/316: rmdir d1/de/d18/d32/d65 39 2024-04-24T08:24:44.433 INFO:tasks.workunit.client.1.smithi149.stdout:1/472: truncate d0/d5/db/d63/f83 751168 0 2024-04-24T08:24:44.439 INFO:tasks.workunit.client.1.smithi149.stdout:9/417: read - d1/da/dd/d79/f56 zero size 2024-04-24T08:24:44.439 INFO:tasks.workunit.client.1.smithi149.stdout:8/317: creat d1/de/d18/f6a x:0 0 0 2024-04-24T08:24:44.444 INFO:tasks.workunit.client.1.smithi149.stdout:3/445: truncate d0/d5/da/d19/d1b/f2c 625701 0 2024-04-24T08:24:44.446 INFO:tasks.workunit.client.1.smithi149.stdout:6/295: rename d0/d1a/l2d to d0/d1a/d28/d31/d38/l60 0 2024-04-24T08:24:44.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/100: dwrite f18 [0,4194304] 0 2024-04-24T08:24:44.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/101: readlink lb 0 2024-04-24T08:24:44.452 INFO:tasks.workunit.client.1.smithi149.stdout:8/318: rename d1/l1c to d1/d1f/d31/l6b 0 2024-04-24T08:24:44.453 INFO:tasks.workunit.client.1.smithi149.stdout:1/473: dread d0/d1b/d4b/f58 [0,4194304] 0 2024-04-24T08:24:44.455 INFO:tasks.workunit.client.1.smithi149.stdout:6/296: mknod d0/d1a/d28/d31/c61 0 2024-04-24T08:24:44.461 INFO:tasks.workunit.client.1.smithi149.stdout:9/418: symlink d1/da/dd/d55/l8f 0 2024-04-24T08:24:44.463 INFO:tasks.workunit.client.1.smithi149.stdout:3/446: fsync d0/d5/da/d19/d1b/d22/d38/f64 0 2024-04-24T08:24:44.465 INFO:tasks.workunit.client.1.smithi149.stdout:6/297: mkdir d0/d1a/d28/d31/d4c/d4e/d62 0 2024-04-24T08:24:44.470 INFO:tasks.workunit.client.1.smithi149.stdout:1/474: truncate d0/d39/d53/d61/f70 547089 0 2024-04-24T08:24:44.474 INFO:tasks.workunit.client.1.smithi149.stdout:3/447: chown d0/d26 201 1 2024-04-24T08:24:44.476 INFO:tasks.workunit.client.0.smithi090.stdout:0/102: write f14 [1723604,7754] 0 2024-04-24T08:24:44.478 INFO:tasks.workunit.client.1.smithi149.stdout:4/384: dread d2/f44 [0,4194304] 0 2024-04-24T08:24:44.481 INFO:tasks.workunit.client.1.smithi149.stdout:9/419: symlink d1/da/d4a/l90 0 2024-04-24T08:24:44.482 INFO:tasks.workunit.client.1.smithi149.stdout:5/436: dwrite db/d12/d22/d45/d5d/f63 [0,4194304] 0 2024-04-24T08:24:44.484 INFO:tasks.workunit.client.1.smithi149.stdout:9/420: stat d1/d2/d34/f78 0 2024-04-24T08:24:44.488 INFO:tasks.workunit.client.0.smithi090.stdout:0/103: dread f18 [0,4194304] 0 2024-04-24T08:24:44.492 INFO:tasks.workunit.client.1.smithi149.stdout:1/475: truncate d0/d1b/d4b/f58 1265986 0 2024-04-24T08:24:44.493 INFO:tasks.workunit.client.1.smithi149.stdout:6/298: write d0/d1a/f29 [3917106,103241] 0 2024-04-24T08:24:44.493 INFO:tasks.workunit.client.1.smithi149.stdout:9/421: write d1/de/d12/f1d [3609143,104438] 0 2024-04-24T08:24:44.494 INFO:tasks.workunit.client.1.smithi149.stdout:7/378: dread d9/dc/de/f12 [0,4194304] 0 2024-04-24T08:24:44.497 INFO:tasks.workunit.client.0.smithi090.stdout:0/104: link f13 d1a/d25/f27 0 2024-04-24T08:24:44.500 INFO:tasks.workunit.client.0.smithi090.stdout:0/105: write d1a/f20 [347049,124635] 0 2024-04-24T08:24:44.501 INFO:tasks.workunit.client.1.smithi149.stdout:9/422: chown d1/d2/d15/d1c/d42 4 1 2024-04-24T08:24:44.503 INFO:tasks.workunit.client.1.smithi149.stdout:6/299: rmdir d0 39 2024-04-24T08:24:44.506 INFO:tasks.workunit.client.1.smithi149.stdout:5/437: mknod db/d12/d22/d6c/d83/c89 0 2024-04-24T08:24:44.509 INFO:tasks.workunit.client.0.smithi090.stdout:0/106: truncate f8 3104303 0 2024-04-24T08:24:44.510 INFO:tasks.workunit.client.1.smithi149.stdout:8/319: dwrite d1/d1f/f68 [0,4194304] 0 2024-04-24T08:24:44.510 INFO:tasks.workunit.client.1.smithi149.stdout:8/320: fdatasync d1/de/f2f 0 2024-04-24T08:24:44.510 INFO:tasks.workunit.client.1.smithi149.stdout:3/448: dread d0/d5/ff [0,4194304] 0 2024-04-24T08:24:44.513 INFO:tasks.workunit.client.1.smithi149.stdout:4/385: link d2/d10/c11 d2/c82 0 2024-04-24T08:24:44.513 INFO:tasks.workunit.client.1.smithi149.stdout:1/476: creat d0/d1b/d5d/f9a x:0 0 0 2024-04-24T08:24:44.515 INFO:tasks.workunit.client.1.smithi149.stdout:2/300: sync 2024-04-24T08:24:44.519 INFO:tasks.workunit.client.1.smithi149.stdout:5/438: creat db/d1f/f8a x:0 0 0 2024-04-24T08:24:44.522 INFO:tasks.workunit.client.1.smithi149.stdout:8/321: mkdir d1/d1f/d5e/d6c 0 2024-04-24T08:24:44.523 INFO:tasks.workunit.client.1.smithi149.stdout:3/449: fsync d0/d5/d53/d34/f3b 0 2024-04-24T08:24:44.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/107: getdents d1a/d25 0 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/4: sync 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/10: sync 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/11: write - no filename 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/12: stat - no entries 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/13: rmdir - no directory 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/14: link - no file 2024-04-24T08:24:44.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/15: rename - no filename 2024-04-24T08:24:44.530 INFO:tasks.workunit.client.0.smithi090.stdout:0/108: rename c19 to d1a/d25/c28 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:6/300: creat d0/d1a/d28/d31/d38/f63 x:0 0 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:8/322: dread - d1/d1f/d31/f35 zero size 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:7/379: dwrite d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:3/450: write d0/d5/d53/d34/d3a/f36 [562606,55711] 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:9/423: dwrite d1/da/dd/d79/f56 [0,4194304] 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:7/380: write d9/dc/d11/d38/f4e [2926293,40186] 0 2024-04-24T08:24:44.534 INFO:tasks.workunit.client.1.smithi149.stdout:8/323: write d1/d1f/f28 [2035669,6670] 0 2024-04-24T08:24:44.536 INFO:tasks.workunit.client.0.smithi090.stdout:4/16: creat f0 x:0 0 0 2024-04-24T08:24:44.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/477: creat d0/d5/db/d82/f9b x:0 0 0 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:4/17: write f0 [671675,76836] 0 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/5: getdents . 0 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/6: readlink - no filename 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/7: dread - no filename 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:4/18: readlink - no filename 2024-04-24T08:24:44.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/8: dwrite - no filename 2024-04-24T08:24:44.539 INFO:tasks.workunit.client.1.smithi149.stdout:9/424: chown d1/d2/d15/d1c/f3a 3438 1 2024-04-24T08:24:44.539 INFO:tasks.workunit.client.1.smithi149.stdout:5/439: chown db/d12/c38 3693440 1 2024-04-24T08:24:44.539 INFO:tasks.workunit.client.1.smithi149.stdout:2/301: creat d0/d6/db/d27/f5f x:0 0 0 2024-04-24T08:24:44.540 INFO:tasks.workunit.client.0.smithi090.stdout:4/19: truncate f0 762372 0 2024-04-24T08:24:44.540 INFO:tasks.workunit.client.0.smithi090.stdout:0/109: rename d1a/d25/c26 to d1a/d25/c29 0 2024-04-24T08:24:44.541 INFO:tasks.workunit.client.1.smithi149.stdout:5/440: readlink db/d12/d2f/l50 0 2024-04-24T08:24:44.543 INFO:tasks.workunit.client.0.smithi090.stdout:0/110: truncate d1a/f20 1224083 0 2024-04-24T08:24:44.548 INFO:tasks.workunit.client.0.smithi090.stdout:9/9: mkdir d0 0 2024-04-24T08:24:44.552 INFO:tasks.workunit.client.0.smithi090.stdout:4/20: rename f0 to f1 0 2024-04-24T08:24:44.553 INFO:tasks.workunit.client.0.smithi090.stdout:4/21: chown f1 26225 1 2024-04-24T08:24:44.554 INFO:tasks.workunit.client.1.smithi149.stdout:1/478: symlink d0/d5/db/d18/l9c 0 2024-04-24T08:24:44.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/22: truncate f1 804821 0 2024-04-24T08:24:44.558 INFO:tasks.workunit.client.1.smithi149.stdout:4/386: rename d2/l26 to d2/d18/d1e/d3c/d48/l83 0 2024-04-24T08:24:44.561 INFO:tasks.workunit.client.1.smithi149.stdout:7/381: creat d9/dc/d7b/f7d x:0 0 0 2024-04-24T08:24:44.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/387: chown d2/l7e 311662698 1 2024-04-24T08:24:44.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/324: dwrite d1/d1f/d31/f35 [0,4194304] 0 2024-04-24T08:24:44.564 INFO:tasks.workunit.client.1.smithi149.stdout:1/479: rename d0/d1b/c66 to d0/d5/db/d82/c9d 0 2024-04-24T08:24:44.565 INFO:tasks.workunit.client.1.smithi149.stdout:2/302: link d0/c56 d0/d6/db/d28/c60 0 2024-04-24T08:24:44.570 INFO:tasks.workunit.client.0.smithi090.stdout:0/111: rename d1a/d25 to d1a/d2a 0 2024-04-24T08:24:44.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/23: fdatasync f1 0 2024-04-24T08:24:44.572 INFO:tasks.workunit.client.0.smithi090.stdout:0/112: write d1a/f1d [5110683,17691] 0 2024-04-24T08:24:44.577 INFO:tasks.workunit.client.0.smithi090.stdout:4/24: write f1 [1244736,62183] 0 2024-04-24T08:24:44.577 INFO:tasks.workunit.client.1.smithi149.stdout:3/451: dwrite d0/d5/da/f97 [0,4194304] 0 2024-04-24T08:24:44.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/113: fsync d1a/d2a/f27 0 2024-04-24T08:24:44.578 INFO:tasks.workunit.client.1.smithi149.stdout:1/480: fdatasync d0/d5/db/f87 0 2024-04-24T08:24:44.583 INFO:tasks.workunit.client.0.smithi090.stdout:4/25: unlink f1 0 2024-04-24T08:24:44.583 INFO:tasks.workunit.client.1.smithi149.stdout:1/481: dread - d0/d5/d4f/d60/f8f zero size 2024-04-24T08:24:44.591 INFO:tasks.workunit.client.0.smithi090.stdout:4/26: symlink l2 0 2024-04-24T08:24:44.592 INFO:tasks.workunit.client.1.smithi149.stdout:2/303: creat d0/d6/db/d39/d3d/d44/f61 x:0 0 0 2024-04-24T08:24:44.593 INFO:tasks.workunit.client.1.smithi149.stdout:5/441: write db/d12/f3d [682137,69830] 0 2024-04-24T08:24:44.594 INFO:tasks.workunit.client.1.smithi149.stdout:7/382: dwrite d9/dc/f35 [0,4194304] 0 2024-04-24T08:24:44.599 INFO:tasks.workunit.client.1.smithi149.stdout:7/383: readlink d9/dc/l25 0 2024-04-24T08:24:44.600 INFO:tasks.workunit.client.0.smithi090.stdout:4/27: mkdir d3 0 2024-04-24T08:24:44.603 INFO:tasks.workunit.client.1.smithi149.stdout:4/388: dread d2/d10/f23 [0,4194304] 0 2024-04-24T08:24:44.604 INFO:tasks.workunit.client.1.smithi149.stdout:1/482: rename d0/d5/db/d18/f2c to d0/d5/d4f/d60/d19/d5c/f9e 0 2024-04-24T08:24:44.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/452: dwrite d0/d5/d1a/f2d [0,4194304] 0 2024-04-24T08:24:44.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/28: rename l2 to d3/l4 0 2024-04-24T08:24:44.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/29: dwrite - no filename 2024-04-24T08:24:44.610 INFO:tasks.workunit.client.0.smithi090.stdout:4/30: rename d3 to d3/d5 22 2024-04-24T08:24:44.611 INFO:tasks.workunit.client.1.smithi149.stdout:2/304: symlink d0/d6/db/d39/l62 0 2024-04-24T08:24:44.619 INFO:tasks.workunit.client.1.smithi149.stdout:7/384: unlink d9/dc/de/c46 0 2024-04-24T08:24:44.622 INFO:tasks.workunit.client.1.smithi149.stdout:7/385: write d9/dc/f17 [5172392,21318] 0 2024-04-24T08:24:44.625 INFO:tasks.workunit.client.0.smithi090.stdout:0/114: dwrite f8 [0,4194304] 0 2024-04-24T08:24:44.630 INFO:tasks.workunit.client.0.smithi090.stdout:0/115: read d1a/f1d [3752242,7619] 0 2024-04-24T08:24:44.633 INFO:tasks.workunit.client.0.smithi090.stdout:0/116: write d1a/f20 [2038130,96263] 0 2024-04-24T08:24:44.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/117: dwrite f12 [0,4194304] 0 2024-04-24T08:24:44.663 INFO:tasks.workunit.client.0.smithi090.stdout:0/118: rename lb to d1a/d2a/l2b 0 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:3/21: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:1/52: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:8/9: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:2/30: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:6/88: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:7/79: sync 2024-04-24T08:24:44.687 INFO:tasks.workunit.client.0.smithi090.stdout:5/74: sync 2024-04-24T08:24:44.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/10: fsync d2/f3 0 2024-04-24T08:24:44.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/31: unlink d0/f5 0 2024-04-24T08:24:44.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/341: sync 2024-04-24T08:24:44.695 INFO:tasks.workunit.client.0.smithi090.stdout:7/80: mkdir db/d20 0 2024-04-24T08:24:44.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/11: symlink d2/l5 0 2024-04-24T08:24:44.699 INFO:tasks.workunit.client.1.smithi149.stdout:5/442: write db/f20 [45120,59649] 0 2024-04-24T08:24:44.700 INFO:tasks.workunit.client.1.smithi149.stdout:1/483: fsync d0/da/f24 0 2024-04-24T08:24:44.701 INFO:tasks.workunit.client.1.smithi149.stdout:4/389: read - d2/d24/d2e/f5b zero size 2024-04-24T08:24:44.702 INFO:tasks.workunit.client.0.smithi090.stdout:7/81: rename db/ff to db/f21 0 2024-04-24T08:24:44.706 INFO:tasks.workunit.client.0.smithi090.stdout:8/12: rename l0 to d2/l6 0 2024-04-24T08:24:44.706 INFO:tasks.workunit.client.1.smithi149.stdout:4/390: chown d2/d24/d2e/d38/f7a 10590555 1 2024-04-24T08:24:44.708 INFO:tasks.workunit.client.0.smithi090.stdout:7/82: chown db/d17/l1c 62 1 2024-04-24T08:24:44.710 INFO:tasks.workunit.client.0.smithi090.stdout:8/13: write d2/f3 [2491261,18985] 0 2024-04-24T08:24:44.711 INFO:tasks.workunit.client.1.smithi149.stdout:1/484: symlink d0/d5/db/d63/d98/l9f 0 2024-04-24T08:24:44.712 INFO:tasks.workunit.client.1.smithi149.stdout:5/443: creat db/d1f/d7f/f8b x:0 0 0 2024-04-24T08:24:44.714 INFO:tasks.workunit.client.1.smithi149.stdout:1/485: readlink d0/d1b/l40 0 2024-04-24T08:24:44.714 INFO:tasks.workunit.client.1.smithi149.stdout:8/325: dread d1/de/d18/d43/f55 [0,4194304] 0 2024-04-24T08:24:44.715 INFO:tasks.workunit.client.1.smithi149.stdout:0/342: mkdir d2/d17/d41/d68 0 2024-04-24T08:24:44.717 INFO:tasks.workunit.client.1.smithi149.stdout:8/326: dread - d1/d1f/f69 zero size 2024-04-24T08:24:44.717 INFO:tasks.workunit.client.0.smithi090.stdout:6/89: dread f0 [0,4194304] 0 2024-04-24T08:24:44.717 INFO:tasks.workunit.client.0.smithi090.stdout:2/32: getdents d0 0 2024-04-24T08:24:44.718 INFO:tasks.workunit.client.0.smithi090.stdout:2/33: write - no filename 2024-04-24T08:24:44.718 INFO:tasks.workunit.client.0.smithi090.stdout:2/34: write - no filename 2024-04-24T08:24:44.718 INFO:tasks.workunit.client.0.smithi090.stdout:2/35: truncate - no filename 2024-04-24T08:24:44.718 INFO:tasks.workunit.client.1.smithi149.stdout:7/386: link d9/d43/d4f/d66/f6e d9/d43/f7e 0 2024-04-24T08:24:44.719 INFO:tasks.workunit.client.1.smithi149.stdout:0/343: chown d2/d9/df/d2f/c31 43239121 1 2024-04-24T08:24:44.721 INFO:tasks.workunit.client.1.smithi149.stdout:5/444: fdatasync db/d12/f69 0 2024-04-24T08:24:44.723 INFO:tasks.workunit.client.0.smithi090.stdout:8/14: creat d2/f7 x:0 0 0 2024-04-24T08:24:44.724 INFO:tasks.workunit.client.0.smithi090.stdout:6/90: write d8/f15 [1371885,100820] 0 2024-04-24T08:24:44.729 INFO:tasks.workunit.client.1.smithi149.stdout:4/391: dwrite d2/d10/d1a/f36 [0,4194304] 0 2024-04-24T08:24:44.729 INFO:tasks.workunit.client.1.smithi149.stdout:5/445: truncate db/d23/f43 440043 0 2024-04-24T08:24:44.729 INFO:tasks.workunit.client.1.smithi149.stdout:0/344: mkdir d2/d9/d69 0 2024-04-24T08:24:44.733 INFO:tasks.workunit.client.0.smithi090.stdout:5/75: dwrite f4 [4194304,4194304] 0 2024-04-24T08:24:44.734 INFO:tasks.workunit.client.1.smithi149.stdout:2/305: dread d0/d6/db/f10 [0,4194304] 0 2024-04-24T08:24:44.736 INFO:tasks.workunit.client.1.smithi149.stdout:1/486: rename d0/d5/d4f/d60/d19/d5c to d0/d39/da0 0 2024-04-24T08:24:44.737 INFO:tasks.workunit.client.0.smithi090.stdout:3/22: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:44.737 INFO:tasks.workunit.client.0.smithi090.stdout:6/91: creat d8/f1b x:0 0 0 2024-04-24T08:24:44.740 INFO:tasks.workunit.client.1.smithi149.stdout:5/446: rmdir db 39 2024-04-24T08:24:44.741 INFO:tasks.workunit.client.0.smithi090.stdout:7/83: link db/c14 db/d17/d1b/c22 0 2024-04-24T08:24:44.748 INFO:tasks.workunit.client.1.smithi149.stdout:4/392: creat d2/d18/d1e/d3c/f84 x:0 0 0 2024-04-24T08:24:44.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/76: dread f7 [0,4194304] 0 2024-04-24T08:24:44.753 INFO:tasks.workunit.client.1.smithi149.stdout:0/345: symlink d2/d9/df/d56/l6a 0 2024-04-24T08:24:44.755 INFO:tasks.workunit.client.1.smithi149.stdout:8/327: dread d1/f9 [0,4194304] 0 2024-04-24T08:24:44.758 INFO:tasks.workunit.client.1.smithi149.stdout:0/346: write d2/d9/d1a/f3f [388360,55552] 0 2024-04-24T08:24:44.763 INFO:tasks.workunit.client.1.smithi149.stdout:1/487: mkdir d0/d1b/d4b/da1 0 2024-04-24T08:24:44.767 INFO:tasks.workunit.client.1.smithi149.stdout:5/447: fsync db/d12/f3d 0 2024-04-24T08:24:44.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/84: creat db/d17/d1b/f23 x:0 0 0 2024-04-24T08:24:44.769 INFO:tasks.workunit.client.1.smithi149.stdout:7/387: link d9/dc/d2e/c7a d9/dc/d2e/c7f 0 2024-04-24T08:24:44.771 INFO:tasks.workunit.client.1.smithi149.stdout:5/448: write db/d12/d2f/f46 [2238667,103605] 0 2024-04-24T08:24:44.773 INFO:tasks.workunit.client.1.smithi149.stdout:4/393: mknod d2/d18/d1e/d3c/d48/d69/c85 0 2024-04-24T08:24:44.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/449: fdatasync db/d1f/f30 0 2024-04-24T08:24:44.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/77: creat d8/da/f1b x:0 0 0 2024-04-24T08:24:44.775 INFO:tasks.workunit.client.0.smithi090.stdout:7/85: dread db/f10 [0,4194304] 0 2024-04-24T08:24:44.775 INFO:tasks.workunit.client.0.smithi090.stdout:6/92: symlink d8/dc/d19/l1c 0 2024-04-24T08:24:44.776 INFO:tasks.workunit.client.0.smithi090.stdout:7/86: dread - db/f1a zero size 2024-04-24T08:24:44.779 INFO:tasks.workunit.client.1.smithi149.stdout:1/488: truncate d0/d1b/f64 1114822 0 2024-04-24T08:24:44.783 INFO:tasks.workunit.client.1.smithi149.stdout:7/388: mknod d9/dc/d7b/c80 0 2024-04-24T08:24:44.784 INFO:tasks.workunit.client.0.smithi090.stdout:8/15: link d2/l6 d2/l8 0 2024-04-24T08:24:44.787 INFO:tasks.workunit.client.1.smithi149.stdout:5/450: fdatasync db/d12/f5b 0 2024-04-24T08:24:44.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/23: getdents d2 0 2024-04-24T08:24:44.793 INFO:tasks.workunit.client.0.smithi090.stdout:8/16: creat d2/f9 x:0 0 0 2024-04-24T08:24:44.794 INFO:tasks.workunit.client.1.smithi149.stdout:4/394: truncate d2/d10/d1a/d34/f4c 3437548 0 2024-04-24T08:24:44.795 INFO:tasks.workunit.client.0.smithi090.stdout:5/78: write d8/da/fd [334745,32932] 0 2024-04-24T08:24:44.796 INFO:tasks.workunit.client.0.smithi090.stdout:8/17: stat l1 0 2024-04-24T08:24:44.802 INFO:tasks.workunit.client.1.smithi149.stdout:8/328: truncate d1/d1f/f68 2477169 0 2024-04-24T08:24:44.803 INFO:tasks.workunit.client.0.smithi090.stdout:7/87: getdents db 0 2024-04-24T08:24:44.805 INFO:tasks.workunit.client.1.smithi149.stdout:5/451: dwrite db/f20 [0,4194304] 0 2024-04-24T08:24:44.806 INFO:tasks.workunit.client.0.smithi090.stdout:5/79: creat d8/f1c x:0 0 0 2024-04-24T08:24:44.809 INFO:tasks.workunit.client.1.smithi149.stdout:4/395: fsync d2/d24/f39 0 2024-04-24T08:24:44.812 INFO:tasks.workunit.client.0.smithi090.stdout:6/93: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:44.813 INFO:tasks.workunit.client.0.smithi090.stdout:5/80: write f4 [583947,109086] 0 2024-04-24T08:24:44.814 INFO:tasks.workunit.client.1.smithi149.stdout:8/329: truncate d1/de/d18/f48 1601854 0 2024-04-24T08:24:44.815 INFO:tasks.workunit.client.1.smithi149.stdout:0/347: dwrite d2/d9/d1a/d36/f47 [0,4194304] 0 2024-04-24T08:24:44.816 INFO:tasks.workunit.client.1.smithi149.stdout:2/306: link d0/d6/db/l4b d0/d6/db/d39/l63 0 2024-04-24T08:24:44.821 INFO:tasks.workunit.client.1.smithi149.stdout:5/452: mknod db/d12/d22/d6c/d83/c8c 0 2024-04-24T08:24:44.823 INFO:tasks.workunit.client.1.smithi149.stdout:5/453: dread - db/d1f/f8a zero size 2024-04-24T08:24:44.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/330: truncate d1/de/f47 663357 0 2024-04-24T08:24:44.828 INFO:tasks.workunit.client.0.smithi090.stdout:5/81: write d8/da/f19 [568359,126915] 0 2024-04-24T08:24:44.829 INFO:tasks.workunit.client.1.smithi149.stdout:8/331: fdatasync d1/de/d18/d43/f55 0 2024-04-24T08:24:44.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/24: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:44.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/18: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:44.842 INFO:tasks.workunit.client.1.smithi149.stdout:6/301: sync 2024-04-24T08:24:44.845 INFO:tasks.workunit.client.1.smithi149.stdout:0/348: rename d2/d17/f23 to d2/d9/d1a/f6b 0 2024-04-24T08:24:44.849 INFO:tasks.workunit.client.0.smithi090.stdout:6/94: dwrite d8/dc/f11 [0,4194304] 0 2024-04-24T08:24:44.854 INFO:tasks.workunit.client.1.smithi149.stdout:1/489: truncate d0/d39/f7c 2307500 0 2024-04-24T08:24:44.857 INFO:tasks.workunit.client.1.smithi149.stdout:7/389: write d9/dc/de/d44/f74 [1040612,87218] 0 2024-04-24T08:24:44.858 INFO:tasks.workunit.client.0.smithi090.stdout:5/82: creat d8/da/f1d x:0 0 0 2024-04-24T08:24:44.858 INFO:tasks.workunit.client.0.smithi090.stdout:3/25: rmdir d2 39 2024-04-24T08:24:44.858 INFO:tasks.workunit.client.0.smithi090.stdout:3/26: stat c1 0 2024-04-24T08:24:44.860 INFO:tasks.workunit.client.1.smithi149.stdout:5/454: dwrite db/d12/f87 [0,4194304] 0 2024-04-24T08:24:44.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/19: mkdir d2/da 0 2024-04-24T08:24:44.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/349: mkdir d2/d9/df/d2f/d6c 0 2024-04-24T08:24:44.869 INFO:tasks.workunit.client.0.smithi090.stdout:7/88: truncate f4 3536795 0 2024-04-24T08:24:44.870 INFO:tasks.workunit.client.1.smithi149.stdout:0/350: chown d2/d3c/f61 26657949 1 2024-04-24T08:24:44.870 INFO:tasks.workunit.client.0.smithi090.stdout:7/89: chown db/d17 333744 1 2024-04-24T08:24:44.872 INFO:tasks.workunit.client.1.smithi149.stdout:8/332: creat d1/de/f6d x:0 0 0 2024-04-24T08:24:44.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/351: write d2/d9/df/f1d [681431,84351] 0 2024-04-24T08:24:44.875 INFO:tasks.workunit.client.0.smithi090.stdout:3/27: dwrite f0 [0,4194304] 0 2024-04-24T08:24:44.876 INFO:tasks.workunit.client.1.smithi149.stdout:4/396: truncate d2/d18/f1b 2428292 0 2024-04-24T08:24:44.876 INFO:tasks.workunit.client.0.smithi090.stdout:3/28: chown f0 0 1 2024-04-24T08:24:44.878 INFO:tasks.workunit.client.1.smithi149.stdout:1/490: creat d0/d5/db/d18/d34/fa2 x:0 0 0 2024-04-24T08:24:44.879 INFO:tasks.workunit.client.1.smithi149.stdout:2/307: write d0/d6/db/d28/f3a [2117518,129484] 0 2024-04-24T08:24:44.880 INFO:tasks.workunit.client.1.smithi149.stdout:0/352: truncate d2/d3c/f61 409446 0 2024-04-24T08:24:44.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/31: sync 2024-04-24T08:24:44.880 INFO:tasks.workunit.client.0.smithi090.stdout:9/10: sync 2024-04-24T08:24:44.880 INFO:tasks.workunit.client.0.smithi090.stdout:9/11: write - no filename 2024-04-24T08:24:44.880 INFO:tasks.workunit.client.0.smithi090.stdout:0/119: sync 2024-04-24T08:24:44.881 INFO:tasks.workunit.client.0.smithi090.stdout:4/32: chown d3 998 1 2024-04-24T08:24:44.881 INFO:tasks.workunit.client.0.smithi090.stdout:4/33: write - no filename 2024-04-24T08:24:44.885 INFO:tasks.workunit.client.1.smithi149.stdout:5/455: dwrite db/d12/d22/d6c/f6d [0,4194304] 0 2024-04-24T08:24:44.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/390: mknod d9/dc/d11/c81 0 2024-04-24T08:24:44.889 INFO:tasks.workunit.client.1.smithi149.stdout:0/353: chown d2/d9/f5a 85411 1 2024-04-24T08:24:44.889 INFO:tasks.workunit.client.0.smithi090.stdout:5/83: creat d8/da/d18/f1e x:0 0 0 2024-04-24T08:24:44.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/84: chown d8/da/f1d 1 1 2024-04-24T08:24:44.891 INFO:tasks.workunit.client.1.smithi149.stdout:9/425: sync 2024-04-24T08:24:44.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/453: sync 2024-04-24T08:24:44.892 INFO:tasks.workunit.client.0.smithi090.stdout:5/85: write d8/da/d18/f1e [477826,102315] 0 2024-04-24T08:24:44.894 INFO:tasks.workunit.client.1.smithi149.stdout:9/426: fsync d1/da/d13/f89 0 2024-04-24T08:24:44.900 INFO:tasks.workunit.client.0.smithi090.stdout:7/90: unlink db/d17/d1b/c22 0 2024-04-24T08:24:44.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/91: write db/f12 [1830993,19826] 0 2024-04-24T08:24:44.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/120: creat d1a/f2c x:0 0 0 2024-04-24T08:24:44.907 INFO:tasks.workunit.client.0.smithi090.stdout:4/34: creat d3/f6 x:0 0 0 2024-04-24T08:24:44.908 INFO:tasks.workunit.client.1.smithi149.stdout:1/491: creat d0/da/fa3 x:0 0 0 2024-04-24T08:24:44.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/86: creat d8/da/d18/f1f x:0 0 0 2024-04-24T08:24:44.911 INFO:tasks.workunit.client.0.smithi090.stdout:3/29: rename c1 to d2/c4 0 2024-04-24T08:24:44.916 INFO:tasks.workunit.client.1.smithi149.stdout:3/454: mkdir d0/d5/da/d5a/d9c 0 2024-04-24T08:24:44.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/12: symlink d0/l1 0 2024-04-24T08:24:44.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/427: dwrite d1/da/dd/f69 [0,4194304] 0 2024-04-24T08:24:44.920 INFO:tasks.workunit.client.0.smithi090.stdout:4/35: creat d3/f7 x:0 0 0 2024-04-24T08:24:44.920 INFO:tasks.workunit.client.1.smithi149.stdout:8/333: creat d1/d1f/d5e/d6c/f6e x:0 0 0 2024-04-24T08:24:44.922 INFO:tasks.workunit.client.0.smithi090.stdout:4/36: read - d3/f7 zero size 2024-04-24T08:24:44.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/92: dwrite fa [0,4194304] 0 2024-04-24T08:24:44.926 INFO:tasks.workunit.client.0.smithi090.stdout:4/37: dread - d3/f6 zero size 2024-04-24T08:24:44.927 INFO:tasks.workunit.client.0.smithi090.stdout:0/121: dread f14 [0,4194304] 0 2024-04-24T08:24:44.927 INFO:tasks.workunit.client.1.smithi149.stdout:1/492: rename d0/d5/db/f1c to d0/d5/db/d18/d2f/fa4 0 2024-04-24T08:24:44.927 INFO:tasks.workunit.client.0.smithi090.stdout:5/87: dread f4 [4194304,4194304] 0 2024-04-24T08:24:44.928 INFO:tasks.workunit.client.1.smithi149.stdout:5/456: mknod db/d12/d2f/d78/c8d 0 2024-04-24T08:24:44.929 INFO:tasks.workunit.client.1.smithi149.stdout:0/354: creat d2/d9/df/d40/d4a/d4e/f6d x:0 0 0 2024-04-24T08:24:44.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/30: dread d2/f3 [0,4194304] 0 2024-04-24T08:24:44.934 INFO:tasks.workunit.client.1.smithi149.stdout:9/428: fsync d1/d2/d15/f23 0 2024-04-24T08:24:44.935 INFO:tasks.workunit.client.0.smithi090.stdout:4/38: symlink d3/l8 0 2024-04-24T08:24:44.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/334: creat d1/de/d18/f6f x:0 0 0 2024-04-24T08:24:44.938 INFO:tasks.workunit.client.1.smithi149.stdout:5/457: mknod db/d12/d2f/c8e 0 2024-04-24T08:24:44.939 INFO:tasks.workunit.client.1.smithi149.stdout:8/335: write d1/de/f2b [3134463,15103] 0 2024-04-24T08:24:44.939 INFO:tasks.workunit.client.0.smithi090.stdout:3/31: rmdir d2 39 2024-04-24T08:24:44.940 INFO:tasks.workunit.client.0.smithi090.stdout:0/122: dread f18 [0,4194304] 0 2024-04-24T08:24:44.941 INFO:tasks.workunit.client.1.smithi149.stdout:1/493: symlink d0/d1b/la5 0 2024-04-24T08:24:44.942 INFO:tasks.workunit.client.1.smithi149.stdout:8/336: readlink d1/d1f/d5e/l62 0 2024-04-24T08:24:44.942 INFO:tasks.workunit.client.0.smithi090.stdout:0/123: chown d1a/f2c 25290 1 2024-04-24T08:24:44.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/39: creat d3/f9 x:0 0 0 2024-04-24T08:24:44.944 INFO:tasks.workunit.client.1.smithi149.stdout:9/429: fsync d1/d2/d15/d1c/f3a 0 2024-04-24T08:24:44.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/93: rmdir db/d20 0 2024-04-24T08:24:44.948 INFO:tasks.workunit.client.1.smithi149.stdout:2/308: dwrite d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:44.948 INFO:tasks.workunit.client.0.smithi090.stdout:0/124: symlink d1a/l2d 0 2024-04-24T08:24:44.951 INFO:tasks.workunit.client.1.smithi149.stdout:9/430: dread d1/d2/d33/f4e [0,4194304] 0 2024-04-24T08:24:44.951 INFO:tasks.workunit.client.1.smithi149.stdout:2/309: readlink d0/d6/db/l32 0 2024-04-24T08:24:44.952 INFO:tasks.workunit.client.0.smithi090.stdout:5/88: dwrite f2 [0,4194304] 0 2024-04-24T08:24:44.954 INFO:tasks.workunit.client.1.smithi149.stdout:2/310: readlink d0/l40 0 2024-04-24T08:24:44.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/494: unlink d0/d5/db/d18/d34/c6d 0 2024-04-24T08:24:44.957 INFO:tasks.workunit.client.0.smithi090.stdout:0/125: symlink d1a/l2e 0 2024-04-24T08:24:44.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/355: getdents d2/d17/d41 0 2024-04-24T08:24:44.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/311: stat d0/d6/c23 0 2024-04-24T08:24:44.966 INFO:tasks.workunit.client.0.smithi090.stdout:0/126: truncate d1a/d2a/f27 1179109 0 2024-04-24T08:24:44.966 INFO:tasks.workunit.client.0.smithi090.stdout:7/94: link db/d17/c1f db/c24 0 2024-04-24T08:24:44.968 INFO:tasks.workunit.client.1.smithi149.stdout:0/356: symlink d2/d3c/l6e 0 2024-04-24T08:24:44.968 INFO:tasks.workunit.client.0.smithi090.stdout:7/95: truncate db/fd 1259187 0 2024-04-24T08:24:44.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/312: mkdir d0/d6/db/d27/d64 0 2024-04-24T08:24:44.971 INFO:tasks.workunit.client.0.smithi090.stdout:5/89: getdents d8 0 2024-04-24T08:24:44.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/357: write d2/d9/df/d2f/f30 [291490,22320] 0 2024-04-24T08:24:44.972 INFO:tasks.workunit.client.0.smithi090.stdout:7/96: symlink db/d17/d1b/l25 0 2024-04-24T08:24:44.975 INFO:tasks.workunit.client.1.smithi149.stdout:2/313: write d0/d6/db/f3e [172215,117246] 0 2024-04-24T08:24:44.975 INFO:tasks.workunit.client.0.smithi090.stdout:5/90: creat d8/da/f20 x:0 0 0 2024-04-24T08:24:44.977 INFO:tasks.workunit.client.1.smithi149.stdout:1/495: dwrite d0/da/fa3 [0,4194304] 0 2024-04-24T08:24:44.979 INFO:tasks.workunit.client.0.smithi090.stdout:5/91: write d8/da/d14/f17 [444544,79800] 0 2024-04-24T08:24:44.981 INFO:tasks.workunit.client.1.smithi149.stdout:2/314: unlink d0/c3 0 2024-04-24T08:24:44.982 INFO:tasks.workunit.client.1.smithi149.stdout:0/358: mkdir d2/d9/df/d40/d4a/d4e/d6f 0 2024-04-24T08:24:44.984 INFO:tasks.workunit.client.1.smithi149.stdout:2/315: write d0/d6/f41 [911268,57271] 0 2024-04-24T08:24:44.984 INFO:tasks.workunit.client.1.smithi149.stdout:0/359: stat d2/d9/df/f1b 0 2024-04-24T08:24:44.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/127: dwrite d1a/f1e [0,4194304] 0 2024-04-24T08:24:44.986 INFO:tasks.workunit.client.1.smithi149.stdout:1/496: link d0/d5/db/f87 d0/d5/db/d63/fa6 0 2024-04-24T08:24:44.988 INFO:tasks.workunit.client.0.smithi090.stdout:0/128: chown d1a/l2d 297612 1 2024-04-24T08:24:44.989 INFO:tasks.workunit.client.1.smithi149.stdout:0/360: creat d2/d9/df/d40/f70 x:0 0 0 2024-04-24T08:24:44.993 INFO:tasks.workunit.client.1.smithi149.stdout:0/361: symlink d2/d9/df/d40/d4a/l71 0 2024-04-24T08:24:44.994 INFO:tasks.workunit.client.1.smithi149.stdout:0/362: fsync d2/fa 0 2024-04-24T08:24:44.999 INFO:tasks.workunit.client.1.smithi149.stdout:2/316: dread d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:45.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/97: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:45.005 INFO:tasks.workunit.client.0.smithi090.stdout:0/129: link d1a/c1c d1a/c2f 0 2024-04-24T08:24:45.005 INFO:tasks.workunit.client.1.smithi149.stdout:2/317: rename d0/d6/db/d28/c60 to d0/d6/db/d28/c65 0 2024-04-24T08:24:45.008 INFO:tasks.workunit.client.1.smithi149.stdout:2/318: mkdir d0/d6/db/d27/d64/d66 0 2024-04-24T08:24:45.010 INFO:tasks.workunit.client.0.smithi090.stdout:5/92: dwrite d8/da/f19 [0,4194304] 0 2024-04-24T08:24:45.012 INFO:tasks.workunit.client.0.smithi090.stdout:5/93: read d8/f15 [5122057,37507] 0 2024-04-24T08:24:45.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/319: rename d0/d6/db/d27/l3c to d0/d6/db/d27/l67 0 2024-04-24T08:24:45.014 INFO:tasks.workunit.client.0.smithi090.stdout:5/94: write d8/da/f1d [486911,36283] 0 2024-04-24T08:24:45.016 INFO:tasks.workunit.client.1.smithi149.stdout:2/320: dread d0/d6/db/d27/f3b [0,4194304] 0 2024-04-24T08:24:45.018 INFO:tasks.workunit.client.1.smithi149.stdout:2/321: chown d0/d6/db/d39/d3d/d58 7252 1 2024-04-24T08:24:45.018 INFO:tasks.workunit.client.0.smithi090.stdout:1/53: sync 2024-04-24T08:24:45.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/36: sync 2024-04-24T08:24:45.019 INFO:tasks.workunit.client.1.smithi149.stdout:2/322: dread - d0/d6/db/f4a zero size 2024-04-24T08:24:45.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/37: rename d0 to d0/d6 22 2024-04-24T08:24:45.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/38: write - no filename 2024-04-24T08:24:45.030 INFO:tasks.workunit.client.0.smithi090.stdout:5/95: mkdir d8/da/d18/d21 0 2024-04-24T08:24:45.035 INFO:tasks.workunit.client.1.smithi149.stdout:4/397: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:45.035 INFO:tasks.workunit.client.0.smithi090.stdout:2/39: mkdir d0/d7 0 2024-04-24T08:24:45.037 INFO:tasks.workunit.client.1.smithi149.stdout:2/323: dwrite d0/d6/db/d39/f4e [0,4194304] 0 2024-04-24T08:24:45.040 INFO:tasks.workunit.client.0.smithi090.stdout:5/96: creat d8/da/d18/f22 x:0 0 0 2024-04-24T08:24:45.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/98: dwrite f9 [0,4194304] 0 2024-04-24T08:24:45.051 INFO:tasks.workunit.client.0.smithi090.stdout:2/40: creat d0/d7/f8 x:0 0 0 2024-04-24T08:24:45.051 INFO:tasks.workunit.client.0.smithi090.stdout:0/130: dwrite f12 [0,4194304] 0 2024-04-24T08:24:45.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/99: creat db/d17/d1b/f26 x:0 0 0 2024-04-24T08:24:45.061 INFO:tasks.workunit.client.0.smithi090.stdout:6/95: truncate d8/f15 317821 0 2024-04-24T08:24:45.061 INFO:tasks.workunit.client.1.smithi149.stdout:7/391: stat d9/dc/d11/f1e 0 2024-04-24T08:24:45.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/97: write f4 [2443737,122500] 0 2024-04-24T08:24:45.062 INFO:tasks.workunit.client.1.smithi149.stdout:4/398: dwrite d2/f44 [0,4194304] 0 2024-04-24T08:24:45.063 INFO:tasks.workunit.client.0.smithi090.stdout:1/54: dwrite f0 [0,4194304] 0 2024-04-24T08:24:45.064 INFO:tasks.workunit.client.1.smithi149.stdout:2/324: dwrite d0/f2 [4194304,4194304] 0 2024-04-24T08:24:45.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/41: creat d0/f9 x:0 0 0 2024-04-24T08:24:45.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/13: getdents d0 0 2024-04-24T08:24:45.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/14: dwrite - no filename 2024-04-24T08:24:45.072 INFO:tasks.workunit.client.0.smithi090.stdout:2/42: dread - d0/f9 zero size 2024-04-24T08:24:45.072 INFO:tasks.workunit.client.0.smithi090.stdout:2/43: readlink - no filename 2024-04-24T08:24:45.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/302: dwrite d0/fe [0,4194304] 0 2024-04-24T08:24:45.079 INFO:tasks.workunit.client.1.smithi149.stdout:8/337: dread d1/de/d18/f45 [0,4194304] 0 2024-04-24T08:24:45.080 INFO:tasks.workunit.client.0.smithi090.stdout:6/96: mkdir d8/d1d 0 2024-04-24T08:24:45.089 INFO:tasks.workunit.client.0.smithi090.stdout:1/55: symlink d3/l13 0 2024-04-24T08:24:45.091 INFO:tasks.workunit.client.0.smithi090.stdout:5/98: rename d8/da/f19 to d8/da/d13/f23 0 2024-04-24T08:24:45.095 INFO:tasks.workunit.client.1.smithi149.stdout:2/325: mknod d0/d6/db/d27/c68 0 2024-04-24T08:24:45.096 INFO:tasks.workunit.client.0.smithi090.stdout:9/15: creat d0/f2 x:0 0 0 2024-04-24T08:24:45.097 INFO:tasks.workunit.client.0.smithi090.stdout:0/131: link ff d1a/f30 0 2024-04-24T08:24:45.101 INFO:tasks.workunit.client.0.smithi090.stdout:2/44: dwrite d0/d7/f8 [0,4194304] 0 2024-04-24T08:24:45.104 INFO:tasks.workunit.client.1.smithi149.stdout:8/338: mknod d1/d1f/d5e/d6c/c70 0 2024-04-24T08:24:45.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/455: dwrite d0/d5/ff [4194304,4194304] 0 2024-04-24T08:24:45.106 INFO:tasks.workunit.client.1.smithi149.stdout:7/392: symlink d9/dc/d11/l82 0 2024-04-24T08:24:45.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/97: write d8/dc/f10 [26965,61719] 0 2024-04-24T08:24:45.108 INFO:tasks.workunit.client.0.smithi090.stdout:4/40: rmdir d3 39 2024-04-24T08:24:45.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/56: creat d3/d5/f14 x:0 0 0 2024-04-24T08:24:45.117 INFO:tasks.workunit.client.1.smithi149.stdout:7/393: dread d9/d43/d4c/f71 [0,4194304] 0 2024-04-24T08:24:45.119 INFO:tasks.workunit.client.0.smithi090.stdout:1/57: write d3/d5/f14 [151220,35396] 0 2024-04-24T08:24:45.119 INFO:tasks.workunit.client.1.smithi149.stdout:9/431: sync 2024-04-24T08:24:45.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/99: mknod d8/da/d14/c24 0 2024-04-24T08:24:45.121 INFO:tasks.workunit.client.0.smithi090.stdout:5/100: fdatasync d8/da/f1d 0 2024-04-24T08:24:45.122 INFO:tasks.workunit.client.1.smithi149.stdout:7/394: dread d9/dc/de/f3a [0,4194304] 0 2024-04-24T08:24:45.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/101: fdatasync d8/da/d18/f22 0 2024-04-24T08:24:45.123 INFO:tasks.workunit.client.1.smithi149.stdout:8/339: creat d1/d1f/d22/f71 x:0 0 0 2024-04-24T08:24:45.123 INFO:tasks.workunit.client.0.smithi090.stdout:5/102: chown d8/f1c 245 1 2024-04-24T08:24:45.125 INFO:tasks.workunit.client.1.smithi149.stdout:5/458: truncate db/d12/d2f/f46 660805 0 2024-04-24T08:24:45.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/32: truncate d2/f3 2413160 0 2024-04-24T08:24:45.126 INFO:tasks.workunit.client.1.smithi149.stdout:3/456: symlink d0/d5/d53/l9d 0 2024-04-24T08:24:45.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/132: mknod d1a/c31 0 2024-04-24T08:24:45.133 INFO:tasks.workunit.client.1.smithi149.stdout:6/303: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:45.134 INFO:tasks.workunit.client.1.smithi149.stdout:2/326: dwrite d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:45.135 INFO:tasks.workunit.client.1.smithi149.stdout:9/432: mkdir d1/d4c/d91 0 2024-04-24T08:24:45.136 INFO:tasks.workunit.client.1.smithi149.stdout:6/304: readlink d0/d1a/d28/d31/d33/l5e 0 2024-04-24T08:24:45.138 INFO:tasks.workunit.client.1.smithi149.stdout:9/433: chown d1/d2/d34/f40 1 1 2024-04-24T08:24:45.146 INFO:tasks.workunit.client.0.smithi090.stdout:8/20: sync 2024-04-24T08:24:45.147 INFO:tasks.workunit.client.1.smithi149.stdout:3/457: dwrite d0/d5/da/f97 [0,4194304] 0 2024-04-24T08:24:45.147 INFO:tasks.workunit.client.0.smithi090.stdout:0/133: dwrite d1a/f20 [0,4194304] 0 2024-04-24T08:24:45.151 INFO:tasks.workunit.client.0.smithi090.stdout:0/134: truncate d1a/f24 63679 0 2024-04-24T08:24:45.151 INFO:tasks.workunit.client.0.smithi090.stdout:5/103: mkdir d8/d25 0 2024-04-24T08:24:45.152 INFO:tasks.workunit.client.1.smithi149.stdout:5/459: mkdir db/d12/d60/d8f 0 2024-04-24T08:24:45.155 INFO:tasks.workunit.client.1.smithi149.stdout:6/305: rmdir d0 39 2024-04-24T08:24:45.155 INFO:tasks.workunit.client.0.smithi090.stdout:7/100: getdents db 0 2024-04-24T08:24:45.158 INFO:tasks.workunit.client.0.smithi090.stdout:3/33: mknod d2/c5 0 2024-04-24T08:24:45.158 INFO:tasks.workunit.client.1.smithi149.stdout:5/460: fsync db/d12/f5b 0 2024-04-24T08:24:45.161 INFO:tasks.workunit.client.1.smithi149.stdout:5/461: write db/d23/f44 [899143,127273] 0 2024-04-24T08:24:45.166 INFO:tasks.workunit.client.1.smithi149.stdout:0/363: dwrite d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:45.169 INFO:tasks.workunit.client.0.smithi090.stdout:4/41: readlink d3/l4 0 2024-04-24T08:24:45.169 INFO:tasks.workunit.client.1.smithi149.stdout:2/327: rename d0/cd to d0/d6/db/d27/c69 0 2024-04-24T08:24:45.171 INFO:tasks.workunit.client.1.smithi149.stdout:1/497: write d0/d39/f7c [373740,99712] 0 2024-04-24T08:24:45.173 INFO:tasks.workunit.client.1.smithi149.stdout:1/498: chown d0/d1b/d5d/f9a 1 1 2024-04-24T08:24:45.174 INFO:tasks.workunit.client.0.smithi090.stdout:6/98: creat d8/d1d/f1e x:0 0 0 2024-04-24T08:24:45.178 INFO:tasks.workunit.client.0.smithi090.stdout:8/21: mknod d2/cb 0 2024-04-24T08:24:45.183 INFO:tasks.workunit.client.1.smithi149.stdout:8/340: truncate d1/de/f2f 3273095 0 2024-04-24T08:24:45.183 INFO:tasks.workunit.client.0.smithi090.stdout:6/99: read d8/fa [4957228,24862] 0 2024-04-24T08:24:45.183 INFO:tasks.workunit.client.0.smithi090.stdout:0/135: creat d1a/d2a/f32 x:0 0 0 2024-04-24T08:24:45.186 INFO:tasks.workunit.client.1.smithi149.stdout:8/341: truncate d1/de/d18/d43/f55 4696448 0 2024-04-24T08:24:45.186 INFO:tasks.workunit.client.1.smithi149.stdout:0/364: rmdir d2/d9/df/d56 39 2024-04-24T08:24:45.190 INFO:tasks.workunit.client.1.smithi149.stdout:3/458: link d0/d5/d53/d6d/f9a d0/d5/da/d59/d79/f9e 0 2024-04-24T08:24:45.194 INFO:tasks.workunit.client.0.smithi090.stdout:4/42: creat d3/fa x:0 0 0 2024-04-24T08:24:45.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/136: chown f8 18 1 2024-04-24T08:24:45.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/16: rmdir d0 39 2024-04-24T08:24:45.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/137: write d1a/d2a/f32 [13406,51564] 0 2024-04-24T08:24:45.200 INFO:tasks.workunit.client.1.smithi149.stdout:5/462: dwrite db/f16 [4194304,4194304] 0 2024-04-24T08:24:45.203 INFO:tasks.workunit.client.1.smithi149.stdout:9/434: dread d1/de/d12/f1d [0,4194304] 0 2024-04-24T08:24:45.205 INFO:tasks.workunit.client.0.smithi090.stdout:7/101: dwrite db/d17/d1b/f26 [0,4194304] 0 2024-04-24T08:24:45.208 INFO:tasks.workunit.client.0.smithi090.stdout:5/104: dwrite d8/da/d18/f22 [0,4194304] 0 2024-04-24T08:24:45.209 INFO:tasks.workunit.client.1.smithi149.stdout:3/459: creat d0/d5/da/d19/f9f x:0 0 0 2024-04-24T08:24:45.210 INFO:tasks.workunit.client.0.smithi090.stdout:2/45: write d0/d7/f8 [5071092,67648] 0 2024-04-24T08:24:45.214 INFO:tasks.workunit.client.1.smithi149.stdout:4/399: dwrite d2/d10/d1a/f6f [0,4194304] 0 2024-04-24T08:24:45.222 INFO:tasks.workunit.client.1.smithi149.stdout:4/400: fdatasync d2/d24/d2e/f51 0 2024-04-24T08:24:45.225 INFO:tasks.workunit.client.1.smithi149.stdout:1/499: dread d0/d1b/d5d/f6c [0,4194304] 0 2024-04-24T08:24:45.227 INFO:tasks.workunit.client.1.smithi149.stdout:4/401: write d2/d18/f64 [2735204,119081] 0 2024-04-24T08:24:45.234 INFO:tasks.workunit.client.1.smithi149.stdout:7/395: truncate d9/d43/d4c/f71 1154751 0 2024-04-24T08:24:45.234 INFO:tasks.workunit.client.0.smithi090.stdout:4/43: dwrite d3/fa [0,4194304] 0 2024-04-24T08:24:45.235 INFO:tasks.workunit.client.0.smithi090.stdout:8/22: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:45.238 INFO:tasks.workunit.client.1.smithi149.stdout:9/435: creat d1/d2/d34/f92 x:0 0 0 2024-04-24T08:24:45.240 INFO:tasks.workunit.client.1.smithi149.stdout:2/328: rename d0/l40 to d0/d6/db/l6a 0 2024-04-24T08:24:45.243 INFO:tasks.workunit.client.1.smithi149.stdout:2/329: chown d0/d6/db/d28/f2c 263959485 1 2024-04-24T08:24:45.244 INFO:tasks.workunit.client.0.smithi090.stdout:8/23: write d2/f9 [238486,84462] 0 2024-04-24T08:24:45.257 INFO:tasks.workunit.client.0.smithi090.stdout:1/58: dwrite d3/f4 [0,4194304] 0 2024-04-24T08:24:45.259 INFO:tasks.workunit.client.1.smithi149.stdout:5/463: symlink db/d12/d22/d45/l90 0 2024-04-24T08:24:45.260 INFO:tasks.workunit.client.1.smithi149.stdout:5/464: fdatasync db/d23/f43 0 2024-04-24T08:24:45.264 INFO:tasks.workunit.client.1.smithi149.stdout:7/396: chown d9/dc/d11/c55 1626210 1 2024-04-24T08:24:45.268 INFO:tasks.workunit.client.1.smithi149.stdout:8/342: rename d1/d1f/d5e/d6c/c70 to d1/de/d18/d32/d63/c72 0 2024-04-24T08:24:45.272 INFO:tasks.workunit.client.0.smithi090.stdout:8/24: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:45.275 INFO:tasks.workunit.client.1.smithi149.stdout:6/306: write d0/d1a/f4d [3821011,28303] 0 2024-04-24T08:24:45.277 INFO:tasks.workunit.client.1.smithi149.stdout:2/330: creat d0/d6/db/d39/d3d/f6b x:0 0 0 2024-04-24T08:24:45.282 INFO:tasks.workunit.client.1.smithi149.stdout:0/365: getdents d2/d9/df/d40/d4a/d4e 0 2024-04-24T08:24:45.284 INFO:tasks.workunit.client.1.smithi149.stdout:5/465: mknod db/d1f/d7f/c91 0 2024-04-24T08:24:45.286 INFO:tasks.workunit.client.1.smithi149.stdout:7/397: chown d9/dc/d11/f37 914190572 1 2024-04-24T08:24:45.289 INFO:tasks.workunit.client.0.smithi090.stdout:8/25: dread d2/f7 [0,4194304] 0 2024-04-24T08:24:45.290 INFO:tasks.workunit.client.1.smithi149.stdout:9/436: dread d1/d2/d33/f80 [0,4194304] 0 2024-04-24T08:24:45.291 INFO:tasks.workunit.client.0.smithi090.stdout:8/26: chown d2/f9 2358253 1 2024-04-24T08:24:45.295 INFO:tasks.workunit.client.1.smithi149.stdout:6/307: creat d0/d1a/d28/d31/d4c/d4e/f64 x:0 0 0 2024-04-24T08:24:45.302 INFO:tasks.workunit.client.1.smithi149.stdout:7/398: rmdir d9 39 2024-04-24T08:24:45.304 INFO:tasks.workunit.client.1.smithi149.stdout:3/460: write d0/d5/d53/d34/f3b [332362,102535] 0 2024-04-24T08:24:45.309 INFO:tasks.workunit.client.1.smithi149.stdout:6/308: dwrite d0/fa [0,4194304] 0 2024-04-24T08:24:45.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/461: dwrite d0/d5/da/d19/f48 [0,4194304] 0 2024-04-24T08:24:45.325 INFO:tasks.workunit.client.1.smithi149.stdout:3/462: fdatasync d0/d5/f66 0 2024-04-24T08:24:45.338 INFO:tasks.workunit.client.1.smithi149.stdout:4/402: rename d2/d24/d2e/d38/c6e to d2/d18/d1e/c86 0 2024-04-24T08:24:45.341 INFO:tasks.workunit.client.1.smithi149.stdout:9/437: mknod d1/de/c93 0 2024-04-24T08:24:45.341 INFO:tasks.workunit.client.1.smithi149.stdout:2/331: write d0/d6/db/f10 [3298421,53310] 0 2024-04-24T08:24:45.343 INFO:tasks.workunit.client.1.smithi149.stdout:1/500: getdents d0/d5/db/d63/d98 0 2024-04-24T08:24:45.343 INFO:tasks.workunit.client.1.smithi149.stdout:9/438: fsync d1/d2/d34/f40 0 2024-04-24T08:24:45.344 INFO:tasks.workunit.client.0.smithi090.stdout:9/17: read - d0/f2 zero size 2024-04-24T08:24:45.345 INFO:tasks.workunit.client.0.smithi090.stdout:9/18: fsync d0/f2 0 2024-04-24T08:24:45.346 INFO:tasks.workunit.client.0.smithi090.stdout:0/138: fdatasync d1a/f1d 0 2024-04-24T08:24:45.347 INFO:tasks.workunit.client.0.smithi090.stdout:5/105: fdatasync f6 0 2024-04-24T08:24:45.347 INFO:tasks.workunit.client.1.smithi149.stdout:0/366: mkdir d2/d9/df/d2f/d6c/d72 0 2024-04-24T08:24:45.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/102: symlink db/d17/l27 0 2024-04-24T08:24:45.350 INFO:tasks.workunit.client.1.smithi149.stdout:5/466: mkdir db/d12/d1e/d85/d92 0 2024-04-24T08:24:45.350 INFO:tasks.workunit.client.0.smithi090.stdout:7/103: read - db/f11 zero size 2024-04-24T08:24:45.351 INFO:tasks.workunit.client.1.smithi149.stdout:5/467: fsync db/f14 0 2024-04-24T08:24:45.351 INFO:tasks.workunit.client.0.smithi090.stdout:2/46: symlink d0/la 0 2024-04-24T08:24:45.352 INFO:tasks.workunit.client.0.smithi090.stdout:2/47: fdatasync d0/f9 0 2024-04-24T08:24:45.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/44: rename d3/l4 to d3/lb 0 2024-04-24T08:24:45.358 INFO:tasks.workunit.client.1.smithi149.stdout:3/463: mkdir d0/d5/d1a/d70/da0 0 2024-04-24T08:24:45.359 INFO:tasks.workunit.client.0.smithi090.stdout:4/45: truncate d3/f9 738198 0 2024-04-24T08:24:45.365 INFO:tasks.workunit.client.1.smithi149.stdout:1/501: mknod d0/d5/db/d82/ca7 0 2024-04-24T08:24:45.367 INFO:tasks.workunit.client.0.smithi090.stdout:9/19: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:45.368 INFO:tasks.workunit.client.0.smithi090.stdout:3/34: write d2/f3 [1888308,123818] 0 2024-04-24T08:24:45.369 INFO:tasks.workunit.client.1.smithi149.stdout:1/502: read d0/da/f24 [3230600,80933] 0 2024-04-24T08:24:45.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/100: getdents d8/dc/d12 0 2024-04-24T08:24:45.371 INFO:tasks.workunit.client.1.smithi149.stdout:7/399: dwrite d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:24:45.373 INFO:tasks.workunit.client.1.smithi149.stdout:9/439: dread d1/d2/d15/f3c [0,4194304] 0 2024-04-24T08:24:45.373 INFO:tasks.workunit.client.1.smithi149.stdout:6/309: truncate d0/f16 4688918 0 2024-04-24T08:24:45.374 INFO:tasks.workunit.client.1.smithi149.stdout:3/464: rmdir d0/d5/da/d5a/d63/d6a 39 2024-04-24T08:24:45.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/104: write db/fd [2231838,47647] 0 2024-04-24T08:24:45.377 INFO:tasks.workunit.client.1.smithi149.stdout:2/332: mkdir d0/d6c 0 2024-04-24T08:24:45.379 INFO:tasks.workunit.client.0.smithi090.stdout:2/48: dwrite d0/d7/f8 [4194304,4194304] 0 2024-04-24T08:24:45.382 INFO:tasks.workunit.client.0.smithi090.stdout:7/105: write db/d17/d1b/f26 [4605295,15389] 0 2024-04-24T08:24:45.384 INFO:tasks.workunit.client.0.smithi090.stdout:7/106: chown db 2605273 1 2024-04-24T08:24:45.387 INFO:tasks.workunit.client.0.smithi090.stdout:4/46: unlink d3/fa 0 2024-04-24T08:24:45.387 INFO:tasks.workunit.client.0.smithi090.stdout:2/49: write d0/d7/f8 [5874217,123149] 0 2024-04-24T08:24:45.390 INFO:tasks.workunit.client.1.smithi149.stdout:7/400: dread d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:45.392 INFO:tasks.workunit.client.0.smithi090.stdout:2/50: chown d0/d7/f8 114 1 2024-04-24T08:24:45.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/51: stat d0/d7 0 2024-04-24T08:24:45.395 INFO:tasks.workunit.client.0.smithi090.stdout:2/52: chown d0/d7/f8 1042 1 2024-04-24T08:24:45.397 INFO:tasks.workunit.client.0.smithi090.stdout:9/20: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:45.397 INFO:tasks.workunit.client.1.smithi149.stdout:9/440: mknod d1/d2/d33/d76/c94 0 2024-04-24T08:24:45.398 INFO:tasks.workunit.client.0.smithi090.stdout:2/53: read d0/d7/f8 [3987341,7103] 0 2024-04-24T08:24:45.399 INFO:tasks.workunit.client.0.smithi090.stdout:2/54: readlink d0/la 0 2024-04-24T08:24:45.400 INFO:tasks.workunit.client.0.smithi090.stdout:9/21: write d0/f2 [2761017,21345] 0 2024-04-24T08:24:45.402 INFO:tasks.workunit.client.0.smithi090.stdout:5/106: creat d8/d25/f26 x:0 0 0 2024-04-24T08:24:45.402 INFO:tasks.workunit.client.1.smithi149.stdout:9/441: write d1/d2/d15/f30 [3912043,35639] 0 2024-04-24T08:24:45.403 INFO:tasks.workunit.client.0.smithi090.stdout:2/55: truncate d0/f9 145003 0 2024-04-24T08:24:45.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/22: write d0/f2 [2846910,40543] 0 2024-04-24T08:24:45.404 INFO:tasks.workunit.client.1.smithi149.stdout:3/465: symlink d0/d5/d53/d4c/d5f/la1 0 2024-04-24T08:24:45.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/35: symlink d2/l6 0 2024-04-24T08:24:45.407 INFO:tasks.workunit.client.0.smithi090.stdout:3/36: write d2/f3 [804585,10579] 0 2024-04-24T08:24:45.408 INFO:tasks.workunit.client.1.smithi149.stdout:5/468: creat db/d12/d22/d45/f93 x:0 0 0 2024-04-24T08:24:45.408 INFO:tasks.workunit.client.0.smithi090.stdout:6/101: mkdir d8/d1d/d1f 0 2024-04-24T08:24:45.415 INFO:tasks.workunit.client.0.smithi090.stdout:1/59: truncate d3/f4 1685938 0 2024-04-24T08:24:45.418 INFO:tasks.workunit.client.0.smithi090.stdout:8/27: truncate d2/f9 33747 0 2024-04-24T08:24:45.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/107: rmdir db/d17/d1b 39 2024-04-24T08:24:45.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/47: chown d3/lb 5091 1 2024-04-24T08:24:45.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/48: stat d3/f6 0 2024-04-24T08:24:45.424 INFO:tasks.workunit.client.0.smithi090.stdout:4/49: fdatasync d3/f7 0 2024-04-24T08:24:45.424 INFO:tasks.workunit.client.1.smithi149.stdout:0/367: getdents d2/d3c 0 2024-04-24T08:24:45.425 INFO:tasks.workunit.client.0.smithi090.stdout:5/107: dwrite d8/f1c [0,4194304] 0 2024-04-24T08:24:45.429 INFO:tasks.workunit.client.1.smithi149.stdout:9/442: unlink d1/da/fc 0 2024-04-24T08:24:45.431 INFO:tasks.workunit.client.1.smithi149.stdout:3/466: creat d0/d5/da/d5e/fa2 x:0 0 0 2024-04-24T08:24:45.431 INFO:tasks.workunit.client.1.smithi149.stdout:2/333: dwrite d0/d6/db/d28/f2c [0,4194304] 0 2024-04-24T08:24:45.433 INFO:tasks.workunit.client.0.smithi090.stdout:9/23: creat d0/f3 x:0 0 0 2024-04-24T08:24:45.436 INFO:tasks.workunit.client.0.smithi090.stdout:9/24: write d0/f2 [1194079,110562] 0 2024-04-24T08:24:45.437 INFO:tasks.workunit.client.0.smithi090.stdout:3/37: creat d2/f7 x:0 0 0 2024-04-24T08:24:45.438 INFO:tasks.workunit.client.0.smithi090.stdout:9/25: readlink d0/l1 0 2024-04-24T08:24:45.448 INFO:tasks.workunit.client.0.smithi090.stdout:1/60: mknod d3/d5/c15 0 2024-04-24T08:24:45.448 INFO:tasks.workunit.client.1.smithi149.stdout:7/401: mkdir d9/dc/d11/d83 0 2024-04-24T08:24:45.449 INFO:tasks.workunit.client.0.smithi090.stdout:1/61: readlink d3/d5/l6 0 2024-04-24T08:24:45.450 INFO:tasks.workunit.client.0.smithi090.stdout:1/62: stat d3/d12 0 2024-04-24T08:24:45.452 INFO:tasks.workunit.client.1.smithi149.stdout:9/443: unlink d1/da/dd/f6c 0 2024-04-24T08:24:45.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/28: chown d2/l6 57 1 2024-04-24T08:24:45.458 INFO:tasks.workunit.client.0.smithi090.stdout:8/29: write d2/f3 [3968975,18237] 0 2024-04-24T08:24:45.458 INFO:tasks.workunit.client.0.smithi090.stdout:3/38: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:45.459 INFO:tasks.workunit.client.1.smithi149.stdout:5/469: dwrite db/d12/d1e/f64 [0,4194304] 0 2024-04-24T08:24:45.459 INFO:tasks.workunit.client.1.smithi149.stdout:6/310: write d0/f16 [1348472,75615] 0 2024-04-24T08:24:45.461 INFO:tasks.workunit.client.1.smithi149.stdout:3/467: mkdir d0/d5/d1a/d70/da3 0 2024-04-24T08:24:45.461 INFO:tasks.workunit.client.1.smithi149.stdout:2/334: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:45.464 INFO:tasks.workunit.client.1.smithi149.stdout:1/503: getdents d0/d1b/d2a 0 2024-04-24T08:24:45.465 INFO:tasks.workunit.client.0.smithi090.stdout:4/50: unlink d3/f6 0 2024-04-24T08:24:45.468 INFO:tasks.workunit.client.0.smithi090.stdout:4/51: write d3/f9 [371702,54841] 0 2024-04-24T08:24:45.469 INFO:tasks.workunit.client.0.smithi090.stdout:1/63: dwrite d3/d5/f14 [0,4194304] 0 2024-04-24T08:24:45.479 INFO:tasks.workunit.client.1.smithi149.stdout:7/402: dwrite d9/fd [0,4194304] 0 2024-04-24T08:24:45.482 INFO:tasks.workunit.client.0.smithi090.stdout:5/108: creat d8/da/d14/f27 x:0 0 0 2024-04-24T08:24:45.482 INFO:tasks.workunit.client.1.smithi149.stdout:7/403: write d9/d43/d64/f73 [654931,2767] 0 2024-04-24T08:24:45.484 INFO:tasks.workunit.client.1.smithi149.stdout:7/404: stat d9/dc/d2e/d3c 0 2024-04-24T08:24:45.487 INFO:tasks.workunit.client.1.smithi149.stdout:7/405: write d9/dc/d11/f37 [19316,59468] 0 2024-04-24T08:24:45.489 INFO:tasks.workunit.client.0.smithi090.stdout:9/26: creat d0/f4 x:0 0 0 2024-04-24T08:24:45.491 INFO:tasks.workunit.client.1.smithi149.stdout:6/311: mknod d0/d1a/d28/d31/d4c/c65 0 2024-04-24T08:24:45.497 INFO:tasks.workunit.client.1.smithi149.stdout:3/468: rename d0/d5/d53/l4f to d0/d5/da/d5e/la4 0 2024-04-24T08:24:45.498 INFO:tasks.workunit.client.1.smithi149.stdout:1/504: write d0/d5/df/f94 [4874635,70019] 0 2024-04-24T08:24:45.502 INFO:tasks.workunit.client.1.smithi149.stdout:2/335: creat d0/d6/db/d39/d3d/d58/f6d x:0 0 0 2024-04-24T08:24:45.506 INFO:tasks.workunit.client.0.smithi090.stdout:3/39: unlink f0 0 2024-04-24T08:24:45.508 INFO:tasks.workunit.client.0.smithi090.stdout:3/40: readlink d2/l6 0 2024-04-24T08:24:45.509 INFO:tasks.workunit.client.0.smithi090.stdout:3/41: dread - d2/f7 zero size 2024-04-24T08:24:45.512 INFO:tasks.workunit.client.1.smithi149.stdout:7/406: readlink d9/dc/d11/d26/l47 0 2024-04-24T08:24:45.515 INFO:tasks.workunit.client.1.smithi149.stdout:3/469: creat d0/d88/fa5 x:0 0 0 2024-04-24T08:24:45.517 INFO:tasks.workunit.client.1.smithi149.stdout:1/505: mknod d0/d5/db/d18/d4d/d50/ca8 0 2024-04-24T08:24:45.517 INFO:tasks.workunit.client.1.smithi149.stdout:3/470: write d0/d5/da/d19/d80/f8c [2236512,34859] 0 2024-04-24T08:24:45.518 INFO:tasks.workunit.client.0.smithi090.stdout:1/64: creat d3/d5/f16 x:0 0 0 2024-04-24T08:24:45.521 INFO:tasks.workunit.client.0.smithi090.stdout:5/109: creat d8/da/d14/f28 x:0 0 0 2024-04-24T08:24:45.522 INFO:tasks.workunit.client.1.smithi149.stdout:2/336: creat d0/d6/db/d27/d64/f6e x:0 0 0 2024-04-24T08:24:45.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/470: creat db/f94 x:0 0 0 2024-04-24T08:24:45.525 INFO:tasks.workunit.client.0.smithi090.stdout:4/52: dread d3/f9 [0,4194304] 0 2024-04-24T08:24:45.526 INFO:tasks.workunit.client.1.smithi149.stdout:5/471: read db/d12/d2f/f62 [2590442,46747] 0 2024-04-24T08:24:45.526 INFO:tasks.workunit.client.0.smithi090.stdout:6/102: rmdir d8/dc/d12 0 2024-04-24T08:24:45.528 INFO:tasks.workunit.client.1.smithi149.stdout:6/312: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d66 0 2024-04-24T08:24:45.529 INFO:tasks.workunit.client.1.smithi149.stdout:6/313: chown d0/d1a/f32 596012 1 2024-04-24T08:24:45.534 INFO:tasks.workunit.client.0.smithi090.stdout:7/108: creat db/d17/f28 x:0 0 0 2024-04-24T08:24:45.536 INFO:tasks.workunit.client.1.smithi149.stdout:9/444: getdents d1/da/dd/d55/d72 0 2024-04-24T08:24:45.536 INFO:tasks.workunit.client.0.smithi090.stdout:3/42: unlink d2/l6 0 2024-04-24T08:24:45.537 INFO:tasks.workunit.client.1.smithi149.stdout:3/471: fsync d0/d5/d53/f47 0 2024-04-24T08:24:45.540 INFO:tasks.workunit.client.0.smithi090.stdout:1/65: read d3/f4 [1511752,129507] 0 2024-04-24T08:24:45.541 INFO:tasks.workunit.client.1.smithi149.stdout:2/337: read d0/f55 [1560367,68974] 0 2024-04-24T08:24:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:45 smithi090 ceph-mon[25016]: pgmap v15: 65 pgs: 65 active+clean; 843 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 105 MiB/s wr, 198 op/s 2024-04-24T08:24:45.545 INFO:tasks.workunit.client.1.smithi149.stdout:1/506: dread d0/d5/f41 [8388608,4194304] 0 2024-04-24T08:24:45.546 INFO:tasks.workunit.client.0.smithi090.stdout:8/30: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:45.546 INFO:tasks.workunit.client.1.smithi149.stdout:7/407: mkdir d9/dc/d11/d84 0 2024-04-24T08:24:45.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/408: write d9/dc/d72/f76 [6203607,93966] 0 2024-04-24T08:24:45.550 INFO:tasks.workunit.client.1.smithi149.stdout:7/409: write d9/dc/d2e/f6f [1634762,67522] 0 2024-04-24T08:24:45.552 INFO:tasks.workunit.client.0.smithi090.stdout:8/31: chown d2/l5 24974 1 2024-04-24T08:24:45.556 INFO:tasks.workunit.client.0.smithi090.stdout:5/110: symlink d8/da/d18/d21/l29 0 2024-04-24T08:24:45.557 INFO:tasks.workunit.client.1.smithi149.stdout:9/445: getdents d1/d2/d33/d5d/d83 0 2024-04-24T08:24:45.558 INFO:tasks.workunit.client.1.smithi149.stdout:7/410: chown d9/dc/d2e/d3c/c48 1267164997 1 2024-04-24T08:24:45.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/27: getdents d0 0 2024-04-24T08:24:45.562 INFO:tasks.workunit.client.1.smithi149.stdout:9/446: rmdir d1/d4c 39 2024-04-24T08:24:45.562 INFO:tasks.workunit.client.1.smithi149.stdout:7/411: symlink d9/d43/d4f/l85 0 2024-04-24T08:24:45.562 INFO:tasks.workunit.client.1.smithi149.stdout:7/412: chown d9/dc/d11/d26/l36 1101 1 2024-04-24T08:24:45.564 INFO:tasks.workunit.client.0.smithi090.stdout:8/32: rename d2/l6 to d2/da/lc 0 2024-04-24T08:24:45.565 INFO:tasks.workunit.client.1.smithi149.stdout:9/447: truncate d1/da/d13/f45 1871022 0 2024-04-24T08:24:45.566 INFO:tasks.workunit.client.1.smithi149.stdout:7/413: creat d9/d43/d4c/f86 x:0 0 0 2024-04-24T08:24:45.567 INFO:tasks.workunit.client.0.smithi090.stdout:5/111: unlink d8/f15 0 2024-04-24T08:24:45.568 INFO:tasks.workunit.client.0.smithi090.stdout:9/28: dread - d0/f3 zero size 2024-04-24T08:24:45.568 INFO:tasks.workunit.client.1.smithi149.stdout:3/472: write d0/d5/da/d59/d79/f98 [1465987,119767] 0 2024-04-24T08:24:45.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/448: rmdir d1/d2/d15/d1c/d42 39 2024-04-24T08:24:45.569 INFO:tasks.workunit.client.0.smithi090.stdout:6/103: write d8/fa [9087945,2298] 0 2024-04-24T08:24:45.572 INFO:tasks.workunit.client.1.smithi149.stdout:7/414: rmdir d9/d43/d4c/d67 39 2024-04-24T08:24:45.574 INFO:tasks.workunit.client.1.smithi149.stdout:3/473: unlink d0/d26/d3c/l62 0 2024-04-24T08:24:45.575 INFO:tasks.workunit.client.0.smithi090.stdout:4/53: truncate d3/f9 358195 0 2024-04-24T08:24:45.576 INFO:tasks.workunit.client.1.smithi149.stdout:5/472: dwrite db/d12/d1e/f4a [4194304,4194304] 0 2024-04-24T08:24:45.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/109: dwrite db/f1a [0,4194304] 0 2024-04-24T08:24:45.576 INFO:tasks.workunit.client.0.smithi090.stdout:3/43: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:45.577 INFO:tasks.workunit.client.0.smithi090.stdout:6/104: stat d8/dc 0 2024-04-24T08:24:45.578 INFO:tasks.workunit.client.0.smithi090.stdout:5/112: read f7 [377247,126878] 0 2024-04-24T08:24:45.580 INFO:tasks.workunit.client.1.smithi149.stdout:5/473: write db/d12/f69 [734745,63238] 0 2024-04-24T08:24:45.581 INFO:tasks.workunit.client.0.smithi090.stdout:7/110: stat db/f15 0 2024-04-24T08:24:45.581 INFO:tasks.workunit.client.1.smithi149.stdout:7/415: write d9/dc/d11/f58 [418842,4485] 0 2024-04-24T08:24:45.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/54: write d3/f7 [27834,128547] 0 2024-04-24T08:24:45.583 INFO:tasks.workunit.client.0.smithi090.stdout:9/29: creat d0/f5 x:0 0 0 2024-04-24T08:24:45.585 INFO:tasks.workunit.client.1.smithi149.stdout:3/474: symlink d0/d5/da/d5e/la6 0 2024-04-24T08:24:45.585 INFO:tasks.workunit.client.1.smithi149.stdout:8/343: sync 2024-04-24T08:24:45.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/403: sync 2024-04-24T08:24:45.586 INFO:tasks.workunit.client.1.smithi149.stdout:5/474: read db/d12/d22/d45/d5d/f63 [3189703,32473] 0 2024-04-24T08:24:45.589 INFO:tasks.workunit.client.0.smithi090.stdout:1/66: dwrite d3/d5/da/dc/fe [0,4194304] 0 2024-04-24T08:24:45.595 INFO:tasks.workunit.client.1.smithi149.stdout:7/416: mknod d9/d43/d64/c87 0 2024-04-24T08:24:45.596 INFO:tasks.workunit.client.0.smithi090.stdout:3/44: mknod d2/c8 0 2024-04-24T08:24:45.598 INFO:tasks.workunit.client.0.smithi090.stdout:5/113: symlink d8/da/d18/l2a 0 2024-04-24T08:24:45.606 INFO:tasks.workunit.client.1.smithi149.stdout:1/507: dread d0/d5/df/f28 [0,4194304] 0 2024-04-24T08:24:45.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/45: read d2/f3 [151061,61224] 0 2024-04-24T08:24:45.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/33: dwrite d2/f7 [4194304,4194304] 0 2024-04-24T08:24:45.614 INFO:tasks.workunit.client.0.smithi090.stdout:6/105: dread d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:45.616 INFO:tasks.workunit.client.0.smithi090.stdout:1/67: creat d3/d5/f17 x:0 0 0 2024-04-24T08:24:45.618 INFO:tasks.workunit.client.1.smithi149.stdout:8/344: creat d1/d1f/d5e/f73 x:0 0 0 2024-04-24T08:24:45.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/30: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:45.619 INFO:tasks.workunit.client.0.smithi090.stdout:1/68: dread - d3/d5/f16 zero size 2024-04-24T08:24:45.621 INFO:tasks.workunit.client.1.smithi149.stdout:5/475: mkdir db/d12/d2f/d95 0 2024-04-24T08:24:45.621 INFO:tasks.workunit.client.0.smithi090.stdout:6/106: truncate d8/d1d/f1e 717004 0 2024-04-24T08:24:45.622 INFO:tasks.workunit.client.0.smithi090.stdout:1/69: dread - d3/d5/f17 zero size 2024-04-24T08:24:45.624 INFO:tasks.workunit.client.0.smithi090.stdout:5/114: chown c5 0 1 2024-04-24T08:24:45.628 INFO:tasks.workunit.client.0.smithi090.stdout:8/34: creat d2/fd x:0 0 0 2024-04-24T08:24:45.629 INFO:tasks.workunit.client.1.smithi149.stdout:7/417: write d9/dc/d11/f1e [680169,44900] 0 2024-04-24T08:24:45.629 INFO:tasks.workunit.client.0.smithi090.stdout:6/107: rename d8/l14 to d8/dc/l20 0 2024-04-24T08:24:45.631 INFO:tasks.workunit.client.1.smithi149.stdout:3/475: getdents d0/d5/d73 0 2024-04-24T08:24:45.634 INFO:tasks.workunit.client.1.smithi149.stdout:9/449: rmdir d1/d4c/d6d 0 2024-04-24T08:24:45.634 INFO:tasks.workunit.client.0.smithi090.stdout:1/70: write d3/d5/ff [341856,123291] 0 2024-04-24T08:24:45.639 INFO:tasks.workunit.client.1.smithi149.stdout:5/476: mknod db/d12/d60/c96 0 2024-04-24T08:24:45.639 INFO:tasks.workunit.client.1.smithi149.stdout:1/508: mknod d0/d5/ca9 0 2024-04-24T08:24:45.641 INFO:tasks.workunit.client.1.smithi149.stdout:1/509: write d0/f25 [1326725,97266] 0 2024-04-24T08:24:45.641 INFO:tasks.workunit.client.1.smithi149.stdout:5/477: write db/d12/d22/d45/f93 [333926,57670] 0 2024-04-24T08:24:45.643 INFO:tasks.workunit.client.1.smithi149.stdout:9/450: mknod d1/d4c/c95 0 2024-04-24T08:24:45.644 INFO:tasks.workunit.client.0.smithi090.stdout:8/35: link d2/l5 d2/da/le 0 2024-04-24T08:24:45.644 INFO:tasks.workunit.client.0.smithi090.stdout:5/115: creat d8/da/d18/f2b x:0 0 0 2024-04-24T08:24:45.646 INFO:tasks.workunit.client.0.smithi090.stdout:6/108: mknod d8/d1d/d1f/c21 0 2024-04-24T08:24:45.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/510: truncate d0/d5/db/d63/f83 1614063 0 2024-04-24T08:24:45.651 INFO:tasks.workunit.client.1.smithi149.stdout:1/511: read d0/fe [2302318,101033] 0 2024-04-24T08:24:45.653 INFO:tasks.workunit.client.0.smithi090.stdout:3/46: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:45.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/345: dwrite d1/de/f1a [4194304,4194304] 0 2024-04-24T08:24:45.655 INFO:tasks.workunit.client.1.smithi149.stdout:1/512: write d0/f16 [1423898,82095] 0 2024-04-24T08:24:45.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/346: chown d1/d1f/f50 3774650 1 2024-04-24T08:24:45.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/418: link d9/d43/l5a d9/d43/l88 0 2024-04-24T08:24:45.664 INFO:tasks.workunit.client.0.smithi090.stdout:7/111: dwrite db/d17/d1b/f23 [0,4194304] 0 2024-04-24T08:24:45.665 INFO:tasks.workunit.client.0.smithi090.stdout:3/47: write d2/f3 [3185930,82281] 0 2024-04-24T08:24:45.667 INFO:tasks.workunit.client.1.smithi149.stdout:4/404: truncate d2/d18/d1e/d3c/d48/d69/f70 706482 0 2024-04-24T08:24:45.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/36: dread d2/f9 [0,4194304] 0 2024-04-24T08:24:45.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/31: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:45.668 INFO:tasks.workunit.client.1.smithi149.stdout:3/476: rename d0/d5/f66 to d0/d5/da/fa7 0 2024-04-24T08:24:45.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/71: creat d3/d12/f18 x:0 0 0 2024-04-24T08:24:45.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/37: read d2/f3 [975236,46194] 0 2024-04-24T08:24:45.675 INFO:tasks.workunit.client.0.smithi090.stdout:6/109: rename d8/dc/d19/l1c to d8/dc/d19/l22 0 2024-04-24T08:24:45.676 INFO:tasks.workunit.client.0.smithi090.stdout:9/32: chown d0/f4 6684131 1 2024-04-24T08:24:45.678 INFO:tasks.workunit.client.1.smithi149.stdout:1/513: rmdir d0/d5/df 39 2024-04-24T08:24:45.682 INFO:tasks.workunit.client.0.smithi090.stdout:6/110: chown d8/dc/f1a 649857 1 2024-04-24T08:24:45.683 INFO:tasks.workunit.client.0.smithi090.stdout:9/33: write d0/f5 [2954002,122065] 0 2024-04-24T08:24:45.685 INFO:tasks.workunit.client.1.smithi149.stdout:1/514: chown d0/d5/db/d63/d98/l9f 6643467 1 2024-04-24T08:24:45.685 INFO:tasks.workunit.client.1.smithi149.stdout:9/451: link d1/f88 d1/de/d12/f96 0 2024-04-24T08:24:45.687 INFO:tasks.workunit.client.1.smithi149.stdout:5/478: dread db/d12/d60/f67 [0,4194304] 0 2024-04-24T08:24:45.689 INFO:tasks.workunit.client.0.smithi090.stdout:5/116: dwrite d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:24:45.692 INFO:tasks.workunit.client.1.smithi149.stdout:4/405: dread d2/d10/d1a/f6f [0,4194304] 0 2024-04-24T08:24:45.693 INFO:tasks.workunit.client.1.smithi149.stdout:9/452: write d1/d2/f2b [2831387,27544] 0 2024-04-24T08:24:45.693 INFO:tasks.workunit.client.0.smithi090.stdout:1/72: creat d3/d5/f19 x:0 0 0 2024-04-24T08:24:45.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/38: fsync d2/f9 0 2024-04-24T08:24:45.703 INFO:tasks.workunit.client.0.smithi090.stdout:8/39: write d2/fd [347259,40178] 0 2024-04-24T08:24:45.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/34: mknod d0/c6 0 2024-04-24T08:24:45.705 INFO:tasks.workunit.client.1.smithi149.stdout:7/419: mknod d9/dc/d11/d83/c89 0 2024-04-24T08:24:45.705 INFO:tasks.workunit.client.0.smithi090.stdout:5/117: dread d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:24:45.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/112: mkdir db/d17/d29 0 2024-04-24T08:24:45.707 INFO:tasks.workunit.client.0.smithi090.stdout:6/111: creat d8/d1d/d1f/f23 x:0 0 0 2024-04-24T08:24:45.709 INFO:tasks.workunit.client.1.smithi149.stdout:1/515: rename d0/d1b/la5 to d0/d39/d76/laa 0 2024-04-24T08:24:45.713 INFO:tasks.workunit.client.1.smithi149.stdout:5/479: rmdir db/d12/d22/d45 39 2024-04-24T08:24:45.715 INFO:tasks.workunit.client.0.smithi090.stdout:1/73: creat d3/d5/f1a x:0 0 0 2024-04-24T08:24:45.716 INFO:tasks.workunit.client.1.smithi149.stdout:4/406: readlink d2/l8 0 2024-04-24T08:24:45.717 INFO:tasks.workunit.client.0.smithi090.stdout:9/35: mkdir d0/d7 0 2024-04-24T08:24:45.718 INFO:tasks.workunit.client.1.smithi149.stdout:9/453: creat d1/d2/f97 x:0 0 0 2024-04-24T08:24:45.720 INFO:tasks.workunit.client.0.smithi090.stdout:9/36: stat d0/d7 0 2024-04-24T08:24:45.720 INFO:tasks.workunit.client.1.smithi149.stdout:0/368: sync 2024-04-24T08:24:45.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/40: write d2/f3 [2184972,19173] 0 2024-04-24T08:24:45.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/37: truncate d0/f4 925413 0 2024-04-24T08:24:45.724 INFO:tasks.workunit.client.0.smithi090.stdout:7/113: chown db/d17/c1f 1683089311 1 2024-04-24T08:24:45.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/112: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:45.728 INFO:tasks.workunit.client.1.smithi149.stdout:1/516: write d0/d5/df/f28 [444083,73392] 0 2024-04-24T08:24:45.730 INFO:tasks.workunit.client.0.smithi090.stdout:1/74: unlink d3/d5/ff 0 2024-04-24T08:24:45.739 INFO:tasks.workunit.client.0.smithi090.stdout:6/113: creat d8/d1d/f24 x:0 0 0 2024-04-24T08:24:45.740 INFO:tasks.workunit.client.1.smithi149.stdout:3/477: link d0/d5/d53/d34/f81 d0/d5/d53/d4c/fa8 0 2024-04-24T08:24:45.741 INFO:tasks.workunit.client.1.smithi149.stdout:1/517: dread d0/d5/db/d63/f83 [0,4194304] 0 2024-04-24T08:24:45.743 INFO:tasks.workunit.client.1.smithi149.stdout:1/518: chown d0/da/fa3 1563 1 2024-04-24T08:24:45.743 INFO:tasks.workunit.client.1.smithi149.stdout:8/347: link d1/l13 d1/de/d18/d32/d63/l74 0 2024-04-24T08:24:45.746 INFO:tasks.workunit.client.0.smithi090.stdout:1/75: rename d3/fb to d3/d12/f1b 0 2024-04-24T08:24:45.746 INFO:tasks.workunit.client.0.smithi090.stdout:5/118: dwrite f7 [4194304,4194304] 0 2024-04-24T08:24:45.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/76: read d3/d5/f14 [1936828,46459] 0 2024-04-24T08:24:45.749 INFO:tasks.workunit.client.0.smithi090.stdout:1/77: fdatasync d3/d5/f17 0 2024-04-24T08:24:45.753 INFO:tasks.workunit.client.1.smithi149.stdout:9/454: creat d1/da/d13/d67/f98 x:0 0 0 2024-04-24T08:24:45.760 INFO:tasks.workunit.client.1.smithi149.stdout:8/348: stat d1/de/d18/d32/d65/f66 0 2024-04-24T08:24:45.768 INFO:tasks.workunit.client.1.smithi149.stdout:4/407: dwrite d2/d18/d1e/d3b/f7c [0,4194304] 0 2024-04-24T08:24:45.769 INFO:tasks.workunit.client.1.smithi149.stdout:5/480: dwrite db/d12/f82 [0,4194304] 0 2024-04-24T08:24:45.772 INFO:tasks.workunit.client.0.smithi090.stdout:5/119: fdatasync f7 0 2024-04-24T08:24:45.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:45 smithi149 ceph-mon[31282]: pgmap v15: 65 pgs: 65 active+clean; 843 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 105 MiB/s wr, 198 op/s 2024-04-24T08:24:45.780 INFO:tasks.workunit.client.0.smithi090.stdout:5/120: dread d8/da/fd [0,4194304] 0 2024-04-24T08:24:45.782 INFO:tasks.workunit.client.1.smithi149.stdout:0/369: dread d2/d9/f13 [0,4194304] 0 2024-04-24T08:24:45.785 INFO:tasks.workunit.client.1.smithi149.stdout:0/370: chown d2/d9/df/f1b 55120238 1 2024-04-24T08:24:45.801 INFO:tasks.workunit.client.0.smithi090.stdout:5/121: dwrite d8/da/d18/f22 [0,4194304] 0 2024-04-24T08:24:45.811 INFO:tasks.workunit.client.0.smithi090.stdout:8/41: truncate d2/f7 3838564 0 2024-04-24T08:24:45.819 INFO:tasks.workunit.client.1.smithi149.stdout:5/481: dread db/d12/d1e/f3c [0,4194304] 0 2024-04-24T08:24:45.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/122: chown f2 0 1 2024-04-24T08:24:45.826 INFO:tasks.workunit.client.1.smithi149.stdout:0/371: mknod d2/c73 0 2024-04-24T08:24:45.827 INFO:tasks.workunit.client.1.smithi149.stdout:0/372: fdatasync d2/d9/df/f16 0 2024-04-24T08:24:45.829 INFO:tasks.workunit.client.1.smithi149.stdout:9/455: mkdir d1/d99 0 2024-04-24T08:24:45.831 INFO:tasks.workunit.client.0.smithi090.stdout:8/42: symlink d2/da/lf 0 2024-04-24T08:24:45.832 INFO:tasks.workunit.client.1.smithi149.stdout:9/456: read d1/d2/d34/f78 [3996013,98826] 0 2024-04-24T08:24:45.832 INFO:tasks.workunit.client.0.smithi090.stdout:5/123: dread d8/da/d13/f23 [0,4194304] 0 2024-04-24T08:24:45.835 INFO:tasks.workunit.client.1.smithi149.stdout:9/457: write d1/da/d13/d67/f98 [632348,41033] 0 2024-04-24T08:24:45.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/78: link d3/l13 d3/d5/l1c 0 2024-04-24T08:24:45.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/114: link db/c14 db/d17/c2a 0 2024-04-24T08:24:45.842 INFO:tasks.workunit.client.0.smithi090.stdout:7/115: write db/f15 [761078,111525] 0 2024-04-24T08:24:45.843 INFO:tasks.workunit.client.0.smithi090.stdout:8/43: creat d2/f10 x:0 0 0 2024-04-24T08:24:45.845 INFO:tasks.workunit.client.0.smithi090.stdout:6/114: link d8/dc/l17 d8/d1d/d1f/l25 0 2024-04-24T08:24:45.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/124: mknod d8/c2c 0 2024-04-24T08:24:45.851 INFO:tasks.workunit.client.0.smithi090.stdout:1/79: readlink d3/d5/l1c 0 2024-04-24T08:24:45.854 INFO:tasks.workunit.client.0.smithi090.stdout:7/116: unlink db/c24 0 2024-04-24T08:24:45.855 INFO:tasks.workunit.client.1.smithi149.stdout:4/408: dwrite d2/f9 [0,4194304] 0 2024-04-24T08:24:45.855 INFO:tasks.workunit.client.1.smithi149.stdout:3/478: getdents d0/d5/d53/d39 0 2024-04-24T08:24:45.855 INFO:tasks.workunit.client.0.smithi090.stdout:7/117: fsync db/fd 0 2024-04-24T08:24:45.858 INFO:tasks.workunit.client.1.smithi149.stdout:8/349: creat d1/de/d18/d32/f75 x:0 0 0 2024-04-24T08:24:45.867 INFO:tasks.workunit.client.0.smithi090.stdout:5/125: mkdir d8/da/d13/d2d 0 2024-04-24T08:24:45.868 INFO:tasks.workunit.client.1.smithi149.stdout:4/409: creat d2/d18/d1e/d3c/d48/f87 x:0 0 0 2024-04-24T08:24:45.869 INFO:tasks.workunit.client.0.smithi090.stdout:1/80: mkdir d3/d1d 0 2024-04-24T08:24:45.874 INFO:tasks.workunit.client.0.smithi090.stdout:7/118: dwrite db/f11 [0,4194304] 0 2024-04-24T08:24:45.874 INFO:tasks.workunit.client.1.smithi149.stdout:8/350: mkdir d1/de/d18/d32/d63/d76 0 2024-04-24T08:24:45.877 INFO:tasks.workunit.client.1.smithi149.stdout:9/458: symlink d1/da/l9a 0 2024-04-24T08:24:45.880 INFO:tasks.workunit.client.1.smithi149.stdout:9/459: dread d1/d2/d33/f5f [0,4194304] 0 2024-04-24T08:24:45.882 INFO:tasks.workunit.client.0.smithi090.stdout:5/126: dread f7 [4194304,4194304] 0 2024-04-24T08:24:45.882 INFO:tasks.workunit.client.0.smithi090.stdout:8/44: link d2/l4 d2/l11 0 2024-04-24T08:24:45.883 INFO:tasks.workunit.client.1.smithi149.stdout:8/351: creat d1/d1f/d31/f77 x:0 0 0 2024-04-24T08:24:45.883 INFO:tasks.workunit.client.0.smithi090.stdout:8/45: stat d2/f3 0 2024-04-24T08:24:45.885 INFO:tasks.workunit.client.0.smithi090.stdout:6/115: link d8/lb d8/dc/l26 0 2024-04-24T08:24:45.886 INFO:tasks.workunit.client.0.smithi090.stdout:6/116: chown d8/d1d 1756281090 1 2024-04-24T08:24:45.890 INFO:tasks.workunit.client.0.smithi090.stdout:6/117: write d8/d1d/d1f/f23 [504189,25036] 0 2024-04-24T08:24:45.891 INFO:tasks.workunit.client.0.smithi090.stdout:1/81: dwrite d3/d12/f18 [0,4194304] 0 2024-04-24T08:24:45.893 INFO:tasks.workunit.client.1.smithi149.stdout:8/352: creat d1/d1f/d25/f78 x:0 0 0 2024-04-24T08:24:45.896 INFO:tasks.workunit.client.0.smithi090.stdout:7/119: rename l8 to db/d17/l2b 0 2024-04-24T08:24:45.897 INFO:tasks.workunit.client.0.smithi090.stdout:7/120: chown db/f12 523278 1 2024-04-24T08:24:45.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/410: truncate d2/d10/d1a/d34/f37 1565371 0 2024-04-24T08:24:45.899 INFO:tasks.workunit.client.1.smithi149.stdout:4/411: fsync d2/d10/f31 0 2024-04-24T08:24:45.904 INFO:tasks.workunit.client.1.smithi149.stdout:4/412: chown d2/d24/d2e/d3f/f40 110174337 1 2024-04-24T08:24:45.904 INFO:tasks.workunit.client.1.smithi149.stdout:8/353: creat d1/d1f/d31/f79 x:0 0 0 2024-04-24T08:24:45.907 INFO:tasks.workunit.client.1.smithi149.stdout:8/354: chown d1/d1f/d25/l58 111 1 2024-04-24T08:24:45.910 INFO:tasks.workunit.client.0.smithi090.stdout:6/118: link d8/d1d/d1f/f23 d8/dc/d19/f27 0 2024-04-24T08:24:45.910 INFO:tasks.workunit.client.1.smithi149.stdout:9/460: dread d1/d2/d15/f30 [0,4194304] 0 2024-04-24T08:24:45.911 INFO:tasks.workunit.client.1.smithi149.stdout:4/413: unlink d2/d18/d1e/d3b/f7b 0 2024-04-24T08:24:45.913 INFO:tasks.workunit.client.1.smithi149.stdout:8/355: mkdir d1/d1f/d25/d7a 0 2024-04-24T08:24:45.914 INFO:tasks.workunit.client.1.smithi149.stdout:6/314: sync 2024-04-24T08:24:45.914 INFO:tasks.workunit.client.1.smithi149.stdout:2/338: sync 2024-04-24T08:24:45.918 INFO:tasks.workunit.client.0.smithi090.stdout:6/119: dread d8/fa [0,4194304] 0 2024-04-24T08:24:45.922 INFO:tasks.workunit.client.1.smithi149.stdout:3/479: rmdir d0/d5/d53/d34 39 2024-04-24T08:24:45.924 INFO:tasks.workunit.client.1.smithi149.stdout:2/339: dread d0/d6/db/d27/f47 [0,4194304] 0 2024-04-24T08:24:45.925 INFO:tasks.workunit.client.0.smithi090.stdout:7/121: link db/c14 db/d17/c2c 0 2024-04-24T08:24:45.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/315: creat d0/d1a/d28/d4a/f67 x:0 0 0 2024-04-24T08:24:45.932 INFO:tasks.workunit.client.0.smithi090.stdout:6/120: getdents d8/d1d 0 2024-04-24T08:24:45.933 INFO:tasks.workunit.client.0.smithi090.stdout:7/122: dread f3 [0,4194304] 0 2024-04-24T08:24:45.934 INFO:tasks.workunit.client.1.smithi149.stdout:8/356: link d1/de/d18/d32/l53 d1/d1f/d31/l7b 0 2024-04-24T08:24:45.934 INFO:tasks.workunit.client.1.smithi149.stdout:9/461: dread d1/de/f18 [0,4194304] 0 2024-04-24T08:24:45.935 INFO:tasks.workunit.client.0.smithi090.stdout:2/56: sync 2024-04-24T08:24:45.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/139: sync 2024-04-24T08:24:45.936 INFO:tasks.workunit.client.1.smithi149.stdout:5/482: rmdir db/d12/d22 39 2024-04-24T08:24:45.937 INFO:tasks.workunit.client.0.smithi090.stdout:2/57: dread d0/f9 [0,4194304] 0 2024-04-24T08:24:45.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/121: chown d8/dc/l17 20 1 2024-04-24T08:24:45.937 INFO:tasks.workunit.client.0.smithi090.stdout:2/58: readlink d0/la 0 2024-04-24T08:24:45.938 INFO:tasks.workunit.client.0.smithi090.stdout:2/59: chown d0/d7 70238 1 2024-04-24T08:24:45.938 INFO:tasks.workunit.client.1.smithi149.stdout:0/373: dwrite d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:45.939 INFO:tasks.workunit.client.1.smithi149.stdout:8/357: truncate d1/de/d18/f4c 165480 0 2024-04-24T08:24:45.940 INFO:tasks.workunit.client.1.smithi149.stdout:6/316: chown d0/f12 508 1 2024-04-24T08:24:45.940 INFO:tasks.workunit.client.1.smithi149.stdout:8/358: stat d1/de/f1a 0 2024-04-24T08:24:45.942 INFO:tasks.workunit.client.1.smithi149.stdout:8/359: dread - d1/d1f/d31/f77 zero size 2024-04-24T08:24:45.947 INFO:tasks.workunit.client.1.smithi149.stdout:3/480: rmdir d0/d26/d3c/d99 0 2024-04-24T08:24:45.948 INFO:tasks.workunit.client.0.smithi090.stdout:6/122: symlink d8/l28 0 2024-04-24T08:24:45.951 INFO:tasks.workunit.client.1.smithi149.stdout:5/483: creat db/d12/d2f/d74/f97 x:0 0 0 2024-04-24T08:24:45.951 INFO:tasks.workunit.client.0.smithi090.stdout:7/123: truncate db/d17/f18 393711 0 2024-04-24T08:24:45.957 INFO:tasks.workunit.client.1.smithi149.stdout:6/317: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:45.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/60: dwrite d0/f9 [0,4194304] 0 2024-04-24T08:24:45.959 INFO:tasks.workunit.client.1.smithi149.stdout:2/340: link d0/f2d d0/d6/db/d28/f6f 0 2024-04-24T08:24:45.960 INFO:tasks.workunit.client.1.smithi149.stdout:2/341: stat d0/d6c 0 2024-04-24T08:24:45.961 INFO:tasks.workunit.client.1.smithi149.stdout:6/318: write d0/d1a/d28/d31/f4b [798734,127242] 0 2024-04-24T08:24:45.963 INFO:tasks.workunit.client.0.smithi090.stdout:7/124: dread db/f12 [0,4194304] 0 2024-04-24T08:24:45.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/125: chown db/d17/f28 3 1 2024-04-24T08:24:45.965 INFO:tasks.workunit.client.1.smithi149.stdout:6/319: dread - d0/d1a/d28/d4a/f67 zero size 2024-04-24T08:24:45.973 INFO:tasks.workunit.client.1.smithi149.stdout:8/360: getdents d1/d1f/d25/d7a 0 2024-04-24T08:24:45.980 INFO:tasks.workunit.client.0.smithi090.stdout:0/140: getdents d1a 0 2024-04-24T08:24:45.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/141: chown d1a/c21 374 1 2024-04-24T08:24:45.984 INFO:tasks.workunit.client.0.smithi090.stdout:1/82: write d3/d5/f14 [4517130,38925] 0 2024-04-24T08:24:45.986 INFO:tasks.workunit.client.1.smithi149.stdout:6/320: dwrite d0/d1a/d28/f2c [0,4194304] 0 2024-04-24T08:24:45.988 INFO:tasks.workunit.client.0.smithi090.stdout:1/83: write d3/d5/da/dc/fe [4230428,53079] 0 2024-04-24T08:24:45.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/321: write d0/d1a/d28/d31/f4b [629854,3290] 0 2024-04-24T08:24:45.992 INFO:tasks.workunit.client.0.smithi090.stdout:5/127: truncate d8/f1c 1722832 0 2024-04-24T08:24:45.993 INFO:tasks.workunit.client.1.smithi149.stdout:1/519: sync 2024-04-24T08:24:45.994 INFO:tasks.workunit.client.1.smithi149.stdout:7/420: sync 2024-04-24T08:24:45.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/142: dread d1a/f20 [0,4194304] 0 2024-04-24T08:24:45.999 INFO:tasks.workunit.client.0.smithi090.stdout:2/61: link d0/la d0/lb 0 2024-04-24T08:24:46.000 INFO:tasks.workunit.client.1.smithi149.stdout:1/520: readlink d0/d39/d53/l73 0 2024-04-24T08:24:46.000 INFO:tasks.workunit.client.1.smithi149.stdout:7/421: readlink d9/dc/de/l1c 0 2024-04-24T08:24:46.000 INFO:tasks.workunit.client.1.smithi149.stdout:8/361: rename d1/de/d18/d32/f75 to d1/d1f/d5e/d6c/f7c 0 2024-04-24T08:24:46.000 INFO:tasks.workunit.client.1.smithi149.stdout:8/362: dread - d1/de/d18/d32/d65/f66 zero size 2024-04-24T08:24:46.001 INFO:tasks.workunit.client.1.smithi149.stdout:0/374: dread d2/fb [4194304,4194304] 0 2024-04-24T08:24:46.001 INFO:tasks.workunit.client.1.smithi149.stdout:8/363: chown d1/de/d18/d32/d63/d76 457 1 2024-04-24T08:24:46.002 INFO:tasks.workunit.client.1.smithi149.stdout:3/481: link d0/d5/da/d19/d1b/d22/d38/f50 d0/d5/d53/d34/fa9 0 2024-04-24T08:24:46.003 INFO:tasks.workunit.client.1.smithi149.stdout:0/375: truncate d2/d9/df/d40/f70 182655 0 2024-04-24T08:24:46.005 INFO:tasks.workunit.client.0.smithi090.stdout:1/84: mknod d3/d12/c1e 0 2024-04-24T08:24:46.006 INFO:tasks.workunit.client.0.smithi090.stdout:1/85: read - d3/d5/f19 zero size 2024-04-24T08:24:46.006 INFO:tasks.workunit.client.1.smithi149.stdout:0/376: write d2/d9/df/d48/f4c [2321144,71701] 0 2024-04-24T08:24:46.007 INFO:tasks.workunit.client.0.smithi090.stdout:7/126: creat db/d17/f2d x:0 0 0 2024-04-24T08:24:46.007 INFO:tasks.workunit.client.1.smithi149.stdout:4/414: write d2/f14 [2953403,79617] 0 2024-04-24T08:24:46.011 INFO:tasks.workunit.client.0.smithi090.stdout:5/128: symlink d8/da/l2e 0 2024-04-24T08:24:46.012 INFO:tasks.workunit.client.0.smithi090.stdout:2/62: symlink d0/lc 0 2024-04-24T08:24:46.015 INFO:tasks.workunit.client.1.smithi149.stdout:2/342: creat d0/f70 x:0 0 0 2024-04-24T08:24:46.019 INFO:tasks.workunit.client.1.smithi149.stdout:3/482: dwrite d0/d88/f8e [0,4194304] 0 2024-04-24T08:24:46.025 INFO:tasks.workunit.client.1.smithi149.stdout:8/364: mkdir d1/d1f/d5e/d6c/d7d 0 2024-04-24T08:24:46.028 INFO:tasks.workunit.client.0.smithi090.stdout:2/63: write d0/d7/f8 [7944106,20492] 0 2024-04-24T08:24:46.029 INFO:tasks.workunit.client.0.smithi090.stdout:5/129: dwrite f2 [0,4194304] 0 2024-04-24T08:24:46.029 INFO:tasks.workunit.client.1.smithi149.stdout:0/377: dwrite d2/d9/df/d48/f66 [0,4194304] 0 2024-04-24T08:24:46.034 INFO:tasks.workunit.client.1.smithi149.stdout:9/462: truncate d1/de/d12/f1d 3376828 0 2024-04-24T08:24:46.034 INFO:tasks.workunit.client.0.smithi090.stdout:6/123: truncate d8/dc/f11 3910137 0 2024-04-24T08:24:46.036 INFO:tasks.workunit.client.1.smithi149.stdout:5/484: link db/d1f/l3a db/d12/d22/d6c/l98 0 2024-04-24T08:24:46.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/86: rename c1 to d3/c1f 0 2024-04-24T08:24:46.041 INFO:tasks.workunit.client.1.smithi149.stdout:0/378: chown d2/d9/df/d40/c45 16826 1 2024-04-24T08:24:46.041 INFO:tasks.workunit.client.1.smithi149.stdout:0/379: chown d2/d9/df 4402371 1 2024-04-24T08:24:46.041 INFO:tasks.workunit.client.1.smithi149.stdout:6/322: creat d0/d1a/d28/d59/f68 x:0 0 0 2024-04-24T08:24:46.045 INFO:tasks.workunit.client.1.smithi149.stdout:7/422: dwrite d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:46.048 INFO:tasks.workunit.client.0.smithi090.stdout:2/64: dread d0/f9 [0,4194304] 0 2024-04-24T08:24:46.050 INFO:tasks.workunit.client.0.smithi090.stdout:5/130: unlink d8/cf 0 2024-04-24T08:24:46.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/131: dread - d8/da/d14/f27 zero size 2024-04-24T08:24:46.052 INFO:tasks.workunit.client.1.smithi149.stdout:1/521: dwrite d0/d5/db/f72 [0,4194304] 0 2024-04-24T08:24:46.053 INFO:tasks.workunit.client.0.smithi090.stdout:6/124: stat d8/f15 0 2024-04-24T08:24:46.054 INFO:tasks.workunit.client.0.smithi090.stdout:7/127: creat db/d17/f2e x:0 0 0 2024-04-24T08:24:46.054 INFO:tasks.workunit.client.0.smithi090.stdout:5/132: dread - d8/da/f1b zero size 2024-04-24T08:24:46.056 INFO:tasks.workunit.client.0.smithi090.stdout:6/125: truncate d8/d1d/f1e 858249 0 2024-04-24T08:24:46.059 INFO:tasks.workunit.client.0.smithi090.stdout:1/87: symlink d3/d5/da/dc/l20 0 2024-04-24T08:24:46.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/483: symlink d0/d5/laa 0 2024-04-24T08:24:46.063 INFO:tasks.workunit.client.1.smithi149.stdout:4/415: link d2/d24/d2e/d38/f5e d2/d18/f88 0 2024-04-24T08:24:46.066 INFO:tasks.workunit.client.0.smithi090.stdout:2/65: rename d0/d7/f8 to d0/d7/fd 0 2024-04-24T08:24:46.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/38: sync 2024-04-24T08:24:46.066 INFO:tasks.workunit.client.0.smithi090.stdout:3/48: sync 2024-04-24T08:24:46.066 INFO:tasks.workunit.client.0.smithi090.stdout:4/55: sync 2024-04-24T08:24:46.069 INFO:tasks.workunit.client.1.smithi149.stdout:5/485: symlink db/d12/d2f/d78/l99 0 2024-04-24T08:24:46.069 INFO:tasks.workunit.client.1.smithi149.stdout:0/380: dread d2/d9/df/f16 [0,4194304] 0 2024-04-24T08:24:46.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/49: fsync d2/f7 0 2024-04-24T08:24:46.073 INFO:tasks.workunit.client.1.smithi149.stdout:2/343: unlink d0/d6/db/l4b 0 2024-04-24T08:24:46.074 INFO:tasks.workunit.client.1.smithi149.stdout:2/344: readlink d0/d6/db/d39/l62 0 2024-04-24T08:24:46.076 INFO:tasks.workunit.client.0.smithi090.stdout:6/126: dwrite d8/d1d/f24 [0,4194304] 0 2024-04-24T08:24:46.077 INFO:tasks.workunit.client.0.smithi090.stdout:0/143: truncate d1a/f1e 3003707 0 2024-04-24T08:24:46.078 INFO:tasks.workunit.client.0.smithi090.stdout:5/133: mkdir d8/da/d14/d2f 0 2024-04-24T08:24:46.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/134: write d8/d25/f26 [502219,22988] 0 2024-04-24T08:24:46.087 INFO:tasks.workunit.client.1.smithi149.stdout:1/522: truncate d0/d39/da0/f5e 976468 0 2024-04-24T08:24:46.088 INFO:tasks.workunit.client.0.smithi090.stdout:1/88: dwrite d3/d12/f18 [4194304,4194304] 0 2024-04-24T08:24:46.089 INFO:tasks.workunit.client.1.smithi149.stdout:9/463: unlink d1/da/dd/d79/c61 0 2024-04-24T08:24:46.090 INFO:tasks.workunit.client.1.smithi149.stdout:2/345: dread d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:46.094 INFO:tasks.workunit.client.0.smithi090.stdout:9/39: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:46.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/66: dread d0/f9 [0,4194304] 0 2024-04-24T08:24:46.101 INFO:tasks.workunit.client.0.smithi090.stdout:3/50: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:46.105 INFO:tasks.workunit.client.0.smithi090.stdout:7/128: link f9 db/d17/d29/f2f 0 2024-04-24T08:24:46.109 INFO:tasks.workunit.client.0.smithi090.stdout:0/144: rmdir d1a/d2a 39 2024-04-24T08:24:46.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/365: dwrite d1/de/f54 [0,4194304] 0 2024-04-24T08:24:46.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/40: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:46.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/423: mknod d9/dc/d11/d38/c8a 0 2024-04-24T08:24:46.122 INFO:tasks.workunit.client.1.smithi149.stdout:4/416: dread d2/d10/d1a/f36 [0,4194304] 0 2024-04-24T08:24:46.125 INFO:tasks.workunit.client.1.smithi149.stdout:4/417: chown d2/l7e 6149069 1 2024-04-24T08:24:46.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/56: truncate d3/f9 1124070 0 2024-04-24T08:24:46.130 INFO:tasks.workunit.client.0.smithi090.stdout:9/41: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:46.131 INFO:tasks.workunit.client.0.smithi090.stdout:4/57: write d3/f7 [1028925,46944] 0 2024-04-24T08:24:46.131 INFO:tasks.workunit.client.1.smithi149.stdout:1/523: mknod d0/d5/d4f/cab 0 2024-04-24T08:24:46.136 INFO:tasks.workunit.client.1.smithi149.stdout:2/346: truncate d0/f35 699030 0 2024-04-24T08:24:46.136 INFO:tasks.workunit.client.0.smithi090.stdout:3/51: rename d2/f3 to d2/f9 0 2024-04-24T08:24:46.137 INFO:tasks.workunit.client.1.smithi149.stdout:5/486: dwrite db/d12/d22/d45/d5d/f71 [0,4194304] 0 2024-04-24T08:24:46.141 INFO:tasks.workunit.client.1.smithi149.stdout:0/381: link d2/f8 d2/d9/d1a/d36/f74 0 2024-04-24T08:24:46.141 INFO:tasks.workunit.client.0.smithi090.stdout:8/46: sync 2024-04-24T08:24:46.145 INFO:tasks.workunit.client.1.smithi149.stdout:0/382: truncate d2/d17/f5e 283937 0 2024-04-24T08:24:46.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/487: truncate db/d12/d2f/d74/f97 825266 0 2024-04-24T08:24:46.148 INFO:tasks.workunit.client.1.smithi149.stdout:8/366: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:24:46.148 INFO:tasks.workunit.client.0.smithi090.stdout:5/135: symlink d8/da/d13/d2d/l30 0 2024-04-24T08:24:46.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/67: dwrite d0/f9 [4194304,4194304] 0 2024-04-24T08:24:46.150 INFO:tasks.workunit.client.0.smithi090.stdout:5/136: readlink d8/le 0 2024-04-24T08:24:46.151 INFO:tasks.workunit.client.1.smithi149.stdout:0/383: write d2/d9/df/d48/f4c [4550668,15585] 0 2024-04-24T08:24:46.153 INFO:tasks.workunit.client.1.smithi149.stdout:4/418: mknod d2/d18/d1e/d3c/d48/c89 0 2024-04-24T08:24:46.155 INFO:tasks.workunit.client.1.smithi149.stdout:4/419: truncate d2/f9 4877864 0 2024-04-24T08:24:46.156 INFO:tasks.workunit.client.1.smithi149.stdout:1/524: dread d0/d5/df/f28 [0,4194304] 0 2024-04-24T08:24:46.161 INFO:tasks.workunit.client.1.smithi149.stdout:1/525: write d0/f25 [2276659,75556] 0 2024-04-24T08:24:46.167 INFO:tasks.workunit.client.1.smithi149.stdout:9/464: dread d1/da/f27 [0,4194304] 0 2024-04-24T08:24:46.179 INFO:tasks.workunit.client.1.smithi149.stdout:6/323: getdents d0/d1a/d28/d31/d4c/d4e/d62 0 2024-04-24T08:24:46.184 INFO:tasks.workunit.client.1.smithi149.stdout:8/367: creat d1/de/d18/f7e x:0 0 0 2024-04-24T08:24:46.185 INFO:tasks.workunit.client.1.smithi149.stdout:0/384: write d2/d9/df/d40/d4a/f64 [1962241,60632] 0 2024-04-24T08:24:46.190 INFO:tasks.workunit.client.1.smithi149.stdout:9/465: creat d1/d2/d34/f9b x:0 0 0 2024-04-24T08:24:46.191 INFO:tasks.workunit.client.1.smithi149.stdout:5/488: mknod db/d12/d60/d8f/c9a 0 2024-04-24T08:24:46.193 INFO:tasks.workunit.client.1.smithi149.stdout:5/489: write db/d12/d22/d45/f93 [123774,19404] 0 2024-04-24T08:24:46.218 INFO:tasks.workunit.client.1.smithi149.stdout:7/424: getdents d9/dc/d7b 0 2024-04-24T08:24:46.218 INFO:tasks.workunit.client.1.smithi149.stdout:4/420: creat d2/f8a x:0 0 0 2024-04-24T08:24:46.221 INFO:tasks.workunit.client.1.smithi149.stdout:7/425: chown d9/dc/de/f31 153682678 1 2024-04-24T08:24:46.221 INFO:tasks.workunit.client.1.smithi149.stdout:5/490: truncate db/d23/f80 666056 0 2024-04-24T08:24:46.221 INFO:tasks.workunit.client.1.smithi149.stdout:0/385: truncate d2/d9/f2d 2256392 0 2024-04-24T08:24:46.222 INFO:tasks.workunit.client.1.smithi149.stdout:9/466: mknod d1/d2/d15/d1c/c9c 0 2024-04-24T08:24:46.222 INFO:tasks.workunit.client.1.smithi149.stdout:1/526: getdents d0/d5/db/d82 0 2024-04-24T08:24:46.225 INFO:tasks.workunit.client.1.smithi149.stdout:1/527: fsync d0/da/fa3 0 2024-04-24T08:24:46.226 INFO:tasks.workunit.client.1.smithi149.stdout:2/347: dread - d0/d6/db/d39/d3d/d44/d57/f5a zero size 2024-04-24T08:24:46.229 INFO:tasks.workunit.client.1.smithi149.stdout:8/368: dwrite d1/d1f/d31/f79 [0,4194304] 0 2024-04-24T08:24:46.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/58: symlink d3/lc 0 2024-04-24T08:24:46.231 INFO:tasks.workunit.client.1.smithi149.stdout:9/467: mkdir d1/de/d9d 0 2024-04-24T08:24:46.233 INFO:tasks.workunit.client.0.smithi090.stdout:3/52: rename d2/f7 to d2/fa 0 2024-04-24T08:24:46.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/491: rename db/d12/d1e/c40 to db/d12/d2f/c9b 0 2024-04-24T08:24:46.235 INFO:tasks.workunit.client.0.smithi090.stdout:7/129: creat db/d17/d29/f30 x:0 0 0 2024-04-24T08:24:46.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/145: readlink d1a/d2a/l2b 0 2024-04-24T08:24:46.236 INFO:tasks.workunit.client.0.smithi090.stdout:2/68: mkdir d0/d7/de 0 2024-04-24T08:24:46.241 INFO:tasks.workunit.client.0.smithi090.stdout:6/127: dwrite d8/f15 [0,4194304] 0 2024-04-24T08:24:46.244 INFO:tasks.workunit.client.0.smithi090.stdout:6/128: fdatasync d8/d1d/f24 0 2024-04-24T08:24:46.245 INFO:tasks.workunit.client.0.smithi090.stdout:4/59: dread d3/f7 [0,4194304] 0 2024-04-24T08:24:46.245 INFO:tasks.workunit.client.0.smithi090.stdout:6/129: dread - d8/dc/f1a zero size 2024-04-24T08:24:46.246 INFO:tasks.workunit.client.1.smithi149.stdout:0/386: dwrite d2/d9/df/d48/f4f [0,4194304] 0 2024-04-24T08:24:46.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/60: write d3/f7 [1812706,103010] 0 2024-04-24T08:24:46.248 INFO:tasks.workunit.client.1.smithi149.stdout:8/369: symlink d1/d1f/d5e/l7f 0 2024-04-24T08:24:46.248 INFO:tasks.workunit.client.1.smithi149.stdout:7/426: dwrite d9/dc/d11/f1e [0,4194304] 0 2024-04-24T08:24:46.254 INFO:tasks.workunit.client.1.smithi149.stdout:5/492: mknod db/d12/d22/d6c/c9c 0 2024-04-24T08:24:46.262 INFO:tasks.workunit.client.1.smithi149.stdout:0/387: creat d2/d9/df/d2f/d6c/f75 x:0 0 0 2024-04-24T08:24:46.265 INFO:tasks.workunit.client.1.smithi149.stdout:8/370: creat d1/d1f/d25/f80 x:0 0 0 2024-04-24T08:24:46.271 INFO:tasks.workunit.client.1.smithi149.stdout:9/468: creat d1/da/dd/f9e x:0 0 0 2024-04-24T08:24:46.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/137: creat d8/f31 x:0 0 0 2024-04-24T08:24:46.283 INFO:tasks.workunit.client.1.smithi149.stdout:9/469: write d1/d2/d15/d1c/f2a [2637391,2108] 0 2024-04-24T08:24:46.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/89: link d3/c9 d3/d5/c21 0 2024-04-24T08:24:46.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/53: mknod d2/cb 0 2024-04-24T08:24:46.288 INFO:tasks.workunit.client.1.smithi149.stdout:2/348: dwrite d0/d6/db/d39/d3d/f6b [0,4194304] 0 2024-04-24T08:24:46.290 INFO:tasks.workunit.client.0.smithi090.stdout:2/69: creat d0/ff x:0 0 0 2024-04-24T08:24:46.290 INFO:tasks.workunit.client.1.smithi149.stdout:1/528: dread d0/d5/db/d63/f6b [0,4194304] 0 2024-04-24T08:24:46.293 INFO:tasks.workunit.client.0.smithi090.stdout:5/138: symlink d8/l32 0 2024-04-24T08:24:46.295 INFO:tasks.workunit.client.0.smithi090.stdout:5/139: dread - d8/da/d14/f27 zero size 2024-04-24T08:24:46.298 INFO:tasks.workunit.client.1.smithi149.stdout:5/493: dwrite db/f94 [0,4194304] 0 2024-04-24T08:24:46.299 INFO:tasks.workunit.client.1.smithi149.stdout:7/427: dwrite d9/dc/d11/f37 [0,4194304] 0 2024-04-24T08:24:46.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/54: symlink d2/lc 0 2024-04-24T08:24:46.305 INFO:tasks.workunit.client.0.smithi090.stdout:7/130: mknod db/d17/c31 0 2024-04-24T08:24:46.307 INFO:tasks.workunit.client.1.smithi149.stdout:7/428: read d9/dc/f18 [228436,125843] 0 2024-04-24T08:24:46.308 INFO:tasks.workunit.client.0.smithi090.stdout:2/70: symlink d0/l10 0 2024-04-24T08:24:46.309 INFO:tasks.workunit.client.0.smithi090.stdout:7/131: write f7 [1940809,73655] 0 2024-04-24T08:24:46.309 INFO:tasks.workunit.client.0.smithi090.stdout:2/71: dread - d0/ff zero size 2024-04-24T08:24:46.309 INFO:tasks.workunit.client.0.smithi090.stdout:6/130: dwrite d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:46.312 INFO:tasks.workunit.client.1.smithi149.stdout:8/371: dwrite d1/d1f/d22/f33 [0,4194304] 0 2024-04-24T08:24:46.312 INFO:tasks.workunit.client.0.smithi090.stdout:6/131: dread - d8/f1b zero size 2024-04-24T08:24:46.315 INFO:tasks.workunit.client.0.smithi090.stdout:8/47: getdents d2/da 0 2024-04-24T08:24:46.315 INFO:tasks.workunit.client.1.smithi149.stdout:8/372: truncate d1/de/f6d 734325 0 2024-04-24T08:24:46.316 INFO:tasks.workunit.client.1.smithi149.stdout:1/529: dread d0/d5/f41 [8388608,4194304] 0 2024-04-24T08:24:46.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/55: symlink d2/ld 0 2024-04-24T08:24:46.324 INFO:tasks.workunit.client.0.smithi090.stdout:7/132: mkdir db/d17/d29/d32 0 2024-04-24T08:24:46.325 INFO:tasks.workunit.client.0.smithi090.stdout:5/140: dwrite d8/f31 [0,4194304] 0 2024-04-24T08:24:46.340 INFO:tasks.workunit.client.0.smithi090.stdout:6/132: fdatasync d8/dc/d19/f27 0 2024-04-24T08:24:46.342 INFO:tasks.workunit.client.0.smithi090.stdout:0/146: getdents d1a/d2a 0 2024-04-24T08:24:46.343 INFO:tasks.workunit.client.0.smithi090.stdout:3/56: mknod d2/ce 0 2024-04-24T08:24:46.348 INFO:tasks.workunit.client.0.smithi090.stdout:7/133: unlink f3 0 2024-04-24T08:24:46.351 INFO:tasks.workunit.client.0.smithi090.stdout:5/141: symlink d8/da/d13/l33 0 2024-04-24T08:24:46.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/90: getdents d3/d5/da/dc 0 2024-04-24T08:24:46.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/142: write d8/da/f1b [530565,99842] 0 2024-04-24T08:24:46.359 INFO:tasks.workunit.client.0.smithi090.stdout:3/57: mkdir d2/df 0 2024-04-24T08:24:46.361 INFO:tasks.workunit.client.1.smithi149.stdout:0/388: creat d2/d9/df/d40/d4a/d4e/d6f/f76 x:0 0 0 2024-04-24T08:24:46.362 INFO:tasks.workunit.client.1.smithi149.stdout:5/494: creat db/d35/f9d x:0 0 0 2024-04-24T08:24:46.363 INFO:tasks.workunit.client.1.smithi149.stdout:0/389: chown d2/d9/df/d48/f66 1020 1 2024-04-24T08:24:46.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/134: unlink db/d17/d1b/l1d 0 2024-04-24T08:24:46.365 INFO:tasks.workunit.client.1.smithi149.stdout:2/349: creat d0/d6/db/d39/d3d/d44/d57/f71 x:0 0 0 2024-04-24T08:24:46.367 INFO:tasks.workunit.client.1.smithi149.stdout:2/350: write d0/d6/f12 [1104693,9739] 0 2024-04-24T08:24:46.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/72: dwrite d0/d7/fd [0,4194304] 0 2024-04-24T08:24:46.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/351: fdatasync d0/d6/db/d27/f5f 0 2024-04-24T08:24:46.370 INFO:tasks.workunit.client.0.smithi090.stdout:2/73: chown d0/d7/de 27941 1 2024-04-24T08:24:46.373 INFO:tasks.workunit.client.0.smithi090.stdout:7/135: symlink db/d17/d29/l33 0 2024-04-24T08:24:46.378 INFO:tasks.workunit.client.0.smithi090.stdout:3/58: creat d2/df/f10 x:0 0 0 2024-04-24T08:24:46.381 INFO:tasks.workunit.client.0.smithi090.stdout:7/136: dread - db/d17/d29/f30 zero size 2024-04-24T08:24:46.381 INFO:tasks.workunit.client.0.smithi090.stdout:5/143: link d8/c12 d8/c34 0 2024-04-24T08:24:46.382 INFO:tasks.workunit.client.0.smithi090.stdout:0/147: link d1a/c21 d1a/c33 0 2024-04-24T08:24:46.386 INFO:tasks.workunit.client.0.smithi090.stdout:8/48: write d2/f7 [1588338,124540] 0 2024-04-24T08:24:46.388 INFO:tasks.workunit.client.0.smithi090.stdout:3/59: write d2/f9 [3910314,14516] 0 2024-04-24T08:24:46.389 INFO:tasks.workunit.client.0.smithi090.stdout:2/74: getdents d0/d7 0 2024-04-24T08:24:46.390 INFO:tasks.workunit.client.0.smithi090.stdout:1/91: dwrite d3/d5/f17 [0,4194304] 0 2024-04-24T08:24:46.393 INFO:tasks.workunit.client.0.smithi090.stdout:4/61: dwrite d3/f9 [0,4194304] 0 2024-04-24T08:24:46.396 INFO:tasks.workunit.client.0.smithi090.stdout:5/144: truncate d8/da/d14/f27 390707 0 2024-04-24T08:24:46.398 INFO:tasks.workunit.client.0.smithi090.stdout:0/148: read d1a/d2a/f27 [417078,35190] 0 2024-04-24T08:24:46.399 INFO:tasks.workunit.client.0.smithi090.stdout:2/75: creat d0/d7/f11 x:0 0 0 2024-04-24T08:24:46.407 INFO:tasks.workunit.client.0.smithi090.stdout:1/92: dread d3/d5/f14 [4194304,4194304] 0 2024-04-24T08:24:46.407 INFO:tasks.workunit.client.0.smithi090.stdout:5/145: fsync d8/da/d18/f1e 0 2024-04-24T08:24:46.410 INFO:tasks.workunit.client.0.smithi090.stdout:7/137: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:46.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/49: dread d2/f3 [0,4194304] 0 2024-04-24T08:24:46.414 INFO:tasks.workunit.client.0.smithi090.stdout:4/62: rename d3/f7 to d3/fd 0 2024-04-24T08:24:46.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/138: creat db/f34 x:0 0 0 2024-04-24T08:24:46.419 INFO:tasks.workunit.client.0.smithi090.stdout:3/60: link d2/c4 d2/df/c11 0 2024-04-24T08:24:46.420 INFO:tasks.workunit.client.0.smithi090.stdout:5/146: write d8/da/d13/f23 [3495828,129019] 0 2024-04-24T08:24:46.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/139: dread - db/d17/f2d zero size 2024-04-24T08:24:46.423 INFO:tasks.workunit.client.1.smithi149.stdout:3/484: sync 2024-04-24T08:24:46.424 INFO:tasks.workunit.client.1.smithi149.stdout:0/390: creat d2/d9/df/d2f/d6c/f77 x:0 0 0 2024-04-24T08:24:46.424 INFO:tasks.workunit.client.0.smithi090.stdout:3/61: chown d2/ce 15746280 1 2024-04-24T08:24:46.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/140: fdatasync f7 0 2024-04-24T08:24:46.425 INFO:tasks.workunit.client.0.smithi090.stdout:0/149: dread f8 [0,4194304] 0 2024-04-24T08:24:46.425 INFO:tasks.workunit.client.1.smithi149.stdout:7/429: truncate d9/dc/de/f33 3850744 0 2024-04-24T08:24:46.427 INFO:tasks.workunit.client.1.smithi149.stdout:7/430: stat d9/d43/l70 0 2024-04-24T08:24:46.428 INFO:tasks.workunit.client.1.smithi149.stdout:2/352: getdents d0/d6/db/d39 0 2024-04-24T08:24:46.429 INFO:tasks.workunit.client.0.smithi090.stdout:1/93: dread d3/d12/f18 [4194304,4194304] 0 2024-04-24T08:24:46.431 INFO:tasks.workunit.client.0.smithi090.stdout:7/141: symlink db/d17/d1b/l35 0 2024-04-24T08:24:46.431 INFO:tasks.workunit.client.0.smithi090.stdout:3/62: chown d2/ld 422 1 2024-04-24T08:24:46.432 INFO:tasks.workunit.client.0.smithi090.stdout:0/150: symlink d1a/l34 0 2024-04-24T08:24:46.432 INFO:tasks.workunit.client.1.smithi149.stdout:5/495: dwrite db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:46.434 INFO:tasks.workunit.client.0.smithi090.stdout:5/147: dread d8/da/f1d [0,4194304] 0 2024-04-24T08:24:46.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/485: read d0/d5/da/d5a/d63/d6a/f7c [611259,90248] 0 2024-04-24T08:24:46.437 INFO:tasks.workunit.client.0.smithi090.stdout:0/151: chown d1a/l22 331618 1 2024-04-24T08:24:46.438 INFO:tasks.workunit.client.0.smithi090.stdout:2/76: dwrite d0/d7/f11 [0,4194304] 0 2024-04-24T08:24:46.438 INFO:tasks.workunit.client.0.smithi090.stdout:3/63: write d2/f9 [880339,82499] 0 2024-04-24T08:24:46.438 INFO:tasks.workunit.client.1.smithi149.stdout:7/431: creat d9/f8b x:0 0 0 2024-04-24T08:24:46.438 INFO:tasks.workunit.client.1.smithi149.stdout:1/530: dread d0/d5/db/d18/f29 [0,4194304] 0 2024-04-24T08:24:46.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/64: chown d2/c8 99 1 2024-04-24T08:24:46.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/50: dwrite d2/f10 [0,4194304] 0 2024-04-24T08:24:46.450 INFO:tasks.workunit.client.1.smithi149.stdout:3/486: mknod d0/d5/d53/d34/d3a/cab 0 2024-04-24T08:24:46.451 INFO:tasks.workunit.client.1.smithi149.stdout:5/496: creat db/d12/d2f/d95/f9e x:0 0 0 2024-04-24T08:24:46.454 INFO:tasks.workunit.client.1.smithi149.stdout:1/531: mkdir d0/d5/d4f/dac 0 2024-04-24T08:24:46.455 INFO:tasks.workunit.client.0.smithi090.stdout:0/152: mkdir d1a/d2a/d35 0 2024-04-24T08:24:46.459 INFO:tasks.workunit.client.0.smithi090.stdout:2/77: symlink d0/l12 0 2024-04-24T08:24:46.459 INFO:tasks.workunit.client.0.smithi090.stdout:4/63: dread d3/fd [0,4194304] 0 2024-04-24T08:24:46.461 INFO:tasks.workunit.client.1.smithi149.stdout:2/353: creat d0/d6/db/f72 x:0 0 0 2024-04-24T08:24:46.463 INFO:tasks.workunit.client.1.smithi149.stdout:5/497: readlink db/d12/l58 0 2024-04-24T08:24:46.465 INFO:tasks.workunit.client.0.smithi090.stdout:7/142: creat db/d17/d29/d32/f36 x:0 0 0 2024-04-24T08:24:46.465 INFO:tasks.workunit.client.0.smithi090.stdout:5/148: dwrite d8/f31 [4194304,4194304] 0 2024-04-24T08:24:46.466 INFO:tasks.workunit.client.1.smithi149.stdout:3/487: mkdir d0/d5/da/d19/d80/dac 0 2024-04-24T08:24:46.467 INFO:tasks.workunit.client.1.smithi149.stdout:1/532: symlink d0/d1b/d4b/lad 0 2024-04-24T08:24:46.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/149: readlink d8/da/l2e 0 2024-04-24T08:24:46.468 INFO:tasks.workunit.client.0.smithi090.stdout:7/143: dread - db/d17/d29/f30 zero size 2024-04-24T08:24:46.469 INFO:tasks.workunit.client.1.smithi149.stdout:2/354: creat d0/d6/db/d28/f73 x:0 0 0 2024-04-24T08:24:46.475 INFO:tasks.workunit.client.0.smithi090.stdout:0/153: mknod d1a/d2a/d35/c36 0 2024-04-24T08:24:46.476 INFO:tasks.workunit.client.0.smithi090.stdout:1/94: dwrite d3/f4 [0,4194304] 0 2024-04-24T08:24:46.478 INFO:tasks.workunit.client.1.smithi149.stdout:7/432: write d9/d43/d4f/d66/f6e [3721630,18547] 0 2024-04-24T08:24:46.480 INFO:tasks.workunit.client.1.smithi149.stdout:1/533: creat d0/d39/d76/fae x:0 0 0 2024-04-24T08:24:46.482 INFO:tasks.workunit.client.0.smithi090.stdout:7/144: mknod db/d17/d29/c37 0 2024-04-24T08:24:46.486 INFO:tasks.workunit.client.0.smithi090.stdout:5/150: dwrite f7 [0,4194304] 0 2024-04-24T08:24:46.488 INFO:tasks.workunit.client.1.smithi149.stdout:3/488: dwrite d0/d5/ff [0,4194304] 0 2024-04-24T08:24:46.488 INFO:tasks.workunit.client.1.smithi149.stdout:3/489: readlink d0/d5/laa 0 2024-04-24T08:24:46.490 INFO:tasks.workunit.client.1.smithi149.stdout:7/433: creat d9/dc/d72/f8c x:0 0 0 2024-04-24T08:24:46.492 INFO:tasks.workunit.client.1.smithi149.stdout:1/534: rmdir d0/d1b/d5d 39 2024-04-24T08:24:46.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/51: dwrite d2/f9 [0,4194304] 0 2024-04-24T08:24:46.495 INFO:tasks.workunit.client.0.smithi090.stdout:2/78: rmdir d0/d7/de 0 2024-04-24T08:24:46.498 INFO:tasks.workunit.client.1.smithi149.stdout:1/535: write d0/d5/d4f/d60/f8f [731546,10331] 0 2024-04-24T08:24:46.502 INFO:tasks.workunit.client.1.smithi149.stdout:2/355: dwrite d0/f2 [4194304,4194304] 0 2024-04-24T08:24:46.509 INFO:tasks.workunit.client.0.smithi090.stdout:7/145: truncate f4 360398 0 2024-04-24T08:24:46.509 INFO:tasks.workunit.client.1.smithi149.stdout:5/498: dwrite db/d12/d2f/f36 [0,4194304] 0 2024-04-24T08:24:46.510 INFO:tasks.workunit.client.1.smithi149.stdout:2/356: write d0/d6/db/f4a [814655,81396] 0 2024-04-24T08:24:46.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/52: creat d2/f12 x:0 0 0 2024-04-24T08:24:46.515 INFO:tasks.workunit.client.1.smithi149.stdout:5/499: write db/d12/d2f/d95/f9e [528577,22100] 0 2024-04-24T08:24:46.517 INFO:tasks.workunit.client.0.smithi090.stdout:2/79: symlink d0/d7/l13 0 2024-04-24T08:24:46.518 INFO:tasks.workunit.client.1.smithi149.stdout:7/434: mknod d9/d43/d64/c8d 0 2024-04-24T08:24:46.525 INFO:tasks.workunit.client.0.smithi090.stdout:9/42: sync 2024-04-24T08:24:46.527 INFO:tasks.workunit.client.1.smithi149.stdout:3/490: rename d0/d5/d53/d34/f81 to d0/d5/d73/fad 0 2024-04-24T08:24:46.527 INFO:tasks.workunit.client.0.smithi090.stdout:7/146: mkdir db/d17/d29/d38 0 2024-04-24T08:24:46.528 INFO:tasks.workunit.client.1.smithi149.stdout:2/357: read d0/d6/db/d27/f3b [2434,129569] 0 2024-04-24T08:24:46.529 INFO:tasks.workunit.client.0.smithi090.stdout:8/53: fsync d2/f3 0 2024-04-24T08:24:46.529 INFO:tasks.workunit.client.0.smithi090.stdout:2/80: mkdir d0/d14 0 2024-04-24T08:24:46.535 INFO:tasks.workunit.client.1.smithi149.stdout:1/536: rename d0/d5/db/d18/d34 to d0/d5/db/d18/d4d/d50/daf 0 2024-04-24T08:24:46.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/537: chown d0/d39/da0 1522529 1 2024-04-24T08:24:46.538 INFO:tasks.workunit.client.1.smithi149.stdout:7/435: dread d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:24:46.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/147: write db/f12 [2296514,53714] 0 2024-04-24T08:24:46.541 INFO:tasks.workunit.client.1.smithi149.stdout:2/358: creat d0/d6/db/d27/d64/f74 x:0 0 0 2024-04-24T08:24:46.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/54: mkdir d2/da/d13 0 2024-04-24T08:24:46.544 INFO:tasks.workunit.client.1.smithi149.stdout:5/500: rename db/d12/d22/d45/d5d/f71 to db/d12/d1e/d85/d92/f9f 0 2024-04-24T08:24:46.546 INFO:tasks.workunit.client.1.smithi149.stdout:5/501: write db/d1f/f30 [2688098,102289] 0 2024-04-24T08:24:46.548 INFO:tasks.workunit.client.0.smithi090.stdout:5/151: getdents d8 0 2024-04-24T08:24:46.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/81: link d0/ff d0/d14/f15 0 2024-04-24T08:24:46.553 INFO:tasks.workunit.client.0.smithi090.stdout:0/154: dread d1a/f1b [0,4194304] 0 2024-04-24T08:24:46.554 INFO:tasks.workunit.client.0.smithi090.stdout:8/55: mknod d2/c14 0 2024-04-24T08:24:46.554 INFO:tasks.workunit.client.0.smithi090.stdout:5/152: write d8/da/d13/f23 [1665131,75696] 0 2024-04-24T08:24:46.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/43: rmdir d0/d7 0 2024-04-24T08:24:46.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/44: chown d0/f2 0 1 2024-04-24T08:24:46.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/148: dwrite db/d17/f2e [0,4194304] 0 2024-04-24T08:24:46.559 INFO:tasks.workunit.client.1.smithi149.stdout:1/538: symlink d0/d5/lb0 0 2024-04-24T08:24:46.559 INFO:tasks.workunit.client.0.smithi090.stdout:9/45: chown d0/f2 2 1 2024-04-24T08:24:46.560 INFO:tasks.workunit.client.1.smithi149.stdout:5/502: readlink db/d12/d22/d45/l55 0 2024-04-24T08:24:46.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/82: symlink d0/l16 0 2024-04-24T08:24:46.563 INFO:tasks.workunit.client.0.smithi090.stdout:0/155: rmdir d1a 39 2024-04-24T08:24:46.566 INFO:tasks.workunit.client.0.smithi090.stdout:5/153: write d8/da/d14/f17 [1429822,1471] 0 2024-04-24T08:24:46.566 INFO:tasks.workunit.client.1.smithi149.stdout:5/503: mkdir db/d12/d60/d8f/da0 0 2024-04-24T08:24:46.567 INFO:tasks.workunit.client.0.smithi090.stdout:2/83: fdatasync d0/d7/fd 0 2024-04-24T08:24:46.567 INFO:tasks.workunit.client.0.smithi090.stdout:8/56: mkdir d2/da/d13/d15 0 2024-04-24T08:24:46.568 INFO:tasks.workunit.client.0.smithi090.stdout:5/154: dread - d8/da/f20 zero size 2024-04-24T08:24:46.569 INFO:tasks.workunit.client.1.smithi149.stdout:2/359: getdents d0/d6 0 2024-04-24T08:24:46.569 INFO:tasks.workunit.client.0.smithi090.stdout:8/57: chown d2/f12 163279 1 2024-04-24T08:24:46.570 INFO:tasks.workunit.client.0.smithi090.stdout:0/156: unlink f14 0 2024-04-24T08:24:46.571 INFO:tasks.workunit.client.0.smithi090.stdout:7/149: mkdir db/d17/d39 0 2024-04-24T08:24:46.572 INFO:tasks.workunit.client.1.smithi149.stdout:7/436: getdents d9/dc/d11/d38 0 2024-04-24T08:24:46.576 INFO:tasks.workunit.client.0.smithi090.stdout:9/46: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:46.577 INFO:tasks.workunit.client.1.smithi149.stdout:2/360: mkdir d0/d6/db/d39/d3d/d44/d75 0 2024-04-24T08:24:46.578 INFO:tasks.workunit.client.1.smithi149.stdout:1/539: getdents d0/d5/db/d18/d2f 0 2024-04-24T08:24:46.580 INFO:tasks.workunit.client.1.smithi149.stdout:5/504: rename db/d1f/c3e to db/d12/d22/ca1 0 2024-04-24T08:24:46.581 INFO:tasks.workunit.client.1.smithi149.stdout:2/361: unlink d0/d6/db/c30 0 2024-04-24T08:24:46.584 INFO:tasks.workunit.client.1.smithi149.stdout:2/362: write d0/d6/db/d28/f3a [2861190,128476] 0 2024-04-24T08:24:46.585 INFO:tasks.workunit.client.0.smithi090.stdout:7/150: dread db/d17/d1b/f23 [0,4194304] 0 2024-04-24T08:24:46.586 INFO:tasks.workunit.client.0.smithi090.stdout:8/58: creat d2/da/d13/d15/f16 x:0 0 0 2024-04-24T08:24:46.586 INFO:tasks.workunit.client.1.smithi149.stdout:5/505: creat db/d12/d2f/d74/fa2 x:0 0 0 2024-04-24T08:24:46.589 INFO:tasks.workunit.client.1.smithi149.stdout:7/437: rename d9/dc/d11/f61 to d9/d43/f8e 0 2024-04-24T08:24:46.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/59: write d2/f10 [1050233,25291] 0 2024-04-24T08:24:46.591 INFO:tasks.workunit.client.0.smithi090.stdout:0/157: rename d1a/l23 to d1a/l37 0 2024-04-24T08:24:46.592 INFO:tasks.workunit.client.1.smithi149.stdout:2/363: creat d0/d6/db/d27/d64/f76 x:0 0 0 2024-04-24T08:24:46.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/60: fsync d2/f12 0 2024-04-24T08:24:46.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/506: symlink db/d12/d1e/d85/la3 0 2024-04-24T08:24:46.595 INFO:tasks.workunit.client.1.smithi149.stdout:5/507: fsync db/d12/f87 0 2024-04-24T08:24:46.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/540: getdents d0/d5/db/d63 0 2024-04-24T08:24:46.596 INFO:tasks.workunit.client.0.smithi090.stdout:2/84: dwrite d0/d14/f15 [0,4194304] 0 2024-04-24T08:24:46.597 INFO:tasks.workunit.client.1.smithi149.stdout:7/438: readlink d9/dc/d11/l27 0 2024-04-24T08:24:46.597 INFO:tasks.workunit.client.1.smithi149.stdout:5/508: chown db/d12/d22/c86 53 1 2024-04-24T08:24:46.598 INFO:tasks.workunit.client.0.smithi090.stdout:9/47: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:46.598 INFO:tasks.workunit.client.1.smithi149.stdout:4/421: sync 2024-04-24T08:24:46.598 INFO:tasks.workunit.client.1.smithi149.stdout:6/324: sync 2024-04-24T08:24:46.599 INFO:tasks.workunit.client.1.smithi149.stdout:7/439: stat d9/dc/d11/d26/f4a 0 2024-04-24T08:24:46.600 INFO:tasks.workunit.client.1.smithi149.stdout:1/541: rmdir d0/d39 39 2024-04-24T08:24:46.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/151: creat db/d17/f3a x:0 0 0 2024-04-24T08:24:46.602 INFO:tasks.workunit.client.0.smithi090.stdout:0/158: chown d1a/d2a/c29 0 1 2024-04-24T08:24:46.603 INFO:tasks.workunit.client.0.smithi090.stdout:8/61: symlink d2/da/d13/d15/l17 0 2024-04-24T08:24:46.604 INFO:tasks.workunit.client.0.smithi090.stdout:0/159: fdatasync d1a/d2a/f32 0 2024-04-24T08:24:46.604 INFO:tasks.workunit.client.0.smithi090.stdout:7/152: write db/fd [2715032,89861] 0 2024-04-24T08:24:46.606 INFO:tasks.workunit.client.1.smithi149.stdout:6/325: fdatasync d0/f3 0 2024-04-24T08:24:46.610 INFO:tasks.workunit.client.0.smithi090.stdout:8/62: chown l1 1 1 2024-04-24T08:24:46.610 INFO:tasks.workunit.client.1.smithi149.stdout:6/326: fsync d0/d1a/d28/d31/f5c 0 2024-04-24T08:24:46.611 INFO:tasks.workunit.client.1.smithi149.stdout:2/364: creat d0/f77 x:0 0 0 2024-04-24T08:24:46.612 INFO:tasks.workunit.client.0.smithi090.stdout:7/153: creat db/d17/d29/f3b x:0 0 0 2024-04-24T08:24:46.616 INFO:tasks.workunit.client.0.smithi090.stdout:7/154: write db/fc [616728,92247] 0 2024-04-24T08:24:46.616 INFO:tasks.workunit.client.0.smithi090.stdout:0/160: truncate d1a/d2a/f27 1914887 0 2024-04-24T08:24:46.616 INFO:tasks.workunit.client.0.smithi090.stdout:2/85: dread d0/ff [0,4194304] 0 2024-04-24T08:24:46.617 INFO:tasks.workunit.client.0.smithi090.stdout:8/63: creat d2/da/d13/f18 x:0 0 0 2024-04-24T08:24:46.618 INFO:tasks.workunit.client.1.smithi149.stdout:2/365: mkdir d0/d6/db/d39/d78 0 2024-04-24T08:24:46.618 INFO:tasks.workunit.client.1.smithi149.stdout:1/542: dread d0/f5f [0,4194304] 0 2024-04-24T08:24:46.621 INFO:tasks.workunit.client.1.smithi149.stdout:4/422: getdents d2/d18/d1e/d3b 0 2024-04-24T08:24:46.622 INFO:tasks.workunit.client.1.smithi149.stdout:6/327: link d0/d1a/f29 d0/d1a/f69 0 2024-04-24T08:24:46.622 INFO:tasks.workunit.client.0.smithi090.stdout:7/155: dread db/f21 [0,4194304] 0 2024-04-24T08:24:46.623 INFO:tasks.workunit.client.1.smithi149.stdout:6/328: fsync d0/fe 0 2024-04-24T08:24:46.625 INFO:tasks.workunit.client.1.smithi149.stdout:4/423: write d2/f44 [611424,33286] 0 2024-04-24T08:24:46.625 INFO:tasks.workunit.client.0.smithi090.stdout:2/86: write d0/d14/f15 [1805261,75988] 0 2024-04-24T08:24:46.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/424: read - d2/d24/d2e/d38/f7a zero size 2024-04-24T08:24:46.628 INFO:tasks.workunit.client.1.smithi149.stdout:5/509: dwrite db/d12/d22/d45/f42 [0,4194304] 0 2024-04-24T08:24:46.630 INFO:tasks.workunit.client.0.smithi090.stdout:8/64: mkdir d2/d19 0 2024-04-24T08:24:46.630 INFO:tasks.workunit.client.1.smithi149.stdout:7/440: dread d9/dc/f17 [4194304,4194304] 0 2024-04-24T08:24:46.638 INFO:tasks.workunit.client.0.smithi090.stdout:0/161: creat d1a/f38 x:0 0 0 2024-04-24T08:24:46.638 INFO:tasks.workunit.client.1.smithi149.stdout:3/491: dwrite d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:46.647 INFO:tasks.workunit.client.1.smithi149.stdout:6/329: mknod d0/c6a 0 2024-04-24T08:24:46.650 INFO:tasks.workunit.client.0.smithi090.stdout:2/87: mkdir d0/d14/d17 0 2024-04-24T08:24:46.652 INFO:tasks.workunit.client.1.smithi149.stdout:9/470: sync 2024-04-24T08:24:46.653 INFO:tasks.workunit.client.1.smithi149.stdout:5/510: mkdir db/da4 0 2024-04-24T08:24:46.655 INFO:tasks.workunit.client.0.smithi090.stdout:8/65: rename d2/l4 to d2/da/d13/d15/l1a 0 2024-04-24T08:24:46.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/66: dread - d2/f12 zero size 2024-04-24T08:24:46.658 INFO:tasks.workunit.client.0.smithi090.stdout:7/156: link db/d17/f18 db/d17/d39/f3c 0 2024-04-24T08:24:46.659 INFO:tasks.workunit.client.0.smithi090.stdout:8/67: write d2/f7 [2331532,121481] 0 2024-04-24T08:24:46.660 INFO:tasks.workunit.client.0.smithi090.stdout:7/157: chown db/d17/d1b/l35 23212947 1 2024-04-24T08:24:46.660 INFO:tasks.workunit.client.1.smithi149.stdout:4/425: symlink d2/d24/d2e/l8b 0 2024-04-24T08:24:46.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/158: truncate db/d17/f3a 106277 0 2024-04-24T08:24:46.663 INFO:tasks.workunit.client.0.smithi090.stdout:8/68: dread d2/fd [0,4194304] 0 2024-04-24T08:24:46.663 INFO:tasks.workunit.client.0.smithi090.stdout:5/155: truncate f2 1505959 0 2024-04-24T08:24:46.664 INFO:tasks.workunit.client.0.smithi090.stdout:2/88: rename d0/f9 to d0/d14/d17/f18 0 2024-04-24T08:24:46.667 INFO:tasks.workunit.client.1.smithi149.stdout:9/471: symlink d1/d2/d33/d5d/l9f 0 2024-04-24T08:24:46.669 INFO:tasks.workunit.client.1.smithi149.stdout:9/472: stat d1/da/dd/c7e 0 2024-04-24T08:24:46.669 INFO:tasks.workunit.client.1.smithi149.stdout:5/511: dwrite db/d12/d2f/f57 [4194304,4194304] 0 2024-04-24T08:24:46.673 INFO:tasks.workunit.client.1.smithi149.stdout:4/426: rename d2/d18/d1e/c72 to d2/d18/c8c 0 2024-04-24T08:24:46.673 INFO:tasks.workunit.client.0.smithi090.stdout:9/48: write d0/f4 [4426134,63928] 0 2024-04-24T08:24:46.674 INFO:tasks.workunit.client.0.smithi090.stdout:7/159: creat db/d17/d29/d32/f3d x:0 0 0 2024-04-24T08:24:46.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/49: write d0/f4 [2611880,20258] 0 2024-04-24T08:24:46.675 INFO:tasks.workunit.client.1.smithi149.stdout:7/441: link d9/dc/d2e/c7a d9/dc/de/c8f 0 2024-04-24T08:24:46.676 INFO:tasks.workunit.client.0.smithi090.stdout:7/160: write db/d17/f28 [388018,123082] 0 2024-04-24T08:24:46.677 INFO:tasks.workunit.client.0.smithi090.stdout:9/50: chown d0/f3 5893 1 2024-04-24T08:24:46.677 INFO:tasks.workunit.client.1.smithi149.stdout:6/330: write d0/f3 [1544025,106918] 0 2024-04-24T08:24:46.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/427: write d2/d18/d1e/f33 [987796,101409] 0 2024-04-24T08:24:46.678 INFO:tasks.workunit.client.0.smithi090.stdout:8/69: readlink d2/da/le 0 2024-04-24T08:24:46.683 INFO:tasks.workunit.client.1.smithi149.stdout:6/331: chown d0/d1a/f29 10690727 1 2024-04-24T08:24:46.686 INFO:tasks.workunit.client.1.smithi149.stdout:5/512: unlink db/d23/f44 0 2024-04-24T08:24:46.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/162: getdents d1a/d2a/d35 0 2024-04-24T08:24:46.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/442: dwrite d9/dc/d72/f8c [0,4194304] 0 2024-04-24T08:24:46.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/51: symlink d0/l8 0 2024-04-24T08:24:46.698 INFO:tasks.workunit.client.1.smithi149.stdout:1/543: write d0/da/f24 [4469699,100989] 0 2024-04-24T08:24:46.699 INFO:tasks.workunit.client.1.smithi149.stdout:6/332: mknod d0/d1a/d28/d31/c6b 0 2024-04-24T08:24:46.701 INFO:tasks.workunit.client.0.smithi090.stdout:8/70: mknod d2/da/c1b 0 2024-04-24T08:24:46.701 INFO:tasks.workunit.client.1.smithi149.stdout:1/544: truncate d0/f84 840752 0 2024-04-24T08:24:46.702 INFO:tasks.workunit.client.1.smithi149.stdout:5/513: creat db/d12/d1e/d85/fa5 x:0 0 0 2024-04-24T08:24:46.707 INFO:tasks.workunit.client.1.smithi149.stdout:8/373: sync 2024-04-24T08:24:46.707 INFO:tasks.workunit.client.1.smithi149.stdout:0/391: sync 2024-04-24T08:24:46.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/156: link d8/da/d18/f22 d8/da/f35 0 2024-04-24T08:24:46.710 INFO:tasks.workunit.client.1.smithi149.stdout:4/428: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d 0 2024-04-24T08:24:46.711 INFO:tasks.workunit.client.1.smithi149.stdout:8/374: dread d1/de/f47 [0,4194304] 0 2024-04-24T08:24:46.712 INFO:tasks.workunit.client.0.smithi090.stdout:0/163: fsync d1a/f20 0 2024-04-24T08:24:46.713 INFO:tasks.workunit.client.1.smithi149.stdout:7/443: mknod d9/d43/d4f/d66/c90 0 2024-04-24T08:24:46.713 INFO:tasks.workunit.client.0.smithi090.stdout:0/164: fdatasync f15 0 2024-04-24T08:24:46.715 INFO:tasks.workunit.client.0.smithi090.stdout:9/52: symlink d0/l9 0 2024-04-24T08:24:46.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/71: rename d2/f9 to d2/da/f1c 0 2024-04-24T08:24:46.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/492: write d0/d5/d53/d6d/f9a [2581179,47904] 0 2024-04-24T08:24:46.722 INFO:tasks.workunit.client.0.smithi090.stdout:2/89: link d0/l16 d0/d7/l19 0 2024-04-24T08:24:46.723 INFO:tasks.workunit.client.1.smithi149.stdout:1/545: chown d0/d39/d53/d61/c7a 1833 1 2024-04-24T08:24:46.723 INFO:tasks.workunit.client.0.smithi090.stdout:5/157: dwrite d8/da/f20 [0,4194304] 0 2024-04-24T08:24:46.725 INFO:tasks.workunit.client.1.smithi149.stdout:5/514: unlink db/d12/d22/d45/d5d/f66 0 2024-04-24T08:24:46.726 INFO:tasks.workunit.client.0.smithi090.stdout:0/165: rmdir d1a/d2a/d35 39 2024-04-24T08:24:46.729 INFO:tasks.workunit.client.1.smithi149.stdout:8/375: dwrite d1/de/f38 [0,4194304] 0 2024-04-24T08:24:46.730 INFO:tasks.workunit.client.0.smithi090.stdout:9/53: creat d0/fa x:0 0 0 2024-04-24T08:24:46.732 INFO:tasks.workunit.client.0.smithi090.stdout:9/54: write d0/f4 [133251,54375] 0 2024-04-24T08:24:46.735 INFO:tasks.workunit.client.1.smithi149.stdout:0/392: creat d2/d3c/d57/f78 x:0 0 0 2024-04-24T08:24:46.736 INFO:tasks.workunit.client.1.smithi149.stdout:4/429: fsync d2/d18/d1e/d3b/f78 0 2024-04-24T08:24:46.739 INFO:tasks.workunit.client.1.smithi149.stdout:9/473: write d1/de/d12/f96 [558087,63278] 0 2024-04-24T08:24:46.740 INFO:tasks.workunit.client.0.smithi090.stdout:8/72: symlink d2/da/d13/l1d 0 2024-04-24T08:24:46.742 INFO:tasks.workunit.client.0.smithi090.stdout:8/73: dread - d2/da/d13/d15/f16 zero size 2024-04-24T08:24:46.744 INFO:tasks.workunit.client.1.smithi149.stdout:2/366: creat d0/d6/db/d39/d3d/d44/d75/f79 x:0 0 0 2024-04-24T08:24:46.749 INFO:tasks.workunit.client.0.smithi090.stdout:5/158: mkdir d8/da/d13/d36 0 2024-04-24T08:24:46.754 INFO:tasks.workunit.client.1.smithi149.stdout:1/546: dwrite d0/d39/d53/d61/f70 [0,4194304] 0 2024-04-24T08:24:46.756 INFO:tasks.workunit.client.1.smithi149.stdout:1/547: chown d0/f25 7 1 2024-04-24T08:24:46.757 INFO:tasks.workunit.client.0.smithi090.stdout:9/55: truncate d0/f2 627264 0 2024-04-24T08:24:46.761 INFO:tasks.workunit.client.1.smithi149.stdout:5/515: mkdir db/d12/d1e/d85/d92/da6 0 2024-04-24T08:24:46.761 INFO:tasks.workunit.client.1.smithi149.stdout:4/430: dwrite d2/d24/d2e/f66 [0,4194304] 0 2024-04-24T08:24:46.762 INFO:tasks.workunit.client.0.smithi090.stdout:8/74: creat d2/da/d13/d15/f1e x:0 0 0 2024-04-24T08:24:46.767 INFO:tasks.workunit.client.0.smithi090.stdout:8/75: truncate d2/da/d13/d15/f16 961096 0 2024-04-24T08:24:46.767 INFO:tasks.workunit.client.0.smithi090.stdout:2/90: rename d0/d7/l19 to d0/d7/l1a 0 2024-04-24T08:24:46.769 INFO:tasks.workunit.client.1.smithi149.stdout:8/376: creat d1/de/d18/d32/d65/f81 x:0 0 0 2024-04-24T08:24:46.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/91: fdatasync d0/d7/fd 0 2024-04-24T08:24:46.771 INFO:tasks.workunit.client.1.smithi149.stdout:8/377: fdatasync d1/d1f/d5e/f73 0 2024-04-24T08:24:46.772 INFO:tasks.workunit.client.1.smithi149.stdout:0/393: fdatasync d2/d9/fe 0 2024-04-24T08:24:46.778 INFO:tasks.workunit.client.1.smithi149.stdout:0/394: write d2/d9/d1a/f25 [1622704,53155] 0 2024-04-24T08:24:46.778 INFO:tasks.workunit.client.1.smithi149.stdout:3/493: dread d0/f11 [0,4194304] 0 2024-04-24T08:24:46.780 INFO:tasks.workunit.client.0.smithi090.stdout:9/56: creat d0/fb x:0 0 0 2024-04-24T08:24:46.781 INFO:tasks.workunit.client.1.smithi149.stdout:0/395: fsync d2/d17/f5e 0 2024-04-24T08:24:46.782 INFO:tasks.workunit.client.1.smithi149.stdout:2/367: dread d0/d6/db/f1c [0,4194304] 0 2024-04-24T08:24:46.783 INFO:tasks.workunit.client.1.smithi149.stdout:9/474: rename d1/da/d13/f64 to d1/d2/fa0 0 2024-04-24T08:24:46.784 INFO:tasks.workunit.client.1.smithi149.stdout:6/333: dwrite d0/d1a/f69 [0,4194304] 0 2024-04-24T08:24:46.788 INFO:tasks.workunit.client.0.smithi090.stdout:2/92: dwrite d0/d7/fd [4194304,4194304] 0 2024-04-24T08:24:46.793 INFO:tasks.workunit.client.0.smithi090.stdout:0/166: rename d1a/c21 to d1a/d2a/c39 0 2024-04-24T08:24:46.802 INFO:tasks.workunit.client.1.smithi149.stdout:3/494: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:46.803 INFO:tasks.workunit.client.0.smithi090.stdout:2/93: creat d0/d14/d17/f1b x:0 0 0 2024-04-24T08:24:46.804 INFO:tasks.workunit.client.1.smithi149.stdout:3/495: stat d0/d5/d53/f47 0 2024-04-24T08:24:46.808 INFO:tasks.workunit.client.1.smithi149.stdout:8/378: mknod d1/d1f/d22/c82 0 2024-04-24T08:24:46.815 INFO:tasks.workunit.client.0.smithi090.stdout:0/167: dwrite d1a/f38 [0,4194304] 0 2024-04-24T08:24:46.815 INFO:tasks.workunit.client.0.smithi090.stdout:5/159: write d8/da/fd [150168,57971] 0 2024-04-24T08:24:46.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/57: getdents d0 0 2024-04-24T08:24:46.824 INFO:tasks.workunit.client.1.smithi149.stdout:7/444: truncate d9/dc/d11/d38/f4e 3294582 0 2024-04-24T08:24:46.825 INFO:tasks.workunit.client.0.smithi090.stdout:5/160: mkdir d8/da/d13/d2d/d37 0 2024-04-24T08:24:46.831 INFO:tasks.workunit.client.0.smithi090.stdout:9/58: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:46.832 INFO:tasks.workunit.client.0.smithi090.stdout:9/59: read d0/f5 [3049252,68048] 0 2024-04-24T08:24:46.835 INFO:tasks.workunit.client.0.smithi090.stdout:5/161: mknod d8/da/d18/c38 0 2024-04-24T08:24:46.835 INFO:tasks.workunit.client.0.smithi090.stdout:2/94: rmdir d0/d14/d17 39 2024-04-24T08:24:46.836 INFO:tasks.workunit.client.1.smithi149.stdout:9/475: creat d1/da/dd/d79/fa1 x:0 0 0 2024-04-24T08:24:46.838 INFO:tasks.workunit.client.1.smithi149.stdout:6/334: creat d0/d1a/d28/d31/f6c x:0 0 0 2024-04-24T08:24:46.840 INFO:tasks.workunit.client.1.smithi149.stdout:9/476: write d1/d2/d15/f81 [807038,39317] 0 2024-04-24T08:24:46.844 INFO:tasks.workunit.client.1.smithi149.stdout:0/396: dread d2/d9/d1a/d36/f47 [0,4194304] 0 2024-04-24T08:24:46.844 INFO:tasks.workunit.client.0.smithi090.stdout:2/95: dread d0/ff [0,4194304] 0 2024-04-24T08:24:46.845 INFO:tasks.workunit.client.0.smithi090.stdout:2/96: fsync d0/d14/f15 0 2024-04-24T08:24:46.848 INFO:tasks.workunit.client.1.smithi149.stdout:5/516: mkdir db/d12/d22/d45/da7 0 2024-04-24T08:24:46.850 INFO:tasks.workunit.client.1.smithi149.stdout:8/379: symlink d1/d1f/d25/l83 0 2024-04-24T08:24:46.856 INFO:tasks.workunit.client.1.smithi149.stdout:4/431: rename d2/d18/c8c to d2/d18/d1e/c8e 0 2024-04-24T08:24:46.857 INFO:tasks.workunit.client.1.smithi149.stdout:2/368: write d0/d6/db/d28/f48 [219784,71363] 0 2024-04-24T08:24:46.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/168: dwrite d1a/f30 [4194304,4194304] 0 2024-04-24T08:24:46.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/169: readlink d1a/l2e 0 2024-04-24T08:24:46.866 INFO:tasks.workunit.client.1.smithi149.stdout:7/445: dread d9/dc/d11/d26/f4a [0,4194304] 0 2024-04-24T08:24:46.870 INFO:tasks.workunit.client.0.smithi090.stdout:2/97: creat d0/d7/f1c x:0 0 0 2024-04-24T08:24:46.875 INFO:tasks.workunit.client.1.smithi149.stdout:1/548: creat d0/d5/d4f/d60/d19/fb1 x:0 0 0 2024-04-24T08:24:46.877 INFO:tasks.workunit.client.1.smithi149.stdout:1/549: dread - d0/d5/d4f/d60/f78 zero size 2024-04-24T08:24:46.878 INFO:tasks.workunit.client.0.smithi090.stdout:6/133: sync 2024-04-24T08:24:46.880 INFO:tasks.workunit.client.0.smithi090.stdout:6/134: fsync f0 0 2024-04-24T08:24:46.881 INFO:tasks.workunit.client.0.smithi090.stdout:5/162: creat d8/da/d13/d2d/d37/f39 x:0 0 0 2024-04-24T08:24:46.883 INFO:tasks.workunit.client.0.smithi090.stdout:5/163: chown d8/da/d13/d2d 115 1 2024-04-24T08:24:46.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/164: fdatasync d8/da/f1b 0 2024-04-24T08:24:46.884 INFO:tasks.workunit.client.1.smithi149.stdout:8/380: rmdir d1/de/d18/d43 39 2024-04-24T08:24:46.886 INFO:tasks.workunit.client.0.smithi090.stdout:2/98: chown d0/d14/d17/f18 5496975 1 2024-04-24T08:24:46.887 INFO:tasks.workunit.client.0.smithi090.stdout:2/99: fdatasync d0/d7/f11 0 2024-04-24T08:24:46.887 INFO:tasks.workunit.client.1.smithi149.stdout:8/381: chown d1/d1f/d31/f35 179853172 1 2024-04-24T08:24:46.890 INFO:tasks.workunit.client.1.smithi149.stdout:9/477: rename d1/da/dd/d55/l8f to d1/de/d12/la2 0 2024-04-24T08:24:46.891 INFO:tasks.workunit.client.0.smithi090.stdout:6/135: mkdir d8/d1d/d29 0 2024-04-24T08:24:46.896 INFO:tasks.workunit.client.1.smithi149.stdout:5/517: dwrite db/d12/f84 [0,4194304] 0 2024-04-24T08:24:46.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/60: truncate d0/f5 3623265 0 2024-04-24T08:24:46.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/61: stat d0/l8 0 2024-04-24T08:24:46.903 INFO:tasks.workunit.client.0.smithi090.stdout:9/62: mkdir d0/dc 0 2024-04-24T08:24:46.905 INFO:tasks.workunit.client.1.smithi149.stdout:7/446: creat d9/dc/de/d44/f91 x:0 0 0 2024-04-24T08:24:46.906 INFO:tasks.workunit.client.0.smithi090.stdout:6/136: dread d8/d1d/f24 [0,4194304] 0 2024-04-24T08:24:46.908 INFO:tasks.workunit.client.0.smithi090.stdout:6/137: write f6 [1320769,100980] 0 2024-04-24T08:24:46.910 INFO:tasks.workunit.client.1.smithi149.stdout:0/397: write d2/d9/df/f21 [463857,66859] 0 2024-04-24T08:24:46.914 INFO:tasks.workunit.client.0.smithi090.stdout:2/100: dread d0/ff [0,4194304] 0 2024-04-24T08:24:46.915 INFO:tasks.workunit.client.0.smithi090.stdout:2/101: readlink d0/l12 0 2024-04-24T08:24:46.917 INFO:tasks.workunit.client.1.smithi149.stdout:2/369: creat d0/d6/db/d27/d64/d66/f7a x:0 0 0 2024-04-24T08:24:46.919 INFO:tasks.workunit.client.1.smithi149.stdout:4/432: dread d2/ff [0,4194304] 0 2024-04-24T08:24:46.923 INFO:tasks.workunit.client.1.smithi149.stdout:2/370: write d0/d6/db/d39/d3d/d58/f6d [208867,46231] 0 2024-04-24T08:24:46.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/63: creat d0/dc/fd x:0 0 0 2024-04-24T08:24:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/447: chown d9/c10 5 1 2024-04-24T08:24:46.927 INFO:tasks.workunit.client.0.smithi090.stdout:0/170: dwrite f13 [0,4194304] 0 2024-04-24T08:24:46.927 INFO:tasks.workunit.client.1.smithi149.stdout:1/550: dwrite d0/d5/df/f94 [0,4194304] 0 2024-04-24T08:24:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:8/382: unlink d1/de/f2f 0 2024-04-24T08:24:46.930 INFO:tasks.workunit.client.0.smithi090.stdout:5/165: getdents d8/da 0 2024-04-24T08:24:46.932 INFO:tasks.workunit.client.1.smithi149.stdout:8/383: truncate d1/de/f6d 1569999 0 2024-04-24T08:24:46.933 INFO:tasks.workunit.client.0.smithi090.stdout:6/138: creat d8/d1d/d29/f2a x:0 0 0 2024-04-24T08:24:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:5/518: rename db/d12/f1b to db/da4/fa8 0 2024-04-24T08:24:46.937 INFO:tasks.workunit.client.1.smithi149.stdout:2/371: dread d0/d6/db/f42 [0,4194304] 0 2024-04-24T08:24:46.937 INFO:tasks.workunit.client.1.smithi149.stdout:6/335: getdents d0/d1a 0 2024-04-24T08:24:46.940 INFO:tasks.workunit.client.0.smithi090.stdout:9/64: symlink d0/dc/le 0 2024-04-24T08:24:46.942 INFO:tasks.workunit.client.1.smithi149.stdout:4/433: unlink d2/d10/f23 0 2024-04-24T08:24:46.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/372: read - d0/d6/db/d27/d64/f76 zero size 2024-04-24T08:24:46.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/64: sync 2024-04-24T08:24:46.944 INFO:tasks.workunit.client.0.smithi090.stdout:3/65: sync 2024-04-24T08:24:46.945 INFO:tasks.workunit.client.0.smithi090.stdout:5/166: rename d8/da/d13 to d8/da/d3a 0 2024-04-24T08:24:46.946 INFO:tasks.workunit.client.0.smithi090.stdout:3/66: write d2/f9 [515174,114315] 0 2024-04-24T08:24:46.948 INFO:tasks.workunit.client.0.smithi090.stdout:6/139: symlink d8/dc/l2b 0 2024-04-24T08:24:46.950 INFO:tasks.workunit.client.1.smithi149.stdout:9/478: dwrite d1/da/dd/f32 [0,4194304] 0 2024-04-24T08:24:46.950 INFO:tasks.workunit.client.0.smithi090.stdout:2/102: creat d0/d14/f1d x:0 0 0 2024-04-24T08:24:46.951 INFO:tasks.workunit.client.0.smithi090.stdout:0/171: mknod d1a/d2a/c3a 0 2024-04-24T08:24:46.954 INFO:tasks.workunit.client.0.smithi090.stdout:2/103: chown d0/d7/f11 1520 1 2024-04-24T08:24:46.955 INFO:tasks.workunit.client.0.smithi090.stdout:0/172: write f13 [3988757,83806] 0 2024-04-24T08:24:46.956 INFO:tasks.workunit.client.1.smithi149.stdout:8/384: dwrite d1/d1f/d25/f78 [0,4194304] 0 2024-04-24T08:24:46.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/65: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:24:46.959 INFO:tasks.workunit.client.1.smithi149.stdout:8/385: chown d1/d1f/d22/f30 14626 1 2024-04-24T08:24:46.959 INFO:tasks.workunit.client.0.smithi090.stdout:0/173: write d1a/f20 [1572114,77031] 0 2024-04-24T08:24:46.961 INFO:tasks.workunit.client.0.smithi090.stdout:9/66: write d0/dc/fd [698490,4487] 0 2024-04-24T08:24:46.962 INFO:tasks.workunit.client.0.smithi090.stdout:5/167: symlink d8/da/d3a/l3b 0 2024-04-24T08:24:46.963 INFO:tasks.workunit.client.0.smithi090.stdout:9/67: chown d0 1328465 1 2024-04-24T08:24:46.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/67: creat d2/df/f12 x:0 0 0 2024-04-24T08:24:46.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/373: truncate d0/f55 1455502 0 2024-04-24T08:24:46.968 INFO:tasks.workunit.client.1.smithi149.stdout:1/551: dread d0/d1b/d4b/f58 [0,4194304] 0 2024-04-24T08:24:46.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/104: symlink d0/d7/l1e 0 2024-04-24T08:24:46.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/479: mknod d1/de/ca3 0 2024-04-24T08:24:46.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/480: readlink d1/da/l9a 0 2024-04-24T08:24:46.973 INFO:tasks.workunit.client.0.smithi090.stdout:2/105: chown d0/d14 209540 1 2024-04-24T08:24:46.973 INFO:tasks.workunit.client.1.smithi149.stdout:0/398: link d2/d9/df/d56/f5f d2/d9/df/d40/d4a/f79 0 2024-04-24T08:24:46.976 INFO:tasks.workunit.client.0.smithi090.stdout:2/106: stat d0/d7/f1c 0 2024-04-24T08:24:46.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/68: symlink d2/l13 0 2024-04-24T08:24:46.980 INFO:tasks.workunit.client.1.smithi149.stdout:7/448: creat d9/d43/d4f/f92 x:0 0 0 2024-04-24T08:24:46.980 INFO:tasks.workunit.client.0.smithi090.stdout:6/140: dwrite d8/d1d/f24 [0,4194304] 0 2024-04-24T08:24:46.981 INFO:tasks.workunit.client.1.smithi149.stdout:8/386: unlink d1/d1f/d31/f35 0 2024-04-24T08:24:46.983 INFO:tasks.workunit.client.1.smithi149.stdout:7/449: write d9/dc/f18 [2949025,45741] 0 2024-04-24T08:24:46.985 INFO:tasks.workunit.client.1.smithi149.stdout:8/387: truncate d1/de/d18/f6a 206697 0 2024-04-24T08:24:46.988 INFO:tasks.workunit.client.0.smithi090.stdout:3/69: creat d2/f14 x:0 0 0 2024-04-24T08:24:46.989 INFO:tasks.workunit.client.0.smithi090.stdout:0/174: dwrite d1a/f20 [0,4194304] 0 2024-04-24T08:24:46.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/175: fdatasync f13 0 2024-04-24T08:24:46.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/176: fsync d1a/f2c 0 2024-04-24T08:24:46.994 INFO:tasks.workunit.client.1.smithi149.stdout:2/374: rmdir d0/d6/db/d39 39 2024-04-24T08:24:46.995 INFO:tasks.workunit.client.0.smithi090.stdout:4/65: write d3/fd [2916076,103618] 0 2024-04-24T08:24:46.995 INFO:tasks.workunit.client.0.smithi090.stdout:9/68: link d0/c6 d0/dc/cf 0 2024-04-24T08:24:46.998 INFO:tasks.workunit.client.1.smithi149.stdout:5/519: symlink db/d12/d1e/d85/la9 0 2024-04-24T08:24:47.004 INFO:tasks.workunit.client.1.smithi149.stdout:7/450: write d9/dc/d11/f2b [2321674,64227] 0 2024-04-24T08:24:47.006 INFO:tasks.workunit.client.1.smithi149.stdout:4/434: dread d2/d10/f31 [0,4194304] 0 2024-04-24T08:24:47.007 INFO:tasks.workunit.client.0.smithi090.stdout:9/69: stat d0/f2 0 2024-04-24T08:24:47.007 INFO:tasks.workunit.client.1.smithi149.stdout:8/388: creat d1/d1f/d31/f84 x:0 0 0 2024-04-24T08:24:47.010 INFO:tasks.workunit.client.1.smithi149.stdout:1/552: mkdir d0/db2 0 2024-04-24T08:24:47.010 INFO:tasks.workunit.client.1.smithi149.stdout:4/435: write d2/d18/d1e/d3b/f7c [255850,68656] 0 2024-04-24T08:24:47.014 INFO:tasks.workunit.client.0.smithi090.stdout:2/107: creat d0/d14/f1f x:0 0 0 2024-04-24T08:24:47.017 INFO:tasks.workunit.client.1.smithi149.stdout:6/336: truncate d0/f3 1558261 0 2024-04-24T08:24:47.019 INFO:tasks.workunit.client.0.smithi090.stdout:4/66: dwrite d3/f9 [0,4194304] 0 2024-04-24T08:24:47.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/168: dwrite d8/da/f35 [0,4194304] 0 2024-04-24T08:24:47.019 INFO:tasks.workunit.client.0.smithi090.stdout:2/108: chown d0/d14/d17/f18 53141 1 2024-04-24T08:24:47.020 INFO:tasks.workunit.client.1.smithi149.stdout:5/520: write db/d12/d60/f67 [8664604,102807] 0 2024-04-24T08:24:47.020 INFO:tasks.workunit.client.1.smithi149.stdout:7/451: rmdir d9/dc/d2e 39 2024-04-24T08:24:47.021 INFO:tasks.workunit.client.0.smithi090.stdout:9/70: rename d0/f3 to d0/f10 0 2024-04-24T08:24:47.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/169: fsync d8/d25/f26 0 2024-04-24T08:24:47.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/177: creat d1a/f3b x:0 0 0 2024-04-24T08:24:47.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/70: dwrite d2/fa [4194304,4194304] 0 2024-04-24T08:24:47.025 INFO:tasks.workunit.client.1.smithi149.stdout:8/389: dread d1/de/d18/f45 [0,4194304] 0 2024-04-24T08:24:47.026 INFO:tasks.workunit.client.1.smithi149.stdout:2/375: truncate d0/d6/db/d39/d3d/d44/f61 356843 0 2024-04-24T08:24:47.029 INFO:tasks.workunit.client.1.smithi149.stdout:9/481: creat d1/d2/d15/d1c/fa4 x:0 0 0 2024-04-24T08:24:47.030 INFO:tasks.workunit.client.1.smithi149.stdout:5/521: mkdir db/d23/daa 0 2024-04-24T08:24:47.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/337: mknod d0/d1a/c6d 0 2024-04-24T08:24:47.035 INFO:tasks.workunit.client.0.smithi090.stdout:2/109: unlink d0/d7/l1a 0 2024-04-24T08:24:47.038 INFO:tasks.workunit.client.1.smithi149.stdout:1/553: mknod d0/d1b/d4b/da1/cb3 0 2024-04-24T08:24:47.040 INFO:tasks.workunit.client.1.smithi149.stdout:8/390: creat d1/de/d18/d32/d63/f85 x:0 0 0 2024-04-24T08:24:47.042 INFO:tasks.workunit.client.0.smithi090.stdout:0/178: mkdir d1a/d2a/d3c 0 2024-04-24T08:24:47.043 INFO:tasks.workunit.client.1.smithi149.stdout:8/391: chown d1/d1f/d22/f5f 5846 1 2024-04-24T08:24:47.043 INFO:tasks.workunit.client.1.smithi149.stdout:1/554: truncate d0/d5/db/d63/f86 777860 0 2024-04-24T08:24:47.044 INFO:tasks.workunit.client.1.smithi149.stdout:2/376: rmdir d0/d6/db/d39/d3d 39 2024-04-24T08:24:47.046 INFO:tasks.workunit.client.0.smithi090.stdout:9/71: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:47.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/170: dwrite d8/da/d3a/d2d/d37/f39 [0,4194304] 0 2024-04-24T08:24:47.048 INFO:tasks.workunit.client.0.smithi090.stdout:5/171: write d8/f9 [2719968,25924] 0 2024-04-24T08:24:47.049 INFO:tasks.workunit.client.0.smithi090.stdout:2/110: chown d0/lb 125 1 2024-04-24T08:24:47.049 INFO:tasks.workunit.client.1.smithi149.stdout:9/482: symlink d1/d2/la5 0 2024-04-24T08:24:47.052 INFO:tasks.workunit.client.0.smithi090.stdout:2/111: chown d0/d7/f1c 3703297 1 2024-04-24T08:24:47.059 INFO:tasks.workunit.client.0.smithi090.stdout:0/179: unlink d1a/d2a/l2b 0 2024-04-24T08:24:47.060 INFO:tasks.workunit.client.1.smithi149.stdout:4/436: write d2/d24/d2e/f51 [2443990,9103] 0 2024-04-24T08:24:47.061 INFO:tasks.workunit.client.0.smithi090.stdout:5/172: dread d8/da/f20 [0,4194304] 0 2024-04-24T08:24:47.061 INFO:tasks.workunit.client.0.smithi090.stdout:0/180: write f15 [581388,7278] 0 2024-04-24T08:24:47.070 INFO:tasks.workunit.client.0.smithi090.stdout:4/67: truncate d3/fd 495714 0 2024-04-24T08:24:47.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/71: truncate d2/f9 3006601 0 2024-04-24T08:24:47.072 INFO:tasks.workunit.client.0.smithi090.stdout:4/68: chown d3/f9 1 1 2024-04-24T08:24:47.076 INFO:tasks.workunit.client.0.smithi090.stdout:9/72: dread d0/f5 [0,4194304] 0 2024-04-24T08:24:47.077 INFO:tasks.workunit.client.1.smithi149.stdout:1/555: write d0/d5/f8 [7280224,42240] 0 2024-04-24T08:24:47.078 INFO:tasks.workunit.client.0.smithi090.stdout:5/173: chown f4 112207 1 2024-04-24T08:24:47.079 INFO:tasks.workunit.client.1.smithi149.stdout:9/483: creat d1/da/dd/d79/fa6 x:0 0 0 2024-04-24T08:24:47.081 INFO:tasks.workunit.client.0.smithi090.stdout:3/72: rmdir d2/df 39 2024-04-24T08:24:47.082 INFO:tasks.workunit.client.0.smithi090.stdout:3/73: write d2/f14 [129553,33142] 0 2024-04-24T08:24:47.084 INFO:tasks.workunit.client.0.smithi090.stdout:8/76: sync 2024-04-24T08:24:47.084 INFO:tasks.workunit.client.0.smithi090.stdout:1/95: sync 2024-04-24T08:24:47.084 INFO:tasks.workunit.client.0.smithi090.stdout:7/161: sync 2024-04-24T08:24:47.085 INFO:tasks.workunit.client.0.smithi090.stdout:7/162: write db/fc [911094,65148] 0 2024-04-24T08:24:47.086 INFO:tasks.workunit.client.1.smithi149.stdout:4/437: chown d2/l3 1775797 1 2024-04-24T08:24:47.086 INFO:tasks.workunit.client.1.smithi149.stdout:5/522: creat db/d23/daa/fab x:0 0 0 2024-04-24T08:24:47.086 INFO:tasks.workunit.client.0.smithi090.stdout:9/73: mknod d0/dc/c11 0 2024-04-24T08:24:47.087 INFO:tasks.workunit.client.1.smithi149.stdout:2/377: dread - d0/d6/db/d39/d3d/f51 zero size 2024-04-24T08:24:47.088 INFO:tasks.workunit.client.1.smithi149.stdout:1/556: fsync d0/d5/db/d18/f6f 0 2024-04-24T08:24:47.090 INFO:tasks.workunit.client.0.smithi090.stdout:0/181: unlink d1a/c1c 0 2024-04-24T08:24:47.091 INFO:tasks.workunit.client.1.smithi149.stdout:7/452: dwrite d9/d43/d4c/f71 [0,4194304] 0 2024-04-24T08:24:47.093 INFO:tasks.workunit.client.1.smithi149.stdout:1/557: dread - d0/d5/db/d82/f9b zero size 2024-04-24T08:24:47.099 INFO:tasks.workunit.client.1.smithi149.stdout:1/558: truncate d0/d5/db/d82/f9b 122528 0 2024-04-24T08:24:47.100 INFO:tasks.workunit.client.1.smithi149.stdout:6/338: rename d0/d1a/l34 to d0/d1a/d28/d31/d4c/l6e 0 2024-04-24T08:24:47.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/112: write d0/ff [418616,125492] 0 2024-04-24T08:24:47.106 INFO:tasks.workunit.client.1.smithi149.stdout:9/484: mknod d1/da/d13/d74/ca7 0 2024-04-24T08:24:47.107 INFO:tasks.workunit.client.0.smithi090.stdout:9/74: mknod d0/dc/c12 0 2024-04-24T08:24:47.109 INFO:tasks.workunit.client.0.smithi090.stdout:4/69: dwrite d3/fd [0,4194304] 0 2024-04-24T08:24:47.111 INFO:tasks.workunit.client.1.smithi149.stdout:5/523: fdatasync db/d12/d1e/f64 0 2024-04-24T08:24:47.116 INFO:tasks.workunit.client.0.smithi090.stdout:1/96: getdents d3/d1d 0 2024-04-24T08:24:47.119 INFO:tasks.workunit.client.0.smithi090.stdout:5/174: dwrite f4 [4194304,4194304] 0 2024-04-24T08:24:47.123 INFO:tasks.workunit.client.1.smithi149.stdout:9/485: creat d1/de/fa8 x:0 0 0 2024-04-24T08:24:47.123 INFO:tasks.workunit.client.1.smithi149.stdout:1/559: mknod d0/db2/cb4 0 2024-04-24T08:24:47.124 INFO:tasks.workunit.client.0.smithi090.stdout:8/77: dwrite d2/fd [0,4194304] 0 2024-04-24T08:24:47.127 INFO:tasks.workunit.client.0.smithi090.stdout:7/163: mkdir db/d17/d29/d38/d3e 0 2024-04-24T08:24:47.129 INFO:tasks.workunit.client.1.smithi149.stdout:5/524: creat db/d12/d60/d8f/fac x:0 0 0 2024-04-24T08:24:47.130 INFO:tasks.workunit.client.0.smithi090.stdout:8/78: write d2/da/d13/d15/f16 [1480800,63647] 0 2024-04-24T08:24:47.134 INFO:tasks.workunit.client.0.smithi090.stdout:9/75: write d0/f10 [3107540,98444] 0 2024-04-24T08:24:47.136 INFO:tasks.workunit.client.1.smithi149.stdout:3/496: sync 2024-04-24T08:24:47.136 INFO:tasks.workunit.client.1.smithi149.stdout:2/378: rename d0/c56 to d0/d6/db/d27/c7b 0 2024-04-24T08:24:47.137 INFO:tasks.workunit.client.0.smithi090.stdout:9/76: chown d0/dc/le 0 1 2024-04-24T08:24:47.139 INFO:tasks.workunit.client.0.smithi090.stdout:1/97: dwrite d3/f4 [4194304,4194304] 0 2024-04-24T08:24:47.139 INFO:tasks.workunit.client.0.smithi090.stdout:5/175: dread d8/f31 [4194304,4194304] 0 2024-04-24T08:24:47.139 INFO:tasks.workunit.client.0.smithi090.stdout:9/77: dread - d0/fa zero size 2024-04-24T08:24:47.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/98: chown d3/d5/f1a 743402567 1 2024-04-24T08:24:47.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/176: truncate d8/da/d14/f28 418407 0 2024-04-24T08:24:47.142 INFO:tasks.workunit.client.1.smithi149.stdout:9/486: mknod d1/de/ca9 0 2024-04-24T08:24:47.144 INFO:tasks.workunit.client.0.smithi090.stdout:5/177: write d8/da/d14/f27 [580093,54346] 0 2024-04-24T08:24:47.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/178: stat d8 0 2024-04-24T08:24:47.147 INFO:tasks.workunit.client.0.smithi090.stdout:4/70: mknod d3/ce 0 2024-04-24T08:24:47.149 INFO:tasks.workunit.client.1.smithi149.stdout:7/453: dwrite d9/dc/f29 [0,4194304] 0 2024-04-24T08:24:47.155 INFO:tasks.workunit.client.1.smithi149.stdout:7/454: fsync d9/d43/d4c/f52 0 2024-04-24T08:24:47.160 INFO:tasks.workunit.client.0.smithi090.stdout:5/179: dread d8/da/d3a/f23 [0,4194304] 0 2024-04-24T08:24:47.161 INFO:tasks.workunit.client.1.smithi149.stdout:6/339: dread - d0/d1a/d28/d59/f68 zero size 2024-04-24T08:24:47.162 INFO:tasks.workunit.client.0.smithi090.stdout:4/71: dread d3/f9 [0,4194304] 0 2024-04-24T08:24:47.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/72: fdatasync d3/f9 0 2024-04-24T08:24:47.166 INFO:tasks.workunit.client.0.smithi090.stdout:7/164: unlink db/d17/d1b/f23 0 2024-04-24T08:24:47.168 INFO:tasks.workunit.client.0.smithi090.stdout:4/73: chown d3/l8 224 1 2024-04-24T08:24:47.168 INFO:tasks.workunit.client.0.smithi090.stdout:2/113: symlink d0/d14/l20 0 2024-04-24T08:24:47.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/525: truncate db/d12/f72 58257 0 2024-04-24T08:24:47.170 INFO:tasks.workunit.client.1.smithi149.stdout:1/560: dwrite d0/d1b/d5d/f9a [0,4194304] 0 2024-04-24T08:24:47.171 INFO:tasks.workunit.client.0.smithi090.stdout:8/79: mknod d2/da/d13/d15/c1f 0 2024-04-24T08:24:47.176 INFO:tasks.workunit.client.1.smithi149.stdout:3/497: rename d0/d5/d53/d39/l3d to d0/d5/da/d19/d1b/d22/d38/lae 0 2024-04-24T08:24:47.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/99: symlink d3/d5/da/l22 0 2024-04-24T08:24:47.180 INFO:tasks.workunit.client.0.smithi090.stdout:6/141: sync 2024-04-24T08:24:47.180 INFO:tasks.workunit.client.0.smithi090.stdout:5/180: dwrite f7 [8388608,4194304] 0 2024-04-24T08:24:47.184 INFO:tasks.workunit.client.0.smithi090.stdout:1/100: write d3/d5/f16 [1044260,53281] 0 2024-04-24T08:24:47.184 INFO:tasks.workunit.client.0.smithi090.stdout:6/142: write d8/dc/f10 [735778,69578] 0 2024-04-24T08:24:47.184 INFO:tasks.workunit.client.1.smithi149.stdout:6/340: mkdir d0/d1a/d28/d59/d6f 0 2024-04-24T08:24:47.191 INFO:tasks.workunit.client.0.smithi090.stdout:4/74: dwrite d3/fd [0,4194304] 0 2024-04-24T08:24:47.191 INFO:tasks.workunit.client.0.smithi090.stdout:1/101: truncate d3/d12/f18 8803085 0 2024-04-24T08:24:47.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/182: creat d1a/f3d x:0 0 0 2024-04-24T08:24:47.196 INFO:tasks.workunit.client.1.smithi149.stdout:1/561: unlink d0/d5/d4f/d60/f8f 0 2024-04-24T08:24:47.200 INFO:tasks.workunit.client.1.smithi149.stdout:7/455: rename d9/d43/d4c/d67/d69/f77 to d9/dc/d11/d83/f93 0 2024-04-24T08:24:47.206 INFO:tasks.workunit.client.0.smithi090.stdout:9/78: write d0/f2 [1249936,95868] 0 2024-04-24T08:24:47.207 INFO:tasks.workunit.client.0.smithi090.stdout:9/79: write d0/fa [428109,21771] 0 2024-04-24T08:24:47.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/487: truncate d1/de/ff 12977 0 2024-04-24T08:24:47.210 INFO:tasks.workunit.client.1.smithi149.stdout:3/498: rename d0/d5/d53/d6d/f9a to d0/d5/da/d19/faf 0 2024-04-24T08:24:47.214 INFO:tasks.workunit.client.0.smithi090.stdout:2/114: write d0/d14/d17/f18 [3434824,58958] 0 2024-04-24T08:24:47.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/115: read - d0/d14/f1f zero size 2024-04-24T08:24:47.215 INFO:tasks.workunit.client.1.smithi149.stdout:6/341: dwrite d0/fe [0,4194304] 0 2024-04-24T08:24:47.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/80: symlink d2/l20 0 2024-04-24T08:24:47.217 INFO:tasks.workunit.client.0.smithi090.stdout:2/116: truncate d0/d7/f1c 374970 0 2024-04-24T08:24:47.218 INFO:tasks.workunit.client.1.smithi149.stdout:7/456: readlink d9/dc/d2e/d3c/l63 0 2024-04-24T08:24:47.223 INFO:tasks.workunit.client.0.smithi090.stdout:9/80: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:47.228 INFO:tasks.workunit.client.0.smithi090.stdout:5/181: mknod d8/da/d3a/d2d/c3c 0 2024-04-24T08:24:47.230 INFO:tasks.workunit.client.1.smithi149.stdout:5/526: dwrite db/d12/d22/f31 [0,4194304] 0 2024-04-24T08:24:47.234 INFO:tasks.workunit.client.0.smithi090.stdout:6/143: creat d8/d1d/d29/f2c x:0 0 0 2024-04-24T08:24:47.237 INFO:tasks.workunit.client.0.smithi090.stdout:6/144: chown d8/f15 5 1 2024-04-24T08:24:47.237 INFO:tasks.workunit.client.0.smithi090.stdout:1/102: symlink d3/l23 0 2024-04-24T08:24:47.239 INFO:tasks.workunit.client.1.smithi149.stdout:3/499: creat d0/d5/d1a/d70/da3/fb0 x:0 0 0 2024-04-24T08:24:47.239 INFO:tasks.workunit.client.0.smithi090.stdout:6/145: dread - d8/f1b zero size 2024-04-24T08:24:47.241 INFO:tasks.workunit.client.1.smithi149.stdout:5/527: mknod db/d12/d22/d45/d5d/cad 0 2024-04-24T08:24:47.242 INFO:tasks.workunit.client.0.smithi090.stdout:2/117: dwrite d0/d14/f1f [0,4194304] 0 2024-04-24T08:24:47.243 INFO:tasks.workunit.client.0.smithi090.stdout:2/118: fsync d0/d14/f1d 0 2024-04-24T08:24:47.247 INFO:tasks.workunit.client.0.smithi090.stdout:7/165: creat db/d17/d29/d38/d3e/f3f x:0 0 0 2024-04-24T08:24:47.249 INFO:tasks.workunit.client.0.smithi090.stdout:6/146: chown d8/d1d/d1f/l25 1169 1 2024-04-24T08:24:47.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/500: mkdir d0/d26/d3c/db1 0 2024-04-24T08:24:47.253 INFO:tasks.workunit.client.1.smithi149.stdout:3/501: chown d0/d5/d53/d4c/d5f 3364 1 2024-04-24T08:24:47.254 INFO:tasks.workunit.client.0.smithi090.stdout:2/119: truncate d0/d14/f1d 312389 0 2024-04-24T08:24:47.254 INFO:tasks.workunit.client.0.smithi090.stdout:9/81: dwrite d0/f10 [0,4194304] 0 2024-04-24T08:24:47.256 INFO:tasks.workunit.client.1.smithi149.stdout:5/528: symlink db/d12/d22/d45/d5d/lae 0 2024-04-24T08:24:47.258 INFO:tasks.workunit.client.1.smithi149.stdout:9/488: link d1/da/dd/d55/c7d d1/de/d9d/caa 0 2024-04-24T08:24:47.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/182: dwrite d8/da/d18/f1f [0,4194304] 0 2024-04-24T08:24:47.263 INFO:tasks.workunit.client.1.smithi149.stdout:1/562: rename d0/d5/d4f/d60/d19 to d0/d1b/db5 0 2024-04-24T08:24:47.265 INFO:tasks.workunit.client.0.smithi090.stdout:5/183: chown d8/da/fc 12264515 1 2024-04-24T08:24:47.265 INFO:tasks.workunit.client.1.smithi149.stdout:1/563: write d0/f84 [1848415,51598] 0 2024-04-24T08:24:47.266 INFO:tasks.workunit.client.1.smithi149.stdout:5/529: creat db/d12/d22/d6c/d83/faf x:0 0 0 2024-04-24T08:24:47.266 INFO:tasks.workunit.client.0.smithi090.stdout:1/103: dwrite d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:47.269 INFO:tasks.workunit.client.0.smithi090.stdout:8/81: creat d2/d19/f21 x:0 0 0 2024-04-24T08:24:47.269 INFO:tasks.workunit.client.1.smithi149.stdout:9/489: unlink d1/da/d13/d67/f98 0 2024-04-24T08:24:47.271 INFO:tasks.workunit.client.0.smithi090.stdout:7/166: creat db/d17/d29/d32/f40 x:0 0 0 2024-04-24T08:24:47.271 INFO:tasks.workunit.client.0.smithi090.stdout:6/147: write d8/fa [3358987,88244] 0 2024-04-24T08:24:47.271 INFO:tasks.workunit.client.0.smithi090.stdout:1/104: chown d3/d5/f1a 148583019 1 2024-04-24T08:24:47.271 INFO:tasks.workunit.client.0.smithi090.stdout:8/82: readlink d2/da/lf 0 2024-04-24T08:24:47.272 INFO:tasks.workunit.client.1.smithi149.stdout:3/502: mknod d0/cb2 0 2024-04-24T08:24:47.273 INFO:tasks.workunit.client.0.smithi090.stdout:7/167: chown db/f12 533 1 2024-04-24T08:24:47.273 INFO:tasks.workunit.client.0.smithi090.stdout:6/148: write d8/f15 [341915,88272] 0 2024-04-24T08:24:47.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/82: creat d0/dc/f13 x:0 0 0 2024-04-24T08:24:47.274 INFO:tasks.workunit.client.1.smithi149.stdout:9/490: chown d1/de/d12/l6b 21 1 2024-04-24T08:24:47.275 INFO:tasks.workunit.client.0.smithi090.stdout:8/83: chown d2/da/d13/d15/f16 89 1 2024-04-24T08:24:47.276 INFO:tasks.workunit.client.0.smithi090.stdout:2/120: creat d0/f21 x:0 0 0 2024-04-24T08:24:47.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/83: write d0/f2 [542732,42447] 0 2024-04-24T08:24:47.277 INFO:tasks.workunit.client.0.smithi090.stdout:5/184: unlink c5 0 2024-04-24T08:24:47.278 INFO:tasks.workunit.client.0.smithi090.stdout:9/84: chown d0/dc/fd 6452854 1 2024-04-24T08:24:47.282 INFO:tasks.workunit.client.1.smithi149.stdout:6/342: rename d0/d1a/l30 to d0/d1a/d28/d31/d4c/d4e/d62/d66/l70 0 2024-04-24T08:24:47.289 INFO:tasks.workunit.client.0.smithi090.stdout:6/149: readlink d8/dc/l26 0 2024-04-24T08:24:47.290 INFO:tasks.workunit.client.1.smithi149.stdout:1/564: dwrite d0/d5/db/f92 [0,4194304] 0 2024-04-24T08:24:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:5/530: creat db/d12/fb0 x:0 0 0 2024-04-24T08:24:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:1/565: stat d0/d5/db/d63/d98/c68 0 2024-04-24T08:24:47.302 INFO:tasks.workunit.client.1.smithi149.stdout:1/566: dread - d0/d5/db/d18/f8a zero size 2024-04-24T08:24:47.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/84: rmdir d2 39 2024-04-24T08:24:47.303 INFO:tasks.workunit.client.1.smithi149.stdout:7/457: rename d9/dc/de/d44/c50 to d9/dc/d11/d26/c94 0 2024-04-24T08:24:47.307 INFO:tasks.workunit.client.0.smithi090.stdout:5/185: chown f6 91 1 2024-04-24T08:24:47.310 INFO:tasks.workunit.client.0.smithi090.stdout:0/183: creat d1a/d2a/d35/f3e x:0 0 0 2024-04-24T08:24:47.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/105: symlink d3/d1d/l24 0 2024-04-24T08:24:47.320 INFO:tasks.workunit.client.0.smithi090.stdout:7/168: creat db/d17/d39/f41 x:0 0 0 2024-04-24T08:24:47.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/85: write d2/f3 [3303088,76753] 0 2024-04-24T08:24:47.322 INFO:tasks.workunit.client.1.smithi149.stdout:5/531: dwrite db/d12/d60/d8f/fac [0,4194304] 0 2024-04-24T08:24:47.323 INFO:tasks.workunit.client.0.smithi090.stdout:2/121: link d0/d7/fd d0/f22 0 2024-04-24T08:24:47.325 INFO:tasks.workunit.client.1.smithi149.stdout:0/399: sync 2024-04-24T08:24:47.325 INFO:tasks.workunit.client.1.smithi149.stdout:5/532: chown db/d35/f4c 0 1 2024-04-24T08:24:47.326 INFO:tasks.workunit.client.0.smithi090.stdout:5/186: creat d8/d25/f3d x:0 0 0 2024-04-24T08:24:47.327 INFO:tasks.workunit.client.0.smithi090.stdout:5/187: write d8/da/f1d [499557,26355] 0 2024-04-24T08:24:47.328 INFO:tasks.workunit.client.0.smithi090.stdout:0/184: truncate d1a/f2c 924875 0 2024-04-24T08:24:47.329 INFO:tasks.workunit.client.0.smithi090.stdout:1/106: mkdir d3/d25 0 2024-04-24T08:24:47.329 INFO:tasks.workunit.client.1.smithi149.stdout:0/400: chown d2/d9/df/d40/d4a/f64 187581052 1 2024-04-24T08:24:47.330 INFO:tasks.workunit.client.1.smithi149.stdout:9/491: dread d1/da/dd/d79/f56 [0,4194304] 0 2024-04-24T08:24:47.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/150: truncate d8/dc/f11 29083 0 2024-04-24T08:24:47.331 INFO:tasks.workunit.client.0.smithi090.stdout:0/185: write d1a/d2a/f27 [3980534,1634] 0 2024-04-24T08:24:47.333 INFO:tasks.workunit.client.0.smithi090.stdout:6/151: chown d8/dc/f1a 57629 1 2024-04-24T08:24:47.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/503: rename d0/d5/da/d5a/l85 to d0/d26/d3c/lb3 0 2024-04-24T08:24:47.335 INFO:tasks.workunit.client.1.smithi149.stdout:0/401: dread d2/d3c/f61 [0,4194304] 0 2024-04-24T08:24:47.335 INFO:tasks.workunit.client.0.smithi090.stdout:6/152: write d8/d1d/f1e [1235724,37054] 0 2024-04-24T08:24:47.338 INFO:tasks.workunit.client.1.smithi149.stdout:7/458: readlink d9/dc/d11/l6a 0 2024-04-24T08:24:47.338 INFO:tasks.workunit.client.1.smithi149.stdout:5/533: dread db/d12/f69 [0,4194304] 0 2024-04-24T08:24:47.341 INFO:tasks.workunit.client.1.smithi149.stdout:7/459: read - d9/dc/d11/d83/f93 zero size 2024-04-24T08:24:47.341 INFO:tasks.workunit.client.0.smithi090.stdout:8/86: dwrite d2/f10 [0,4194304] 0 2024-04-24T08:24:47.344 INFO:tasks.workunit.client.0.smithi090.stdout:7/169: link db/d17/d29/d32/f36 db/d17/d29/f42 0 2024-04-24T08:24:47.348 INFO:tasks.workunit.client.0.smithi090.stdout:1/107: dwrite d3/d12/f1b [0,4194304] 0 2024-04-24T08:24:47.348 INFO:tasks.workunit.client.0.smithi090.stdout:9/85: getdents d0/dc 0 2024-04-24T08:24:47.349 INFO:tasks.workunit.client.1.smithi149.stdout:7/460: mkdir d9/d43/d95 0 2024-04-24T08:24:47.350 INFO:tasks.workunit.client.0.smithi090.stdout:3/74: sync 2024-04-24T08:24:47.350 INFO:tasks.workunit.client.0.smithi090.stdout:5/188: mknod d8/da/d14/d2f/c3e 0 2024-04-24T08:24:47.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/87: creat d2/f22 x:0 0 0 2024-04-24T08:24:47.352 INFO:tasks.workunit.client.1.smithi149.stdout:6/343: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 x:0 0 0 2024-04-24T08:24:47.352 INFO:tasks.workunit.client.0.smithi090.stdout:8/88: stat d2/cb 0 2024-04-24T08:24:47.353 INFO:tasks.workunit.client.0.smithi090.stdout:6/153: dread d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:47.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/189: chown d8/f1c 16 1 2024-04-24T08:24:47.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/190: chown d8/da/d3a 79 1 2024-04-24T08:24:47.367 INFO:tasks.workunit.client.0.smithi090.stdout:1/108: dread d3/d5/f17 [0,4194304] 0 2024-04-24T08:24:47.372 INFO:tasks.workunit.client.1.smithi149.stdout:6/344: creat d0/d1a/d28/d4a/f72 x:0 0 0 2024-04-24T08:24:47.374 INFO:tasks.workunit.client.0.smithi090.stdout:9/86: rename d0/dc/c12 to d0/c14 0 2024-04-24T08:24:47.376 INFO:tasks.workunit.client.1.smithi149.stdout:5/534: mknod db/d12/d1e/d85/d92/da6/cb1 0 2024-04-24T08:24:47.378 INFO:tasks.workunit.client.0.smithi090.stdout:7/170: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:47.380 INFO:tasks.workunit.client.1.smithi149.stdout:1/567: getdents d0/d1b/db5 0 2024-04-24T08:24:47.381 INFO:tasks.workunit.client.0.smithi090.stdout:0/186: dwrite f18 [0,4194304] 0 2024-04-24T08:24:47.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/122: getdents d0/d7 0 2024-04-24T08:24:47.390 INFO:tasks.workunit.client.1.smithi149.stdout:0/402: dread d2/d9/df/d40/d4a/f64 [0,4194304] 0 2024-04-24T08:24:47.391 INFO:tasks.workunit.client.0.smithi090.stdout:6/154: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:47.392 INFO:tasks.workunit.client.0.smithi090.stdout:9/87: read d0/fa [84640,70687] 0 2024-04-24T08:24:47.392 INFO:tasks.workunit.client.0.smithi090.stdout:3/75: mkdir d2/d15 0 2024-04-24T08:24:47.394 INFO:tasks.workunit.client.1.smithi149.stdout:3/504: getdents d0/d5/da/d19/d1b/d22/d38 0 2024-04-24T08:24:47.395 INFO:tasks.workunit.client.0.smithi090.stdout:3/76: stat d2 0 2024-04-24T08:24:47.399 INFO:tasks.workunit.client.1.smithi149.stdout:3/505: chown d0/d5/da/d19/f48 533285 1 2024-04-24T08:24:47.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/345: creat d0/f73 x:0 0 0 2024-04-24T08:24:47.400 INFO:tasks.workunit.client.0.smithi090.stdout:7/171: dread - db/d17/d29/d38/d3e/f3f zero size 2024-04-24T08:24:47.402 INFO:tasks.workunit.client.1.smithi149.stdout:6/346: stat d0/f16 0 2024-04-24T08:24:47.404 INFO:tasks.workunit.client.0.smithi090.stdout:8/89: dwrite d2/f10 [0,4194304] 0 2024-04-24T08:24:47.405 INFO:tasks.workunit.client.1.smithi149.stdout:6/347: write d0/fe [2884002,41273] 0 2024-04-24T08:24:47.406 INFO:tasks.workunit.client.1.smithi149.stdout:0/403: mkdir d2/d9/df/d40/d4a/d4e/d6f/d7a 0 2024-04-24T08:24:47.406 INFO:tasks.workunit.client.1.smithi149.stdout:3/506: dread - d0/d5/da/d19/d1b/d22/d38/f7e zero size 2024-04-24T08:24:47.407 INFO:tasks.workunit.client.1.smithi149.stdout:5/535: dwrite db/d12/d60/d8f/fac [0,4194304] 0 2024-04-24T08:24:47.408 INFO:tasks.workunit.client.0.smithi090.stdout:7/172: dread - db/d17/f2d zero size 2024-04-24T08:24:47.409 INFO:tasks.workunit.client.0.smithi090.stdout:0/187: fsync f13 0 2024-04-24T08:24:47.411 INFO:tasks.workunit.client.0.smithi090.stdout:2/123: symlink d0/d7/l23 0 2024-04-24T08:24:47.414 INFO:tasks.workunit.client.0.smithi090.stdout:6/155: symlink d8/dc/d19/l2d 0 2024-04-24T08:24:47.414 INFO:tasks.workunit.client.1.smithi149.stdout:9/492: truncate d1/da/dd/d79/f56 1732229 0 2024-04-24T08:24:47.416 INFO:tasks.workunit.client.0.smithi090.stdout:5/191: dwrite d8/da/fc [4194304,4194304] 0 2024-04-24T08:24:47.421 INFO:tasks.workunit.client.1.smithi149.stdout:6/348: creat d0/d1a/d28/d59/f74 x:0 0 0 2024-04-24T08:24:47.422 INFO:tasks.workunit.client.1.smithi149.stdout:5/536: read db/d12/d2f/d95/f9e [465995,40595] 0 2024-04-24T08:24:47.426 INFO:tasks.workunit.client.0.smithi090.stdout:6/156: read - d8/dc/f1a zero size 2024-04-24T08:24:47.426 INFO:tasks.workunit.client.0.smithi090.stdout:5/192: chown d8/da/d3a/d36 479 1 2024-04-24T08:24:47.431 INFO:tasks.workunit.client.0.smithi090.stdout:7/173: creat db/d17/d39/f43 x:0 0 0 2024-04-24T08:24:47.434 INFO:tasks.workunit.client.1.smithi149.stdout:7/461: truncate d9/dc/d2e/d51/f59 2195679 0 2024-04-24T08:24:47.437 INFO:tasks.workunit.client.0.smithi090.stdout:9/88: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:47.438 INFO:tasks.workunit.client.1.smithi149.stdout:1/568: truncate d0/d1b/d5d/f6c 1398501 0 2024-04-24T08:24:47.441 INFO:tasks.workunit.client.0.smithi090.stdout:9/89: chown d0/f5 909069 1 2024-04-24T08:24:47.442 INFO:tasks.workunit.client.0.smithi090.stdout:3/77: truncate d2/f9 2918034 0 2024-04-24T08:24:47.443 INFO:tasks.workunit.client.1.smithi149.stdout:9/493: symlink d1/lab 0 2024-04-24T08:24:47.445 INFO:tasks.workunit.client.0.smithi090.stdout:9/90: write d0/dc/fd [377379,60474] 0 2024-04-24T08:24:47.445 INFO:tasks.workunit.client.0.smithi090.stdout:2/124: dread d0/d14/d17/f18 [4194304,4194304] 0 2024-04-24T08:24:47.446 INFO:tasks.workunit.client.1.smithi149.stdout:1/569: write d0/d5/db/d18/d4d/d50/daf/f56 [623787,10359] 0 2024-04-24T08:24:47.447 INFO:tasks.workunit.client.0.smithi090.stdout:1/109: truncate d3/d12/f18 8686364 0 2024-04-24T08:24:47.450 INFO:tasks.workunit.client.0.smithi090.stdout:6/157: fdatasync d8/dc/f10 0 2024-04-24T08:24:47.452 INFO:tasks.workunit.client.0.smithi090.stdout:6/158: chown d8/d1d/d1f/l25 474217 1 2024-04-24T08:24:47.453 INFO:tasks.workunit.client.0.smithi090.stdout:5/193: creat d8/da/f3f x:0 0 0 2024-04-24T08:24:47.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/379: sync 2024-04-24T08:24:47.454 INFO:tasks.workunit.client.1.smithi149.stdout:4/438: sync 2024-04-24T08:24:47.456 INFO:tasks.workunit.client.0.smithi090.stdout:5/194: write d8/d25/f26 [1420991,117775] 0 2024-04-24T08:24:47.456 INFO:tasks.workunit.client.1.smithi149.stdout:8/392: sync 2024-04-24T08:24:47.457 INFO:tasks.workunit.client.1.smithi149.stdout:7/462: mkdir d9/d43/d4f/d66/d96 0 2024-04-24T08:24:47.466 INFO:tasks.workunit.client.1.smithi149.stdout:2/380: write d0/d6/db/d27/f5f [844349,110154] 0 2024-04-24T08:24:47.466 INFO:tasks.workunit.client.1.smithi149.stdout:1/570: creat d0/d5/d4f/d60/fb6 x:0 0 0 2024-04-24T08:24:47.468 INFO:tasks.workunit.client.1.smithi149.stdout:6/349: rename d0/f16 to d0/d1a/f75 0 2024-04-24T08:24:47.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/125: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:47.472 INFO:tasks.workunit.client.0.smithi090.stdout:2/126: chown d0/d7/l23 250463 1 2024-04-24T08:24:47.473 INFO:tasks.workunit.client.1.smithi149.stdout:8/393: read d1/de/f54 [1451698,111725] 0 2024-04-24T08:24:47.475 INFO:tasks.workunit.client.1.smithi149.stdout:3/507: truncate d0/d26/d3c/f65 930652 0 2024-04-24T08:24:47.479 INFO:tasks.workunit.client.1.smithi149.stdout:9/494: dwrite d1/d2/d34/f92 [0,4194304] 0 2024-04-24T08:24:47.483 INFO:tasks.workunit.client.1.smithi149.stdout:0/404: dread d2/fa [0,4194304] 0 2024-04-24T08:24:47.487 INFO:tasks.workunit.client.0.smithi090.stdout:1/110: creat d3/f26 x:0 0 0 2024-04-24T08:24:47.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/111: chown d3/d25 188690 1 2024-04-24T08:24:47.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/112: stat d3/d1d/l24 0 2024-04-24T08:24:47.490 INFO:tasks.workunit.client.0.smithi090.stdout:1/113: fsync d3/d5/f1a 0 2024-04-24T08:24:47.493 INFO:tasks.workunit.client.0.smithi090.stdout:0/188: creat d1a/f3f x:0 0 0 2024-04-24T08:24:47.495 INFO:tasks.workunit.client.0.smithi090.stdout:6/159: rename d8/dc/f1a to d8/d1d/d1f/f2e 0 2024-04-24T08:24:47.498 INFO:tasks.workunit.client.0.smithi090.stdout:9/91: dwrite d0/fa [0,4194304] 0 2024-04-24T08:24:47.501 INFO:tasks.workunit.client.1.smithi149.stdout:5/537: rename db/d12/d22/d6c to db/d12/d60/db2 0 2024-04-24T08:24:47.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/195: mknod d8/d25/c40 0 2024-04-24T08:24:47.504 INFO:tasks.workunit.client.1.smithi149.stdout:1/571: dread d0/d1b/f64 [0,4194304] 0 2024-04-24T08:24:47.505 INFO:tasks.workunit.client.1.smithi149.stdout:2/381: rmdir d0/d6/db/d39/d3d/d44/d57 39 2024-04-24T08:24:47.506 INFO:tasks.workunit.client.1.smithi149.stdout:1/572: fsync d0/d1b/f91 0 2024-04-24T08:24:47.508 INFO:tasks.workunit.client.1.smithi149.stdout:1/573: readlink d0/d5/d4f/d60/l42 0 2024-04-24T08:24:47.508 INFO:tasks.workunit.client.0.smithi090.stdout:9/92: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:47.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/439: mknod d2/c8f 0 2024-04-24T08:24:47.510 INFO:tasks.workunit.client.0.smithi090.stdout:9/93: stat d0/f10 0 2024-04-24T08:24:47.511 INFO:tasks.workunit.client.0.smithi090.stdout:2/127: mkdir d0/d7/d24 0 2024-04-24T08:24:47.512 INFO:tasks.workunit.client.1.smithi149.stdout:8/394: readlink d1/de/d18/d32/l53 0 2024-04-24T08:24:47.513 INFO:tasks.workunit.client.1.smithi149.stdout:1/574: dread d0/d39/d53/d61/f71 [0,4194304] 0 2024-04-24T08:24:47.514 INFO:tasks.workunit.client.0.smithi090.stdout:3/78: creat d2/d15/f16 x:0 0 0 2024-04-24T08:24:47.516 INFO:tasks.workunit.client.0.smithi090.stdout:3/79: write d2/f14 [113466,106949] 0 2024-04-24T08:24:47.516 INFO:tasks.workunit.client.1.smithi149.stdout:3/508: symlink d0/d5/d1a/lb4 0 2024-04-24T08:24:47.517 INFO:tasks.workunit.client.0.smithi090.stdout:3/80: read - d2/d15/f16 zero size 2024-04-24T08:24:47.519 INFO:tasks.workunit.client.0.smithi090.stdout:4/75: sync 2024-04-24T08:24:47.521 INFO:tasks.workunit.client.1.smithi149.stdout:0/405: mknod d2/d3c/c7b 0 2024-04-24T08:24:47.524 INFO:tasks.workunit.client.1.smithi149.stdout:2/382: dwrite d0/f4d [0,4194304] 0 2024-04-24T08:24:47.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/189: read ff [5833044,36947] 0 2024-04-24T08:24:47.526 INFO:tasks.workunit.client.0.smithi090.stdout:0/190: write f18 [5197792,109604] 0 2024-04-24T08:24:47.527 INFO:tasks.workunit.client.1.smithi149.stdout:2/383: truncate d0/d6/db/d39/d3d/f51 989371 0 2024-04-24T08:24:47.531 INFO:tasks.workunit.client.0.smithi090.stdout:6/160: rmdir d8/d1d/d1f 39 2024-04-24T08:24:47.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/350: mkdir d0/d1a/d28/d31/d4c/d76 0 2024-04-24T08:24:47.538 INFO:tasks.workunit.client.0.smithi090.stdout:1/114: dread f0 [0,4194304] 0 2024-04-24T08:24:47.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/94: rename d0/dc/f13 to d0/f15 0 2024-04-24T08:24:47.540 INFO:tasks.workunit.client.0.smithi090.stdout:2/128: rmdir d0/d14/d17 39 2024-04-24T08:24:47.544 INFO:tasks.workunit.client.0.smithi090.stdout:7/174: dwrite db/d17/d39/f3c [0,4194304] 0 2024-04-24T08:24:47.544 INFO:tasks.workunit.client.0.smithi090.stdout:3/81: stat d2/df 0 2024-04-24T08:24:47.545 INFO:tasks.workunit.client.1.smithi149.stdout:1/575: dread - d0/da/f48 zero size 2024-04-24T08:24:47.547 INFO:tasks.workunit.client.0.smithi090.stdout:4/76: mkdir d3/df 0 2024-04-24T08:24:47.548 INFO:tasks.workunit.client.1.smithi149.stdout:1/576: write d0/d1b/d5d/f85 [316365,89188] 0 2024-04-24T08:24:47.549 INFO:tasks.workunit.client.0.smithi090.stdout:6/161: dwrite d8/d1d/d29/f2a [0,4194304] 0 2024-04-24T08:24:47.550 INFO:tasks.workunit.client.1.smithi149.stdout:3/509: creat d0/d5/d53/d34/d3a/d93/fb5 x:0 0 0 2024-04-24T08:24:47.554 INFO:tasks.workunit.client.0.smithi090.stdout:5/196: dread d8/f31 [0,4194304] 0 2024-04-24T08:24:47.554 INFO:tasks.workunit.client.0.smithi090.stdout:0/191: write d1a/d2a/f32 [118221,6011] 0 2024-04-24T08:24:47.557 INFO:tasks.workunit.client.0.smithi090.stdout:1/115: fdatasync d3/d5/f17 0 2024-04-24T08:24:47.560 INFO:tasks.workunit.client.1.smithi149.stdout:4/440: dwrite d2/d24/d2e/d3f/f67 [0,4194304] 0 2024-04-24T08:24:47.562 INFO:tasks.workunit.client.0.smithi090.stdout:9/95: mknod d0/c16 0 2024-04-24T08:24:47.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/197: dwrite d8/da/d18/f1f [0,4194304] 0 2024-04-24T08:24:47.570 INFO:tasks.workunit.client.1.smithi149.stdout:9/495: creat d1/da/dd/fac x:0 0 0 2024-04-24T08:24:47.583 INFO:tasks.workunit.client.1.smithi149.stdout:2/384: mkdir d0/d6/db/d39/d78/d7c 0 2024-04-24T08:24:47.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/351: getdents d0/d1a/d28/d31/d38/d54 0 2024-04-24T08:24:47.584 INFO:tasks.workunit.client.1.smithi149.stdout:1/577: mknod d0/d39/cb7 0 2024-04-24T08:24:47.584 INFO:tasks.workunit.client.1.smithi149.stdout:5/538: link db/d12/d22/l7b db/d12/d2f/d74/lb3 0 2024-04-24T08:24:47.586 INFO:tasks.workunit.client.1.smithi149.stdout:6/352: write d0/d1a/d28/d31/d38/f3a [1201085,95496] 0 2024-04-24T08:24:47.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/578: truncate d0/d5/db/d63/f86 1004508 0 2024-04-24T08:24:47.588 INFO:tasks.workunit.client.1.smithi149.stdout:9/496: mkdir d1/da/d4a/dad 0 2024-04-24T08:24:47.588 INFO:tasks.workunit.client.1.smithi149.stdout:0/406: link d2/l3 d2/d9/df/d2f/d6c/l7c 0 2024-04-24T08:24:47.589 INFO:tasks.workunit.client.1.smithi149.stdout:2/385: rmdir d0/d6/db/d39/d3d/d58 39 2024-04-24T08:24:47.589 INFO:tasks.workunit.client.1.smithi149.stdout:3/510: creat d0/d26/d3c/db1/fb6 x:0 0 0 2024-04-24T08:24:47.590 INFO:tasks.workunit.client.1.smithi149.stdout:1/579: fsync d0/d5/d4f/d60/fb6 0 2024-04-24T08:24:47.592 INFO:tasks.workunit.client.1.smithi149.stdout:5/539: stat db/d12/d1e/d65/l6f 0 2024-04-24T08:24:47.593 INFO:tasks.workunit.client.1.smithi149.stdout:0/407: write d2/f8 [3269519,102954] 0 2024-04-24T08:24:47.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/90: sync 2024-04-24T08:24:47.595 INFO:tasks.workunit.client.1.smithi149.stdout:3/511: fsync d0/d5/da/d5a/d63/d6a/f7c 0 2024-04-24T08:24:47.599 INFO:tasks.workunit.client.1.smithi149.stdout:9/497: dread d1/f88 [0,4194304] 0 2024-04-24T08:24:47.600 INFO:tasks.workunit.client.0.smithi090.stdout:8/91: chown d2/f12 1658625 1 2024-04-24T08:24:47.600 INFO:tasks.workunit.client.0.smithi090.stdout:7/175: creat db/d17/d29/d38/f44 x:0 0 0 2024-04-24T08:24:47.601 INFO:tasks.workunit.client.0.smithi090.stdout:1/116: dread f0 [0,4194304] 0 2024-04-24T08:24:47.601 INFO:tasks.workunit.client.0.smithi090.stdout:5/198: write d8/da/f1d [50041,21768] 0 2024-04-24T08:24:47.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/77: truncate d3/f9 3050952 0 2024-04-24T08:24:47.602 INFO:tasks.workunit.client.1.smithi149.stdout:1/580: getdents d0/d5/db/d18/d2f 0 2024-04-24T08:24:47.602 INFO:tasks.workunit.client.1.smithi149.stdout:0/408: creat d2/d17/d41/f7d x:0 0 0 2024-04-24T08:24:47.603 INFO:tasks.workunit.client.1.smithi149.stdout:5/540: creat db/d12/d1e/d65/fb4 x:0 0 0 2024-04-24T08:24:47.603 INFO:tasks.workunit.client.1.smithi149.stdout:9/498: chown d1/da/dd/c7e 210977 1 2024-04-24T08:24:47.604 INFO:tasks.workunit.client.0.smithi090.stdout:5/199: write f7 [8262393,106803] 0 2024-04-24T08:24:47.605 INFO:tasks.workunit.client.1.smithi149.stdout:2/386: getdents d0/d6/db 0 2024-04-24T08:24:47.607 INFO:tasks.workunit.client.0.smithi090.stdout:2/129: symlink d0/d14/d17/l25 0 2024-04-24T08:24:47.610 INFO:tasks.workunit.client.1.smithi149.stdout:6/353: dread d0/d1a/f69 [0,4194304] 0 2024-04-24T08:24:47.612 INFO:tasks.workunit.client.0.smithi090.stdout:4/78: creat d3/f10 x:0 0 0 2024-04-24T08:24:47.614 INFO:tasks.workunit.client.1.smithi149.stdout:5/541: creat db/da4/fb5 x:0 0 0 2024-04-24T08:24:47.614 INFO:tasks.workunit.client.1.smithi149.stdout:0/409: dread d2/d9/d1a/d36/f38 [0,4194304] 0 2024-04-24T08:24:47.615 INFO:tasks.workunit.client.0.smithi090.stdout:5/200: fsync d8/f9 0 2024-04-24T08:24:47.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/387: mknod d0/d6/db/c7d 0 2024-04-24T08:24:47.617 INFO:tasks.workunit.client.1.smithi149.stdout:8/395: truncate d1/de/f6d 1310379 0 2024-04-24T08:24:47.618 INFO:tasks.workunit.client.0.smithi090.stdout:5/201: write d8/da/fd [1321902,109526] 0 2024-04-24T08:24:47.620 INFO:tasks.workunit.client.0.smithi090.stdout:1/117: dread d3/d12/f1b [0,4194304] 0 2024-04-24T08:24:47.621 INFO:tasks.workunit.client.0.smithi090.stdout:5/202: chown d8/da/d3a/l3b 472951 1 2024-04-24T08:24:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/396: dread d1/de/d18/f4c [0,4194304] 0 2024-04-24T08:24:47.623 INFO:tasks.workunit.client.1.smithi149.stdout:3/512: rmdir d0/d5/d53/d4c/d5f/d86 0 2024-04-24T08:24:47.623 INFO:tasks.workunit.client.0.smithi090.stdout:9/96: dwrite d0/f15 [0,4194304] 0 2024-04-24T08:24:47.623 INFO:tasks.workunit.client.0.smithi090.stdout:1/118: chown d3/d5/da/dc/fe 5792972 1 2024-04-24T08:24:47.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/119: chown d3/d12/f1b 262 1 2024-04-24T08:24:47.624 INFO:tasks.workunit.client.0.smithi090.stdout:3/82: creat d2/f17 x:0 0 0 2024-04-24T08:24:47.624 INFO:tasks.workunit.client.1.smithi149.stdout:9/499: dwrite d1/d2/d15/d1c/f20 [0,4194304] 0 2024-04-24T08:24:47.624 INFO:tasks.workunit.client.1.smithi149.stdout:7/463: truncate d9/dc/d2e/d51/f59 2057657 0 2024-04-24T08:24:47.625 INFO:tasks.workunit.client.0.smithi090.stdout:3/83: write d2/df/f10 [913036,107202] 0 2024-04-24T08:24:47.630 INFO:tasks.workunit.client.1.smithi149.stdout:7/464: stat d9/dc/d2e/c5d 0 2024-04-24T08:24:47.630 INFO:tasks.workunit.client.0.smithi090.stdout:3/84: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:47.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/465: stat d9/dc/d2e/f39 0 2024-04-24T08:24:47.635 INFO:tasks.workunit.client.1.smithi149.stdout:0/410: symlink d2/d9/l7e 0 2024-04-24T08:24:47.638 INFO:tasks.workunit.client.1.smithi149.stdout:2/388: unlink d0/d6/c13 0 2024-04-24T08:24:47.647 INFO:tasks.workunit.client.1.smithi149.stdout:5/542: fdatasync db/d12/f72 0 2024-04-24T08:24:47.649 INFO:tasks.workunit.client.0.smithi090.stdout:5/203: creat d8/da/d18/d21/f41 x:0 0 0 2024-04-24T08:24:47.650 INFO:tasks.workunit.client.1.smithi149.stdout:7/466: mknod d9/dc/de/d44/c97 0 2024-04-24T08:24:47.650 INFO:tasks.workunit.client.0.smithi090.stdout:5/204: readlink d8/da/l11 0 2024-04-24T08:24:47.651 INFO:tasks.workunit.client.0.smithi090.stdout:9/97: mknod d0/c17 0 2024-04-24T08:24:47.651 INFO:tasks.workunit.client.0.smithi090.stdout:9/98: readlink d0/l9 0 2024-04-24T08:24:47.658 INFO:tasks.workunit.client.0.smithi090.stdout:2/130: mknod d0/d7/d24/c26 0 2024-04-24T08:24:47.659 INFO:tasks.workunit.client.1.smithi149.stdout:9/500: mkdir d1/d2/d15/d1c/d42/dae 0 2024-04-24T08:24:47.661 INFO:tasks.workunit.client.1.smithi149.stdout:6/354: getdents d0/d1a/d28/d31/d4c/d4e/d62 0 2024-04-24T08:24:47.661 INFO:tasks.workunit.client.0.smithi090.stdout:1/120: dread d3/d12/f18 [4194304,4194304] 0 2024-04-24T08:24:47.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/176: truncate db/f1a 1457145 0 2024-04-24T08:24:47.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/177: chown f9 526 1 2024-04-24T08:24:47.665 INFO:tasks.workunit.client.0.smithi090.stdout:4/79: link d3/f10 d3/df/f11 0 2024-04-24T08:24:47.665 INFO:tasks.workunit.client.1.smithi149.stdout:1/581: truncate d0/d5/db/d18/d4d/f90 67232 0 2024-04-24T08:24:47.666 INFO:tasks.workunit.client.0.smithi090.stdout:4/80: chown d3/lb 2199 1 2024-04-24T08:24:47.666 INFO:tasks.workunit.client.0.smithi090.stdout:0/192: link l10 d1a/d2a/d35/l40 0 2024-04-24T08:24:47.669 INFO:tasks.workunit.client.1.smithi149.stdout:7/467: unlink d9/d43/d4c/f86 0 2024-04-24T08:24:47.670 INFO:tasks.workunit.client.1.smithi149.stdout:7/468: chown d9/dc/de 8210 1 2024-04-24T08:24:47.671 INFO:tasks.workunit.client.0.smithi090.stdout:2/131: dwrite d0/d7/f11 [4194304,4194304] 0 2024-04-24T08:24:47.673 INFO:tasks.workunit.client.1.smithi149.stdout:0/411: link d2/d17/f24 d2/f7f 0 2024-04-24T08:24:47.677 INFO:tasks.workunit.client.0.smithi090.stdout:3/85: symlink d2/l18 0 2024-04-24T08:24:47.678 INFO:tasks.workunit.client.0.smithi090.stdout:8/92: getdents d2/da 0 2024-04-24T08:24:47.684 INFO:tasks.workunit.client.1.smithi149.stdout:6/355: creat d0/d1a/d28/d4a/f77 x:0 0 0 2024-04-24T08:24:47.685 INFO:tasks.workunit.client.0.smithi090.stdout:2/132: dwrite d0/d7/f11 [8388608,4194304] 0 2024-04-24T08:24:47.687 INFO:tasks.workunit.client.0.smithi090.stdout:2/133: stat d0/d14/d17/l25 0 2024-04-24T08:24:47.687 INFO:tasks.workunit.client.1.smithi149.stdout:5/543: creat db/d12/d60/d8f/da0/fb6 x:0 0 0 2024-04-24T08:24:47.689 INFO:tasks.workunit.client.1.smithi149.stdout:6/356: read d0/d1a/f75 [4664151,16905] 0 2024-04-24T08:24:47.690 INFO:tasks.workunit.client.0.smithi090.stdout:7/178: symlink db/d17/d29/d32/l45 0 2024-04-24T08:24:47.692 INFO:tasks.workunit.client.1.smithi149.stdout:5/544: chown db/d12/d22/l2e 26022 1 2024-04-24T08:24:47.693 INFO:tasks.workunit.client.1.smithi149.stdout:9/501: dwrite d1/d2/fa0 [0,4194304] 0 2024-04-24T08:24:47.703 INFO:tasks.workunit.client.1.smithi149.stdout:0/412: creat d2/d9/df/d56/f80 x:0 0 0 2024-04-24T08:24:47.706 INFO:tasks.workunit.client.1.smithi149.stdout:0/413: write d2/fa [3861513,87379] 0 2024-04-24T08:24:47.710 INFO:tasks.workunit.client.1.smithi149.stdout:0/414: chown d2/d9/df/d40/c63 486649611 1 2024-04-24T08:24:47.712 INFO:tasks.workunit.client.1.smithi149.stdout:0/415: stat d2/d9/df/d40/d4a/d4e/d6f/d7a 0 2024-04-24T08:24:47.713 INFO:tasks.workunit.client.1.smithi149.stdout:6/357: dwrite d0/d1a/d28/d4a/f67 [0,4194304] 0 2024-04-24T08:24:47.714 INFO:tasks.workunit.client.0.smithi090.stdout:3/86: creat d2/df/f19 x:0 0 0 2024-04-24T08:24:47.715 INFO:tasks.workunit.client.1.smithi149.stdout:6/358: readlink d0/ld 0 2024-04-24T08:24:47.718 INFO:tasks.workunit.client.0.smithi090.stdout:9/99: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:47.718 INFO:tasks.workunit.client.0.smithi090.stdout:7/179: dread db/f10 [0,4194304] 0 2024-04-24T08:24:47.719 INFO:tasks.workunit.client.0.smithi090.stdout:8/93: chown d2/da/lc 4219 1 2024-04-24T08:24:47.719 INFO:tasks.workunit.client.1.smithi149.stdout:5/545: symlink db/d12/d60/db2/lb7 0 2024-04-24T08:24:47.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/502: read - d1/d2/d15/d1c/d42/f65 zero size 2024-04-24T08:24:47.723 INFO:tasks.workunit.client.0.smithi090.stdout:2/134: creat d0/d7/d24/f27 x:0 0 0 2024-04-24T08:24:47.724 INFO:tasks.workunit.client.1.smithi149.stdout:1/582: write d0/d1b/d4b/f58 [40827,27391] 0 2024-04-24T08:24:47.724 INFO:tasks.workunit.client.0.smithi090.stdout:8/94: truncate d2/fd 4908436 0 2024-04-24T08:24:47.729 INFO:tasks.workunit.client.0.smithi090.stdout:0/193: unlink c16 0 2024-04-24T08:24:47.733 INFO:tasks.workunit.client.1.smithi149.stdout:6/359: rmdir d0/d1a/d28 39 2024-04-24T08:24:47.736 INFO:tasks.workunit.client.0.smithi090.stdout:1/121: dwrite d3/d12/f18 [4194304,4194304] 0 2024-04-24T08:24:47.737 INFO:tasks.workunit.client.0.smithi090.stdout:4/81: dwrite d3/df/f11 [0,4194304] 0 2024-04-24T08:24:47.741 INFO:tasks.workunit.client.0.smithi090.stdout:7/180: dread db/f15 [0,4194304] 0 2024-04-24T08:24:47.741 INFO:tasks.workunit.client.1.smithi149.stdout:9/503: read d1/da/dd/f24 [3463626,23281] 0 2024-04-24T08:24:47.741 INFO:tasks.workunit.client.1.smithi149.stdout:5/546: mknod db/d12/d1e/d65/cb8 0 2024-04-24T08:24:47.744 INFO:tasks.workunit.client.1.smithi149.stdout:2/389: link d0/f2d d0/d6/db/f7e 0 2024-04-24T08:24:47.744 INFO:tasks.workunit.client.0.smithi090.stdout:9/100: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:47.745 INFO:tasks.workunit.client.1.smithi149.stdout:5/547: truncate db/d12/d1e/d85/fa5 218426 0 2024-04-24T08:24:47.745 INFO:tasks.workunit.client.1.smithi149.stdout:2/390: dread - d0/f70 zero size 2024-04-24T08:24:47.745 INFO:tasks.workunit.client.1.smithi149.stdout:0/416: dread d2/f7f [0,4194304] 0 2024-04-24T08:24:47.749 INFO:tasks.workunit.client.0.smithi090.stdout:7/181: write db/d17/f2d [474124,18723] 0 2024-04-24T08:24:47.749 INFO:tasks.workunit.client.0.smithi090.stdout:3/87: dwrite d2/df/f10 [0,4194304] 0 2024-04-24T08:24:47.749 INFO:tasks.workunit.client.1.smithi149.stdout:5/548: write db/d23/f43 [413507,60848] 0 2024-04-24T08:24:47.750 INFO:tasks.workunit.client.1.smithi149.stdout:9/504: rmdir d1 39 2024-04-24T08:24:47.753 INFO:tasks.workunit.client.0.smithi090.stdout:2/135: rmdir d0/d14 39 2024-04-24T08:24:47.756 INFO:tasks.workunit.client.0.smithi090.stdout:0/194: rmdir d1a/d2a/d35 39 2024-04-24T08:24:47.758 INFO:tasks.workunit.client.0.smithi090.stdout:8/95: mknod d2/da/d13/c23 0 2024-04-24T08:24:47.760 INFO:tasks.workunit.client.0.smithi090.stdout:4/82: symlink d3/l12 0 2024-04-24T08:24:47.760 INFO:tasks.workunit.client.0.smithi090.stdout:8/96: dread - d2/f22 zero size 2024-04-24T08:24:47.762 INFO:tasks.workunit.client.0.smithi090.stdout:4/83: readlink d3/l12 0 2024-04-24T08:24:47.762 INFO:tasks.workunit.client.1.smithi149.stdout:0/417: symlink d2/d9/d1a/d36/l81 0 2024-04-24T08:24:47.766 INFO:tasks.workunit.client.0.smithi090.stdout:1/122: dwrite d3/f26 [0,4194304] 0 2024-04-24T08:24:47.767 INFO:tasks.workunit.client.1.smithi149.stdout:5/549: mknod db/d12/cb9 0 2024-04-24T08:24:47.768 INFO:tasks.workunit.client.0.smithi090.stdout:3/88: read d2/f9 [1529736,3581] 0 2024-04-24T08:24:47.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/136: dread - d0/d14/d17/f1b zero size 2024-04-24T08:24:47.771 INFO:tasks.workunit.client.1.smithi149.stdout:5/550: chown db/d12/d22/d45/f42 15890364 1 2024-04-24T08:24:47.771 INFO:tasks.workunit.client.0.smithi090.stdout:7/182: dwrite db/d17/d29/d32/f3d [0,4194304] 0 2024-04-24T08:24:47.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/101: link d0/dc/fd d0/f18 0 2024-04-24T08:24:47.773 INFO:tasks.workunit.client.1.smithi149.stdout:0/418: creat d2/d17/d41/d68/f82 x:0 0 0 2024-04-24T08:24:47.775 INFO:tasks.workunit.client.0.smithi090.stdout:9/102: read d0/f10 [1401822,55846] 0 2024-04-24T08:24:47.775 INFO:tasks.workunit.client.0.smithi090.stdout:7/183: write db/f11 [1521381,92787] 0 2024-04-24T08:24:47.775 INFO:tasks.workunit.client.0.smithi090.stdout:4/84: read d3/f9 [686133,70035] 0 2024-04-24T08:24:47.776 INFO:tasks.workunit.client.1.smithi149.stdout:9/505: rename d1/d2/d15/d1c/fa4 to d1/d99/faf 0 2024-04-24T08:24:47.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/103: fsync d0/f10 0 2024-04-24T08:24:47.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:47 smithi149 ceph-mon[31282]: pgmap v16: 65 pgs: 65 active+clean; 983 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 16 MiB/s rd, 128 MiB/s wr, 274 op/s 2024-04-24T08:24:47.778 INFO:tasks.workunit.client.0.smithi090.stdout:7/184: chown db/d17/d39/f41 63164 1 2024-04-24T08:24:47.780 INFO:tasks.workunit.client.0.smithi090.stdout:7/185: write db/f11 [4463136,67859] 0 2024-04-24T08:24:47.781 INFO:tasks.workunit.client.1.smithi149.stdout:9/506: write d1/da/dd/d79/fa1 [309425,20358] 0 2024-04-24T08:24:47.784 INFO:tasks.workunit.client.0.smithi090.stdout:0/195: mkdir d1a/d2a/d35/d41 0 2024-04-24T08:24:47.784 INFO:tasks.workunit.client.0.smithi090.stdout:7/186: dread - db/d17/d39/f43 zero size 2024-04-24T08:24:47.785 INFO:tasks.workunit.client.0.smithi090.stdout:7/187: chown db/d17/d29/d32/f40 32115 1 2024-04-24T08:24:47.787 INFO:tasks.workunit.client.0.smithi090.stdout:7/188: readlink db/d17/d1b/l35 0 2024-04-24T08:24:47.788 INFO:tasks.workunit.client.1.smithi149.stdout:5/551: rename db/d12/d22/d45/l90 to db/d12/d60/d8f/lba 0 2024-04-24T08:24:47.788 INFO:tasks.workunit.client.0.smithi090.stdout:0/196: read d1a/d2a/f32 [24886,130793] 0 2024-04-24T08:24:47.788 INFO:tasks.workunit.client.1.smithi149.stdout:5/552: chown db/da4 52792284 1 2024-04-24T08:24:47.790 INFO:tasks.workunit.client.0.smithi090.stdout:7/189: chown db/d17/d39/f3c 15874 1 2024-04-24T08:24:47.792 INFO:tasks.workunit.client.1.smithi149.stdout:1/583: dread d0/fe [0,4194304] 0 2024-04-24T08:24:47.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:47 smithi090 ceph-mon[25016]: pgmap v16: 65 pgs: 65 active+clean; 983 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 16 MiB/s rd, 128 MiB/s wr, 274 op/s 2024-04-24T08:24:47.793 INFO:tasks.workunit.client.0.smithi090.stdout:2/137: mkdir d0/d7/d28 0 2024-04-24T08:24:47.793 INFO:tasks.workunit.client.1.smithi149.stdout:5/553: write db/d12/f84 [1553118,127436] 0 2024-04-24T08:24:47.797 INFO:tasks.workunit.client.0.smithi090.stdout:3/89: truncate d2/f9 3819298 0 2024-04-24T08:24:47.798 INFO:tasks.workunit.client.1.smithi149.stdout:4/441: sync 2024-04-24T08:24:47.799 INFO:tasks.workunit.client.1.smithi149.stdout:5/554: read db/d12/d22/d45/f42 [1712130,44456] 0 2024-04-24T08:24:47.799 INFO:tasks.workunit.client.1.smithi149.stdout:6/360: truncate d0/d1a/f4d 477604 0 2024-04-24T08:24:47.800 INFO:tasks.workunit.client.1.smithi149.stdout:1/584: write d0/d39/d53/d61/f70 [1563547,13638] 0 2024-04-24T08:24:47.800 INFO:tasks.workunit.client.1.smithi149.stdout:9/507: rename d1/da/dd/c5e to d1/d2/d15/d1c/d42/cb0 0 2024-04-24T08:24:47.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/555: fdatasync db/d12/f84 0 2024-04-24T08:24:47.808 INFO:tasks.workunit.client.0.smithi090.stdout:9/104: mkdir d0/d19 0 2024-04-24T08:24:47.809 INFO:tasks.workunit.client.1.smithi149.stdout:2/391: mkdir d0/d6/db/d28/d7f 0 2024-04-24T08:24:47.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/85: rename d3/fd to d3/f13 0 2024-04-24T08:24:47.823 INFO:tasks.workunit.client.1.smithi149.stdout:0/419: dwrite d2/d9/df/d40/d4a/f79 [0,4194304] 0 2024-04-24T08:24:47.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/197: fdatasync f12 0 2024-04-24T08:24:47.826 INFO:tasks.workunit.client.0.smithi090.stdout:7/190: mkdir db/d17/d29/d38/d3e/d46 0 2024-04-24T08:24:47.831 INFO:tasks.workunit.client.0.smithi090.stdout:2/138: creat d0/d7/d24/f29 x:0 0 0 2024-04-24T08:24:47.835 INFO:tasks.workunit.client.0.smithi090.stdout:8/97: truncate d2/f7 2204868 0 2024-04-24T08:24:47.838 INFO:tasks.workunit.client.1.smithi149.stdout:2/392: stat d0/d6/db/d27/f3b 0 2024-04-24T08:24:47.844 INFO:tasks.workunit.client.1.smithi149.stdout:0/420: dwrite d2/d9/df/f59 [0,4194304] 0 2024-04-24T08:24:47.845 INFO:tasks.workunit.client.1.smithi149.stdout:1/585: unlink d0/c81 0 2024-04-24T08:24:47.847 INFO:tasks.workunit.client.1.smithi149.stdout:6/361: dwrite d0/f12 [0,4194304] 0 2024-04-24T08:24:47.852 INFO:tasks.workunit.client.1.smithi149.stdout:9/508: truncate d1/da/d13/f45 1229775 0 2024-04-24T08:24:47.856 INFO:tasks.workunit.client.0.smithi090.stdout:7/191: mknod db/d17/d39/c47 0 2024-04-24T08:24:47.859 INFO:tasks.workunit.client.1.smithi149.stdout:1/586: dread d0/d5/db/d63/f83 [0,4194304] 0 2024-04-24T08:24:47.859 INFO:tasks.workunit.client.0.smithi090.stdout:3/90: fsync d2/f9 0 2024-04-24T08:24:47.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/105: creat d0/d19/f1a x:0 0 0 2024-04-24T08:24:47.864 INFO:tasks.workunit.client.0.smithi090.stdout:3/91: read - d2/df/f19 zero size 2024-04-24T08:24:47.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/442: dwrite d2/f6c [0,4194304] 0 2024-04-24T08:24:47.867 INFO:tasks.workunit.client.0.smithi090.stdout:8/98: rename d2/d19 to d2/da/d13/d15/d24 0 2024-04-24T08:24:47.867 INFO:tasks.workunit.client.0.smithi090.stdout:4/86: mknod d3/df/c14 0 2024-04-24T08:24:47.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/123: getdents d3/d5/da 0 2024-04-24T08:24:47.868 INFO:tasks.workunit.client.1.smithi149.stdout:5/556: link db/d12/d22/c86 db/d12/d60/d8f/da0/cbb 0 2024-04-24T08:24:47.875 INFO:tasks.workunit.client.0.smithi090.stdout:0/198: dwrite d1a/f24 [0,4194304] 0 2024-04-24T08:24:47.875 INFO:tasks.workunit.client.0.smithi090.stdout:7/192: mknod db/d17/d29/d32/c48 0 2024-04-24T08:24:47.879 INFO:tasks.workunit.client.0.smithi090.stdout:7/193: write db/fd [3501480,34699] 0 2024-04-24T08:24:47.881 INFO:tasks.workunit.client.1.smithi149.stdout:1/587: dwrite d0/f84 [0,4194304] 0 2024-04-24T08:24:47.883 INFO:tasks.workunit.client.0.smithi090.stdout:9/106: unlink d0/dc/le 0 2024-04-24T08:24:47.887 INFO:tasks.workunit.client.0.smithi090.stdout:3/92: dwrite d2/df/f10 [0,4194304] 0 2024-04-24T08:24:47.890 INFO:tasks.workunit.client.1.smithi149.stdout:4/443: mknod d2/d18/d1e/d3c/c90 0 2024-04-24T08:24:47.893 INFO:tasks.workunit.client.0.smithi090.stdout:8/99: symlink d2/da/d13/l25 0 2024-04-24T08:24:47.894 INFO:tasks.workunit.client.0.smithi090.stdout:1/124: creat d3/d5/da/dc/f27 x:0 0 0 2024-04-24T08:24:47.895 INFO:tasks.workunit.client.1.smithi149.stdout:4/444: stat d2/d18/d1e/d3c/f84 0 2024-04-24T08:24:47.897 INFO:tasks.workunit.client.1.smithi149.stdout:0/421: rename d2/c73 to d2/d17/c83 0 2024-04-24T08:24:47.898 INFO:tasks.workunit.client.1.smithi149.stdout:5/557: dwrite db/d12/d22/d45/f93 [0,4194304] 0 2024-04-24T08:24:47.902 INFO:tasks.workunit.client.1.smithi149.stdout:5/558: readlink db/d12/d2f/l39 0 2024-04-24T08:24:47.904 INFO:tasks.workunit.client.0.smithi090.stdout:7/194: rename db/d17/d39/f43 to db/d17/d39/f49 0 2024-04-24T08:24:47.905 INFO:tasks.workunit.client.1.smithi149.stdout:9/509: creat d1/da/dd/d55/fb1 x:0 0 0 2024-04-24T08:24:47.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/195: read - db/d17/d29/d38/d3e/f3f zero size 2024-04-24T08:24:47.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/107: creat d0/f1b x:0 0 0 2024-04-24T08:24:47.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/196: readlink db/d17/d29/l33 0 2024-04-24T08:24:47.909 INFO:tasks.workunit.client.1.smithi149.stdout:5/559: write db/d12/f5b [581558,89449] 0 2024-04-24T08:24:47.909 INFO:tasks.workunit.client.1.smithi149.stdout:2/393: creat d0/d6/db/d39/f80 x:0 0 0 2024-04-24T08:24:47.912 INFO:tasks.workunit.client.1.smithi149.stdout:6/362: rename d0/d1a/d28/d31/d4c/c4f to d0/d1a/d28/d59/c78 0 2024-04-24T08:24:47.914 INFO:tasks.workunit.client.1.smithi149.stdout:5/560: chown db/d12/d2f/d95/f9e 59717 1 2024-04-24T08:24:47.914 INFO:tasks.workunit.client.0.smithi090.stdout:3/93: mkdir d2/df/d1a 0 2024-04-24T08:24:47.915 INFO:tasks.workunit.client.1.smithi149.stdout:6/363: write d0/fc [1559748,75924] 0 2024-04-24T08:24:47.919 INFO:tasks.workunit.client.1.smithi149.stdout:5/561: write db/d12/d60/d8f/da0/fb6 [260208,51184] 0 2024-04-24T08:24:47.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/139: getdents d0 0 2024-04-24T08:24:47.921 INFO:tasks.workunit.client.0.smithi090.stdout:1/125: creat d3/d5/f28 x:0 0 0 2024-04-24T08:24:47.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/140: dread - d0/d14/d17/f1b zero size 2024-04-24T08:24:47.925 INFO:tasks.workunit.client.0.smithi090.stdout:8/100: rename d2/fd to d2/da/d13/d15/f26 0 2024-04-24T08:24:47.927 INFO:tasks.workunit.client.0.smithi090.stdout:8/101: fsync d2/da/d13/d15/f16 0 2024-04-24T08:24:47.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/108: creat d0/d19/f1c x:0 0 0 2024-04-24T08:24:47.929 INFO:tasks.workunit.client.0.smithi090.stdout:3/94: dread d2/df/f10 [0,4194304] 0 2024-04-24T08:24:47.933 INFO:tasks.workunit.client.1.smithi149.stdout:5/562: mknod db/d12/d60/db2/d83/cbc 0 2024-04-24T08:24:47.934 INFO:tasks.workunit.client.1.smithi149.stdout:8/397: sync 2024-04-24T08:24:47.935 INFO:tasks.workunit.client.1.smithi149.stdout:9/510: unlink d1/da/c31 0 2024-04-24T08:24:47.935 INFO:tasks.workunit.client.0.smithi090.stdout:1/126: dread f0 [0,4194304] 0 2024-04-24T08:24:47.939 INFO:tasks.workunit.client.1.smithi149.stdout:5/563: truncate db/d35/f7c 527997 0 2024-04-24T08:24:47.939 INFO:tasks.workunit.client.0.smithi090.stdout:0/199: rmdir d1a/d2a/d35/d41 0 2024-04-24T08:24:47.940 INFO:tasks.workunit.client.0.smithi090.stdout:2/141: dread d0/d14/f1f [0,4194304] 0 2024-04-24T08:24:47.941 INFO:tasks.workunit.client.1.smithi149.stdout:0/422: link d2/d9/d1a/c32 d2/d9/df/d40/d4a/d4e/c84 0 2024-04-24T08:24:47.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/95: dread d2/df/f10 [0,4194304] 0 2024-04-24T08:24:47.945 INFO:tasks.workunit.client.0.smithi090.stdout:2/142: read d0/d14/d17/f18 [3865448,82150] 0 2024-04-24T08:24:47.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/398: rename d1/de/f2b to d1/de/d18/d32/d63/d76/f86 0 2024-04-24T08:24:47.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/143: readlink d0/lb 0 2024-04-24T08:24:47.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/102: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:47.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/394: dwrite d0/d6/db/f10 [4194304,4194304] 0 2024-04-24T08:24:47.947 INFO:tasks.workunit.client.1.smithi149.stdout:6/364: dread d0/f26 [0,4194304] 0 2024-04-24T08:24:47.947 INFO:tasks.workunit.client.0.smithi090.stdout:9/109: chown d0/c6 228743 1 2024-04-24T08:24:47.951 INFO:tasks.workunit.client.1.smithi149.stdout:8/399: chown d1/d1f/d25/c4e 1 1 2024-04-24T08:24:47.952 INFO:tasks.workunit.client.1.smithi149.stdout:8/400: dread d1/de/d18/f6a [0,4194304] 0 2024-04-24T08:24:47.955 INFO:tasks.workunit.client.1.smithi149.stdout:9/511: truncate d1/de/d12/f1d 4162668 0 2024-04-24T08:24:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:9/512: write d1/d2/d33/d76/f84 [583059,99031] 0 2024-04-24T08:24:47.958 INFO:tasks.workunit.client.0.smithi090.stdout:0/200: creat d1a/d2a/d35/f42 x:0 0 0 2024-04-24T08:24:47.960 INFO:tasks.workunit.client.1.smithi149.stdout:5/564: rename db/d12/d60/db2/lb7 to db/d12/d60/lbd 0 2024-04-24T08:24:47.964 INFO:tasks.workunit.client.0.smithi090.stdout:7/197: dread db/f12 [0,4194304] 0 2024-04-24T08:24:47.965 INFO:tasks.workunit.client.0.smithi090.stdout:3/96: rename d2/c5 to d2/df/d1a/c1b 0 2024-04-24T08:24:47.966 INFO:tasks.workunit.client.0.smithi090.stdout:8/103: mknod d2/da/d13/c27 0 2024-04-24T08:24:47.968 INFO:tasks.workunit.client.0.smithi090.stdout:3/97: write d2/df/f19 [1037528,41496] 0 2024-04-24T08:24:47.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/395: rmdir d0/d6/db/d27 39 2024-04-24T08:24:47.970 INFO:tasks.workunit.client.0.smithi090.stdout:7/198: write db/d17/d29/f3b [1021163,37262] 0 2024-04-24T08:24:47.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/144: mkdir d0/d14/d17/d2a 0 2024-04-24T08:24:47.971 INFO:tasks.workunit.client.1.smithi149.stdout:6/365: mknod d0/d1a/d28/d4a/c79 0 2024-04-24T08:24:47.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/110: creat d0/d19/f1d x:0 0 0 2024-04-24T08:24:47.973 INFO:tasks.workunit.client.0.smithi090.stdout:1/127: dwrite d3/d12/f1b [0,4194304] 0 2024-04-24T08:24:47.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/128: write d3/d12/f18 [9543337,1079] 0 2024-04-24T08:24:47.977 INFO:tasks.workunit.client.1.smithi149.stdout:1/588: write d0/d5/db/d18/f6f [6850,14918] 0 2024-04-24T08:24:47.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/201: dread f15 [0,4194304] 0 2024-04-24T08:24:47.983 INFO:tasks.workunit.client.1.smithi149.stdout:3/513: sync 2024-04-24T08:24:47.983 INFO:tasks.workunit.client.1.smithi149.stdout:7/469: sync 2024-04-24T08:24:47.983 INFO:tasks.workunit.client.1.smithi149.stdout:9/513: creat d1/d2/d33/fb2 x:0 0 0 2024-04-24T08:24:47.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/98: symlink d2/d15/l1c 0 2024-04-24T08:24:47.988 INFO:tasks.workunit.client.1.smithi149.stdout:3/514: readlink d0/d5/laa 0 2024-04-24T08:24:47.989 INFO:tasks.workunit.client.0.smithi090.stdout:6/162: sync 2024-04-24T08:24:47.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/205: sync 2024-04-24T08:24:47.990 INFO:tasks.workunit.client.0.smithi090.stdout:7/199: creat db/d17/d29/d32/f4a x:0 0 0 2024-04-24T08:24:47.992 INFO:tasks.workunit.client.0.smithi090.stdout:7/200: write f7 [4412399,97362] 0 2024-04-24T08:24:47.992 INFO:tasks.workunit.client.0.smithi090.stdout:7/201: fsync db/f21 0 2024-04-24T08:24:47.993 INFO:tasks.workunit.client.1.smithi149.stdout:6/366: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d7a 0 2024-04-24T08:24:47.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/111: creat d0/d19/f1e x:0 0 0 2024-04-24T08:24:47.995 INFO:tasks.workunit.client.0.smithi090.stdout:9/112: chown d0/c16 6340261 1 2024-04-24T08:24:47.996 INFO:tasks.workunit.client.1.smithi149.stdout:8/401: fsync d1/d1f/f68 0 2024-04-24T08:24:47.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/589: write d0/d1b/d5d/f69 [326681,42960] 0 2024-04-24T08:24:48.001 INFO:tasks.workunit.client.0.smithi090.stdout:1/129: unlink d3/d1d/l24 0 2024-04-24T08:24:48.004 INFO:tasks.workunit.client.1.smithi149.stdout:4/445: dwrite d2/d10/d1a/d34/f37 [0,4194304] 0 2024-04-24T08:24:48.004 INFO:tasks.workunit.client.0.smithi090.stdout:6/163: dwrite d8/f1b [0,4194304] 0 2024-04-24T08:24:48.008 INFO:tasks.workunit.client.1.smithi149.stdout:5/565: unlink db/d12/d1e/f4a 0 2024-04-24T08:24:48.008 INFO:tasks.workunit.client.1.smithi149.stdout:7/470: creat d9/d43/d4c/d67/d69/f98 x:0 0 0 2024-04-24T08:24:48.013 INFO:tasks.workunit.client.1.smithi149.stdout:3/515: rename d0/d5/da/d5a/d63/d6a/l78 to d0/d5/da/d19/d76/lb7 0 2024-04-24T08:24:48.019 INFO:tasks.workunit.client.0.smithi090.stdout:0/202: dwrite d1a/f2c [0,4194304] 0 2024-04-24T08:24:48.023 INFO:tasks.workunit.client.0.smithi090.stdout:5/206: rename d8/da/d3a/f23 to d8/da/d14/d2f/f42 0 2024-04-24T08:24:48.025 INFO:tasks.workunit.client.1.smithi149.stdout:6/367: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/f7b x:0 0 0 2024-04-24T08:24:48.026 INFO:tasks.workunit.client.1.smithi149.stdout:7/471: dwrite d9/d43/d4f/d66/f6e [4194304,4194304] 0 2024-04-24T08:24:48.028 INFO:tasks.workunit.client.1.smithi149.stdout:6/368: truncate d0/f73 687820 0 2024-04-24T08:24:48.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/202: write db/f15 [1710133,10871] 0 2024-04-24T08:24:48.032 INFO:tasks.workunit.client.1.smithi149.stdout:8/402: creat d1/d1f/d5e/f87 x:0 0 0 2024-04-24T08:24:48.037 INFO:tasks.workunit.client.1.smithi149.stdout:0/423: truncate d2/f8 267167 0 2024-04-24T08:24:48.041 INFO:tasks.workunit.client.1.smithi149.stdout:9/514: rmdir d1/d2/d33 39 2024-04-24T08:24:48.042 INFO:tasks.workunit.client.0.smithi090.stdout:1/130: mknod d3/d12/c29 0 2024-04-24T08:24:48.042 INFO:tasks.workunit.client.0.smithi090.stdout:1/131: stat f0 0 2024-04-24T08:24:48.048 INFO:tasks.workunit.client.1.smithi149.stdout:5/566: symlink db/d12/d60/d8f/lbe 0 2024-04-24T08:24:48.049 INFO:tasks.workunit.client.1.smithi149.stdout:7/472: dwrite d9/d43/d4f/d66/f6e [8388608,4194304] 0 2024-04-24T08:24:48.051 INFO:tasks.workunit.client.1.smithi149.stdout:3/516: mknod d0/d5/da/d19/d1b/d22/cb8 0 2024-04-24T08:24:48.053 INFO:tasks.workunit.client.0.smithi090.stdout:2/145: creat d0/d14/f2b x:0 0 0 2024-04-24T08:24:48.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/396: unlink d0/d6/db/d27/f45 0 2024-04-24T08:24:48.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/99: write d2/f14 [514881,66127] 0 2024-04-24T08:24:48.061 INFO:tasks.workunit.client.0.smithi090.stdout:5/207: creat d8/da/d3a/f43 x:0 0 0 2024-04-24T08:24:48.064 INFO:tasks.workunit.client.0.smithi090.stdout:7/203: symlink db/d17/d39/l4b 0 2024-04-24T08:24:48.064 INFO:tasks.workunit.client.0.smithi090.stdout:4/87: sync 2024-04-24T08:24:48.065 INFO:tasks.workunit.client.0.smithi090.stdout:5/208: chown d8/da/d3a/d2d/d37/f39 179967811 1 2024-04-24T08:24:48.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/113: link d0/d19/f1d d0/dc/f1f 0 2024-04-24T08:24:48.066 INFO:tasks.workunit.client.0.smithi090.stdout:5/209: readlink d8/da/d14/l16 0 2024-04-24T08:24:48.068 INFO:tasks.workunit.client.1.smithi149.stdout:4/446: mkdir d2/d10/d91 0 2024-04-24T08:24:48.068 INFO:tasks.workunit.client.0.smithi090.stdout:9/114: stat d0/f15 0 2024-04-24T08:24:48.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/210: dread - d8/da/d18/d21/f41 zero size 2024-04-24T08:24:48.070 INFO:tasks.workunit.client.1.smithi149.stdout:0/424: mkdir d2/d9/d1a/d36/d85 0 2024-04-24T08:24:48.071 INFO:tasks.workunit.client.0.smithi090.stdout:8/104: getdents d2/da/d13/d15 0 2024-04-24T08:24:48.072 INFO:tasks.workunit.client.1.smithi149.stdout:9/515: symlink d1/de/d12/lb3 0 2024-04-24T08:24:48.073 INFO:tasks.workunit.client.0.smithi090.stdout:3/100: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:48.073 INFO:tasks.workunit.client.0.smithi090.stdout:6/164: mkdir d8/d1d/d2f 0 2024-04-24T08:24:48.073 INFO:tasks.workunit.client.1.smithi149.stdout:9/516: chown d1/da/d4a 474249365 1 2024-04-24T08:24:48.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/165: fsync f0 0 2024-04-24T08:24:48.074 INFO:tasks.workunit.client.0.smithi090.stdout:1/132: rmdir d3 39 2024-04-24T08:24:48.076 INFO:tasks.workunit.client.0.smithi090.stdout:8/105: chown d2/da/d13/d15/c1f 132 1 2024-04-24T08:24:48.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/146: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:48.078 INFO:tasks.workunit.client.1.smithi149.stdout:3/517: truncate d0/d5/da/d19/d1b/d22/d38/f64 742 0 2024-04-24T08:24:48.082 INFO:tasks.workunit.client.1.smithi149.stdout:3/518: fsync d0/d5/da/d5a/d63/d6a/f7c 0 2024-04-24T08:24:48.083 INFO:tasks.workunit.client.0.smithi090.stdout:4/88: creat d3/f15 x:0 0 0 2024-04-24T08:24:48.084 INFO:tasks.workunit.client.1.smithi149.stdout:2/397: mkdir d0/d6/db/d27/d81 0 2024-04-24T08:24:48.085 INFO:tasks.workunit.client.1.smithi149.stdout:0/425: dread d2/d9/df/d40/d4a/f64 [0,4194304] 0 2024-04-24T08:24:48.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/426: chown d2/fb 61666963 1 2024-04-24T08:24:48.090 INFO:tasks.workunit.client.1.smithi149.stdout:0/427: dread d2/d9/df/d2f/f43 [0,4194304] 0 2024-04-24T08:24:48.090 INFO:tasks.workunit.client.0.smithi090.stdout:7/204: creat db/d17/d29/d32/f4c x:0 0 0 2024-04-24T08:24:48.090 INFO:tasks.workunit.client.0.smithi090.stdout:3/101: dread d2/df/f10 [0,4194304] 0 2024-04-24T08:24:48.090 INFO:tasks.workunit.client.0.smithi090.stdout:5/211: mkdir d8/da/d14/d2f/d44 0 2024-04-24T08:24:48.092 INFO:tasks.workunit.client.1.smithi149.stdout:4/447: chown d2/cd 354235182 1 2024-04-24T08:24:48.093 INFO:tasks.workunit.client.1.smithi149.stdout:9/517: creat d1/d2/d34/fb4 x:0 0 0 2024-04-24T08:24:48.093 INFO:tasks.workunit.client.0.smithi090.stdout:1/133: truncate d3/d5/f19 272877 0 2024-04-24T08:24:48.094 INFO:tasks.workunit.client.1.smithi149.stdout:1/590: truncate d0/d1b/db5/f8b 3431106 0 2024-04-24T08:24:48.094 INFO:tasks.workunit.client.0.smithi090.stdout:6/166: dwrite d8/d1d/f24 [0,4194304] 0 2024-04-24T08:24:48.096 INFO:tasks.workunit.client.1.smithi149.stdout:7/473: symlink d9/d43/d4c/l99 0 2024-04-24T08:24:48.098 INFO:tasks.workunit.client.1.smithi149.stdout:7/474: write d9/d43/d64/f73 [1428036,101419] 0 2024-04-24T08:24:48.099 INFO:tasks.workunit.client.0.smithi090.stdout:5/212: dread d8/da/f1b [0,4194304] 0 2024-04-24T08:24:48.102 INFO:tasks.workunit.client.1.smithi149.stdout:7/475: write d9/d43/d64/f73 [224160,73255] 0 2024-04-24T08:24:48.105 INFO:tasks.workunit.client.0.smithi090.stdout:6/167: dread d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:48.109 INFO:tasks.workunit.client.1.smithi149.stdout:2/398: chown d0/d6/db/d39/d78 4 1 2024-04-24T08:24:48.109 INFO:tasks.workunit.client.0.smithi090.stdout:4/89: creat d3/f16 x:0 0 0 2024-04-24T08:24:48.111 INFO:tasks.workunit.client.0.smithi090.stdout:3/102: write d2/df/f10 [2351794,78706] 0 2024-04-24T08:24:48.112 INFO:tasks.workunit.client.1.smithi149.stdout:8/403: link d1/de/d18/d43/f55 d1/de/d18/d43/f88 0 2024-04-24T08:24:48.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/106: dwrite d2/da/d13/d15/f26 [0,4194304] 0 2024-04-24T08:24:48.114 INFO:tasks.workunit.client.1.smithi149.stdout:5/567: dwrite db/d12/d22/d45/d5d/f63 [0,4194304] 0 2024-04-24T08:24:48.115 INFO:tasks.workunit.client.1.smithi149.stdout:8/404: readlink d1/de/d18/l59 0 2024-04-24T08:24:48.115 INFO:tasks.workunit.client.0.smithi090.stdout:1/134: mknod d3/c2a 0 2024-04-24T08:24:48.115 INFO:tasks.workunit.client.0.smithi090.stdout:8/107: readlink d2/da/lf 0 2024-04-24T08:24:48.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/135: write d3/d12/f1b [8190457,24782] 0 2024-04-24T08:24:48.118 INFO:tasks.workunit.client.1.smithi149.stdout:2/399: truncate d0/d6/db/d27/d64/f74 512594 0 2024-04-24T08:24:48.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/568: write db/d12/f82 [4324083,51318] 0 2024-04-24T08:24:48.119 INFO:tasks.workunit.client.1.smithi149.stdout:1/591: mkdir d0/d39/da0/db8 0 2024-04-24T08:24:48.123 INFO:tasks.workunit.client.0.smithi090.stdout:4/90: dread d3/f10 [0,4194304] 0 2024-04-24T08:24:48.125 INFO:tasks.workunit.client.1.smithi149.stdout:7/476: symlink d9/dc/de/l9a 0 2024-04-24T08:24:48.127 INFO:tasks.workunit.client.1.smithi149.stdout:6/369: getdents d0 0 2024-04-24T08:24:48.128 INFO:tasks.workunit.client.1.smithi149.stdout:3/519: rename d0/d5/da/d5a/l67 to d0/d5/da/d19/d80/lb9 0 2024-04-24T08:24:48.129 INFO:tasks.workunit.client.0.smithi090.stdout:3/103: dwrite d2/df/f12 [0,4194304] 0 2024-04-24T08:24:48.131 INFO:tasks.workunit.client.1.smithi149.stdout:6/370: write d0/f73 [654197,24223] 0 2024-04-24T08:24:48.132 INFO:tasks.workunit.client.1.smithi149.stdout:5/569: fsync db/d12/d2f/f57 0 2024-04-24T08:24:48.134 INFO:tasks.workunit.client.0.smithi090.stdout:0/203: getdents d1a 0 2024-04-24T08:24:48.135 INFO:tasks.workunit.client.0.smithi090.stdout:7/205: getdents db/d17/d29/d38/d3e/d46 0 2024-04-24T08:24:48.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/213: symlink d8/da/d3a/d36/l45 0 2024-04-24T08:24:48.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/592: creat d0/d5/db/d18/fb9 x:0 0 0 2024-04-24T08:24:48.137 INFO:tasks.workunit.client.0.smithi090.stdout:7/206: read - db/d17/d39/f41 zero size 2024-04-24T08:24:48.138 INFO:tasks.workunit.client.1.smithi149.stdout:6/371: dread - d0/d1a/d28/d31/d4c/d4e/f64 zero size 2024-04-24T08:24:48.141 INFO:tasks.workunit.client.0.smithi090.stdout:1/136: creat d3/d5/da/dc/f2b x:0 0 0 2024-04-24T08:24:48.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/137: fdatasync d3/d5/f28 0 2024-04-24T08:24:48.143 INFO:tasks.workunit.client.1.smithi149.stdout:8/405: symlink d1/d1f/d5e/d6c/d7d/l89 0 2024-04-24T08:24:48.147 INFO:tasks.workunit.client.1.smithi149.stdout:3/520: creat d0/d5/d1a/d70/fba x:0 0 0 2024-04-24T08:24:48.147 INFO:tasks.workunit.client.1.smithi149.stdout:2/400: dwrite d0/f4d [0,4194304] 0 2024-04-24T08:24:48.154 INFO:tasks.workunit.client.1.smithi149.stdout:8/406: symlink d1/de/d18/d43/l8a 0 2024-04-24T08:24:48.155 INFO:tasks.workunit.client.0.smithi090.stdout:7/207: dwrite db/d17/d1b/f26 [0,4194304] 0 2024-04-24T08:24:48.157 INFO:tasks.workunit.client.1.smithi149.stdout:5/570: mknod db/d12/d22/d45/cbf 0 2024-04-24T08:24:48.157 INFO:tasks.workunit.client.1.smithi149.stdout:2/401: write d0/d6/db/d28/f38 [479149,11782] 0 2024-04-24T08:24:48.163 INFO:tasks.workunit.client.1.smithi149.stdout:5/571: symlink db/d12/d22/lc0 0 2024-04-24T08:24:48.167 INFO:tasks.workunit.client.1.smithi149.stdout:6/372: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:48.175 INFO:tasks.workunit.client.1.smithi149.stdout:6/373: mkdir d0/d1a/d28/d4a/d7c 0 2024-04-24T08:24:48.175 INFO:tasks.workunit.client.0.smithi090.stdout:5/214: unlink d8/da/fd 0 2024-04-24T08:24:48.176 INFO:tasks.workunit.client.0.smithi090.stdout:1/138: unlink d3/d12/f1b 0 2024-04-24T08:24:48.180 INFO:tasks.workunit.client.0.smithi090.stdout:7/208: fdatasync f4 0 2024-04-24T08:24:48.180 INFO:tasks.workunit.client.0.smithi090.stdout:5/215: mknod d8/d25/c46 0 2024-04-24T08:24:48.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/108: getdents d2/da/d13 0 2024-04-24T08:24:48.184 INFO:tasks.workunit.client.1.smithi149.stdout:5/572: dwrite db/d12/d60/d8f/fac [4194304,4194304] 0 2024-04-24T08:24:48.185 INFO:tasks.workunit.client.0.smithi090.stdout:3/104: getdents d2/d15 0 2024-04-24T08:24:48.187 INFO:tasks.workunit.client.1.smithi149.stdout:6/374: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d 0 2024-04-24T08:24:48.189 INFO:tasks.workunit.client.0.smithi090.stdout:1/139: dread d3/d5/f16 [0,4194304] 0 2024-04-24T08:24:48.191 INFO:tasks.workunit.client.1.smithi149.stdout:6/375: chown d0/d1a/d28/d31/d38/f3a 303 1 2024-04-24T08:24:48.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/209: symlink db/d17/d29/d32/l4d 0 2024-04-24T08:24:48.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/210: stat db/d17/d29/c37 0 2024-04-24T08:24:48.199 INFO:tasks.workunit.client.1.smithi149.stdout:2/402: dread d0/d6/db/d28/f3a [0,4194304] 0 2024-04-24T08:24:48.200 INFO:tasks.workunit.client.0.smithi090.stdout:8/109: dread d2/f10 [0,4194304] 0 2024-04-24T08:24:48.201 INFO:tasks.workunit.client.0.smithi090.stdout:5/216: rename d8/da/f3f to d8/da/d14/d2f/f47 0 2024-04-24T08:24:48.203 INFO:tasks.workunit.client.0.smithi090.stdout:5/217: chown d8/c2c 331113269 1 2024-04-24T08:24:48.205 INFO:tasks.workunit.client.0.smithi090.stdout:5/218: stat d8/da/d3a/l33 0 2024-04-24T08:24:48.206 INFO:tasks.workunit.client.0.smithi090.stdout:1/140: mknod d3/d1d/c2c 0 2024-04-24T08:24:48.207 INFO:tasks.workunit.client.0.smithi090.stdout:2/147: write d0/d7/fd [1131281,17284] 0 2024-04-24T08:24:48.208 INFO:tasks.workunit.client.1.smithi149.stdout:6/376: mkdir d0/d1a/d7e 0 2024-04-24T08:24:48.210 INFO:tasks.workunit.client.0.smithi090.stdout:7/211: fsync f4 0 2024-04-24T08:24:48.212 INFO:tasks.workunit.client.1.smithi149.stdout:5/573: getdents db/d12/d2f 0 2024-04-24T08:24:48.212 INFO:tasks.workunit.client.0.smithi090.stdout:7/212: dread - db/d17/d29/f30 zero size 2024-04-24T08:24:48.213 INFO:tasks.workunit.client.1.smithi149.stdout:0/428: write d2/d9/df/d2f/f34 [807015,6219] 0 2024-04-24T08:24:48.215 INFO:tasks.workunit.client.0.smithi090.stdout:5/219: creat d8/da/d18/d21/f48 x:0 0 0 2024-04-24T08:24:48.215 INFO:tasks.workunit.client.1.smithi149.stdout:3/521: dread d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:48.215 INFO:tasks.workunit.client.1.smithi149.stdout:4/448: write d2/d24/f39 [81494,4710] 0 2024-04-24T08:24:48.216 INFO:tasks.workunit.client.1.smithi149.stdout:0/429: chown d2/d9/l7e 60011532 1 2024-04-24T08:24:48.217 INFO:tasks.workunit.client.0.smithi090.stdout:5/220: chown d8/da/f20 709 1 2024-04-24T08:24:48.217 INFO:tasks.workunit.client.0.smithi090.stdout:1/141: rename d3/d5/c21 to d3/d5/da/c2d 0 2024-04-24T08:24:48.219 INFO:tasks.workunit.client.1.smithi149.stdout:2/403: creat d0/d6/db/d39/d78/d7c/f82 x:0 0 0 2024-04-24T08:24:48.219 INFO:tasks.workunit.client.0.smithi090.stdout:8/110: dwrite d2/da/d13/d15/f1e [0,4194304] 0 2024-04-24T08:24:48.222 INFO:tasks.workunit.client.0.smithi090.stdout:2/148: mkdir d0/d2c 0 2024-04-24T08:24:48.230 INFO:tasks.workunit.client.1.smithi149.stdout:9/518: dwrite d1/da/dd/f24 [0,4194304] 0 2024-04-24T08:24:48.237 INFO:tasks.workunit.client.0.smithi090.stdout:7/213: rename db/d17/f18 to db/d17/d29/d38/f4e 0 2024-04-24T08:24:48.242 INFO:tasks.workunit.client.1.smithi149.stdout:3/522: truncate d0/d5/da/f28 2648471 0 2024-04-24T08:24:48.242 INFO:tasks.workunit.client.1.smithi149.stdout:3/523: chown d0/d5/d1a/f1e 656 1 2024-04-24T08:24:48.245 INFO:tasks.workunit.client.1.smithi149.stdout:0/430: dwrite d2/d9/df/d48/f66 [0,4194304] 0 2024-04-24T08:24:48.245 INFO:tasks.workunit.client.1.smithi149.stdout:2/404: fsync d0/f35 0 2024-04-24T08:24:48.252 INFO:tasks.workunit.client.1.smithi149.stdout:6/377: dread d0/d1a/f75 [0,4194304] 0 2024-04-24T08:24:48.253 INFO:tasks.workunit.client.1.smithi149.stdout:8/407: rename d1/d1f/d5e/d6c to d1/de/d18/d32/d65/d8b 0 2024-04-24T08:24:48.254 INFO:tasks.workunit.client.1.smithi149.stdout:5/574: creat db/d12/d1e/fc1 x:0 0 0 2024-04-24T08:24:48.259 INFO:tasks.workunit.client.1.smithi149.stdout:2/405: symlink d0/d6/db/d28/l83 0 2024-04-24T08:24:48.264 INFO:tasks.workunit.client.1.smithi149.stdout:5/575: chown db/d12/d60/db2/d83/c8c 21685 1 2024-04-24T08:24:48.264 INFO:tasks.workunit.client.1.smithi149.stdout:7/477: dwrite d9/dc/d11/d26/f28 [0,4194304] 0 2024-04-24T08:24:48.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/378: creat d0/d1a/d28/d4a/f7f x:0 0 0 2024-04-24T08:24:48.268 INFO:tasks.workunit.client.1.smithi149.stdout:4/449: rename d2/d24/d2e/d3f/f40 to d2/d18/d1e/d3c/d48/f92 0 2024-04-24T08:24:48.271 INFO:tasks.workunit.client.0.smithi090.stdout:6/168: write d8/dc/f11 [988531,111959] 0 2024-04-24T08:24:48.274 INFO:tasks.workunit.client.0.smithi090.stdout:1/142: dwrite d3/d5/f14 [0,4194304] 0 2024-04-24T08:24:48.276 INFO:tasks.workunit.client.1.smithi149.stdout:1/593: truncate d0/f25 66527 0 2024-04-24T08:24:48.277 INFO:tasks.workunit.client.0.smithi090.stdout:7/214: symlink db/d17/l4f 0 2024-04-24T08:24:48.278 INFO:tasks.workunit.client.1.smithi149.stdout:2/406: unlink d0/d6/c23 0 2024-04-24T08:24:48.280 INFO:tasks.workunit.client.1.smithi149.stdout:8/408: dread d1/de/f6d [0,4194304] 0 2024-04-24T08:24:48.280 INFO:tasks.workunit.client.1.smithi149.stdout:4/450: mknod d2/d10/d1a/c93 0 2024-04-24T08:24:48.281 INFO:tasks.workunit.client.1.smithi149.stdout:2/407: dread - d0/d6/db/d27/d64/f76 zero size 2024-04-24T08:24:48.284 INFO:tasks.workunit.client.1.smithi149.stdout:6/379: rename d0/d1a/d28/d59 to d0/d1a/d28/d31/d4c/d80 0 2024-04-24T08:24:48.284 INFO:tasks.workunit.client.0.smithi090.stdout:0/204: dwrite d1a/f30 [4194304,4194304] 0 2024-04-24T08:24:48.289 INFO:tasks.workunit.client.0.smithi090.stdout:1/143: rename d3/d5/f19 to d3/f2e 0 2024-04-24T08:24:48.291 INFO:tasks.workunit.client.0.smithi090.stdout:7/215: mkdir db/d17/d29/d38/d50 0 2024-04-24T08:24:48.302 INFO:tasks.workunit.client.0.smithi090.stdout:1/144: dread d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:48.305 INFO:tasks.workunit.client.0.smithi090.stdout:0/205: symlink d1a/d2a/d3c/l43 0 2024-04-24T08:24:48.311 INFO:tasks.workunit.client.0.smithi090.stdout:1/145: write d3/d5/f16 [931278,34923] 0 2024-04-24T08:24:48.311 INFO:tasks.workunit.client.0.smithi090.stdout:7/216: dwrite db/d17/d1b/f26 [0,4194304] 0 2024-04-24T08:24:48.312 INFO:tasks.workunit.client.1.smithi149.stdout:6/380: fsync d0/d1a/f2b 0 2024-04-24T08:24:48.312 INFO:tasks.workunit.client.1.smithi149.stdout:3/524: getdents d0/d5/d53/d34/d3a 0 2024-04-24T08:24:48.312 INFO:tasks.workunit.client.1.smithi149.stdout:4/451: write d2/d24/d2e/f5b [56393,44090] 0 2024-04-24T08:24:48.312 INFO:tasks.workunit.client.0.smithi090.stdout:6/169: dread d8/fa [4194304,4194304] 0 2024-04-24T08:24:48.315 INFO:tasks.workunit.client.1.smithi149.stdout:7/478: dwrite d9/dc/f17 [0,4194304] 0 2024-04-24T08:24:48.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/170: write d8/d1d/d29/f2a [2126156,51585] 0 2024-04-24T08:24:48.318 INFO:tasks.workunit.client.1.smithi149.stdout:6/381: stat d0/d1a/d28/d31/d4c/d4e/d62/d7a 0 2024-04-24T08:24:48.318 INFO:tasks.workunit.client.0.smithi090.stdout:6/171: readlink d8/dc/l17 0 2024-04-24T08:24:48.319 INFO:tasks.workunit.client.0.smithi090.stdout:7/217: rmdir db/d17/d39 39 2024-04-24T08:24:48.322 INFO:tasks.workunit.client.1.smithi149.stdout:7/479: chown d9/dc/d2e/d3c/c6d 5 1 2024-04-24T08:24:48.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/218: truncate db/d17/d29/d38/f44 154241 0 2024-04-24T08:24:48.333 INFO:tasks.workunit.client.0.smithi090.stdout:1/146: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:48.333 INFO:tasks.workunit.client.0.smithi090.stdout:6/172: chown d8/d1d/d1f/l25 911468169 1 2024-04-24T08:24:48.335 INFO:tasks.workunit.client.1.smithi149.stdout:8/409: dwrite d1/de/d18/d32/d65/d8b/f7c [0,4194304] 0 2024-04-24T08:24:48.335 INFO:tasks.workunit.client.0.smithi090.stdout:7/219: mknod db/d17/d29/d32/c51 0 2024-04-24T08:24:48.336 INFO:tasks.workunit.client.1.smithi149.stdout:3/525: dread d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:48.341 INFO:tasks.workunit.client.1.smithi149.stdout:9/519: dwrite d1/d2/f19 [0,4194304] 0 2024-04-24T08:24:48.344 INFO:tasks.workunit.client.0.smithi090.stdout:2/149: dread d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:48.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/173: dread d8/dc/d19/f27 [0,4194304] 0 2024-04-24T08:24:48.347 INFO:tasks.workunit.client.0.smithi090.stdout:0/206: dread f12 [0,4194304] 0 2024-04-24T08:24:48.348 INFO:tasks.workunit.client.0.smithi090.stdout:6/174: stat d8/ce 0 2024-04-24T08:24:48.358 INFO:tasks.workunit.client.1.smithi149.stdout:0/431: dwrite d2/d9/f13 [0,4194304] 0 2024-04-24T08:24:48.360 INFO:tasks.workunit.client.1.smithi149.stdout:3/526: dread d0/d5/d53/d34/d3a/d93/f77 [0,4194304] 0 2024-04-24T08:24:48.360 INFO:tasks.workunit.client.0.smithi090.stdout:1/147: unlink d3/c10 0 2024-04-24T08:24:48.364 INFO:tasks.workunit.client.1.smithi149.stdout:0/432: read d2/d9/df/d40/f70 [160014,6123] 0 2024-04-24T08:24:48.364 INFO:tasks.workunit.client.0.smithi090.stdout:0/207: symlink d1a/d2a/d35/l44 0 2024-04-24T08:24:48.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/208: fdatasync d1a/d2a/d35/f3e 0 2024-04-24T08:24:48.368 INFO:tasks.workunit.client.0.smithi090.stdout:6/175: truncate d8/dc/d19/f27 456946 0 2024-04-24T08:24:48.377 INFO:tasks.workunit.client.1.smithi149.stdout:2/408: dwrite d0/d6/db/d27/f31 [0,4194304] 0 2024-04-24T08:24:48.377 INFO:tasks.workunit.client.0.smithi090.stdout:0/209: dread f12 [0,4194304] 0 2024-04-24T08:24:48.379 INFO:tasks.workunit.client.0.smithi090.stdout:2/150: getdents d0 0 2024-04-24T08:24:48.384 INFO:tasks.workunit.client.1.smithi149.stdout:4/452: mkdir d2/d24/d2e/d3f/d94 0 2024-04-24T08:24:48.384 INFO:tasks.workunit.client.1.smithi149.stdout:1/594: dwrite d0/d5/db/d63/f6e [0,4194304] 0 2024-04-24T08:24:48.388 INFO:tasks.workunit.client.0.smithi090.stdout:2/151: symlink d0/d7/d24/l2d 0 2024-04-24T08:24:48.390 INFO:tasks.workunit.client.0.smithi090.stdout:2/152: chown d0/d14/f1f 4153012 1 2024-04-24T08:24:48.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/210: dread d1a/f30 [4194304,4194304] 0 2024-04-24T08:24:48.397 INFO:tasks.workunit.client.1.smithi149.stdout:7/480: creat d9/dc/d11/d26/f9b x:0 0 0 2024-04-24T08:24:48.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/410: unlink d1/de/d18/d32/d65/d8b/f6e 0 2024-04-24T08:24:48.401 INFO:tasks.workunit.client.1.smithi149.stdout:7/481: dread - d9/dc/d7b/f7d zero size 2024-04-24T08:24:48.403 INFO:tasks.workunit.client.0.smithi090.stdout:0/211: rename d1a/d2a/d35/l44 to d1a/l45 0 2024-04-24T08:24:48.408 INFO:tasks.workunit.client.0.smithi090.stdout:6/176: read d8/dc/d19/f27 [252040,48409] 0 2024-04-24T08:24:48.408 INFO:tasks.workunit.client.1.smithi149.stdout:9/520: chown d1/da/dd/d55/c7d 410148674 1 2024-04-24T08:24:48.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/153: dwrite d0/d14/d17/f18 [0,4194304] 0 2024-04-24T08:24:48.409 INFO:tasks.workunit.client.1.smithi149.stdout:0/433: rmdir d2/d9/df 39 2024-04-24T08:24:48.412 INFO:tasks.workunit.client.0.smithi090.stdout:2/154: read d0/d14/f15 [703245,96397] 0 2024-04-24T08:24:48.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/148: dwrite d3/d5/f17 [0,4194304] 0 2024-04-24T08:24:48.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/91: sync 2024-04-24T08:24:48.422 INFO:tasks.workunit.client.0.smithi090.stdout:3/105: sync 2024-04-24T08:24:48.423 INFO:tasks.workunit.client.0.smithi090.stdout:9/115: sync 2024-04-24T08:24:48.427 INFO:tasks.workunit.client.1.smithi149.stdout:2/409: symlink d0/d6/db/d27/d64/l84 0 2024-04-24T08:24:48.427 INFO:tasks.workunit.client.1.smithi149.stdout:8/411: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:24:48.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/116: read - d0/dc/f1f zero size 2024-04-24T08:24:48.428 INFO:tasks.workunit.client.1.smithi149.stdout:4/453: readlink d2/d18/d1e/d3c/d48/l83 0 2024-04-24T08:24:48.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/410: readlink d0/d6/l11 0 2024-04-24T08:24:48.430 INFO:tasks.workunit.client.0.smithi090.stdout:9/117: readlink d0/l9 0 2024-04-24T08:24:48.431 INFO:tasks.workunit.client.0.smithi090.stdout:9/118: readlink d0/l9 0 2024-04-24T08:24:48.431 INFO:tasks.workunit.client.0.smithi090.stdout:6/177: dwrite d8/dc/f11 [0,4194304] 0 2024-04-24T08:24:48.432 INFO:tasks.workunit.client.1.smithi149.stdout:2/411: readlink d0/d6/db/d39/l62 0 2024-04-24T08:24:48.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/412: dread - d1/d1f/d5e/f87 zero size 2024-04-24T08:24:48.433 INFO:tasks.workunit.client.1.smithi149.stdout:9/521: dwrite d1/d2/fa0 [4194304,4194304] 0 2024-04-24T08:24:48.433 INFO:tasks.workunit.client.1.smithi149.stdout:7/482: dread d9/dc/de/d44/f74 [0,4194304] 0 2024-04-24T08:24:48.434 INFO:tasks.workunit.client.0.smithi090.stdout:2/155: dwrite d0/d7/d24/f29 [0,4194304] 0 2024-04-24T08:24:48.436 INFO:tasks.workunit.client.0.smithi090.stdout:2/156: fsync d0/d7/d24/f27 0 2024-04-24T08:24:48.437 INFO:tasks.workunit.client.1.smithi149.stdout:7/483: dread - d9/dc/de/d44/f78 zero size 2024-04-24T08:24:48.440 INFO:tasks.workunit.client.1.smithi149.stdout:7/484: write d9/dc/de/f3a [268367,38161] 0 2024-04-24T08:24:48.441 INFO:tasks.workunit.client.0.smithi090.stdout:1/149: mknod d3/d5/da/c2f 0 2024-04-24T08:24:48.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/106: creat d2/d15/f1d x:0 0 0 2024-04-24T08:24:48.449 INFO:tasks.workunit.client.0.smithi090.stdout:1/150: dread d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:48.455 INFO:tasks.workunit.client.0.smithi090.stdout:6/178: fdatasync d8/f16 0 2024-04-24T08:24:48.456 INFO:tasks.workunit.client.1.smithi149.stdout:3/527: creat d0/d5/da/d19/d1b/d72/fbb x:0 0 0 2024-04-24T08:24:48.457 INFO:tasks.workunit.client.0.smithi090.stdout:9/119: mkdir d0/d19/d20 0 2024-04-24T08:24:48.458 INFO:tasks.workunit.client.0.smithi090.stdout:9/120: stat d0/dc/c11 0 2024-04-24T08:24:48.460 INFO:tasks.workunit.client.1.smithi149.stdout:4/454: symlink d2/d18/d1e/l95 0 2024-04-24T08:24:48.461 INFO:tasks.workunit.client.0.smithi090.stdout:2/157: mkdir d0/d2e 0 2024-04-24T08:24:48.466 INFO:tasks.workunit.client.0.smithi090.stdout:1/151: dwrite d3/d5/f17 [0,4194304] 0 2024-04-24T08:24:48.468 INFO:tasks.workunit.client.1.smithi149.stdout:8/413: creat d1/d1f/d5e/f8c x:0 0 0 2024-04-24T08:24:48.471 INFO:tasks.workunit.client.1.smithi149.stdout:2/412: dread - d0/d6/db/d39/d3d/d44/d57/f71 zero size 2024-04-24T08:24:48.472 INFO:tasks.workunit.client.1.smithi149.stdout:6/382: rmdir d0/d1a/d28/d4a/d7c 0 2024-04-24T08:24:48.479 INFO:tasks.workunit.client.0.smithi090.stdout:9/121: mkdir d0/d19/d21 0 2024-04-24T08:24:48.480 INFO:tasks.workunit.client.0.smithi090.stdout:9/122: fsync d0/d19/f1c 0 2024-04-24T08:24:48.485 INFO:tasks.workunit.client.1.smithi149.stdout:4/455: rename d2/d24/d2e/d38/c5a to d2/d18/d1e/c96 0 2024-04-24T08:24:48.488 INFO:tasks.workunit.client.1.smithi149.stdout:2/413: chown d0/d6/db/d39/l63 0 1 2024-04-24T08:24:48.490 INFO:tasks.workunit.client.0.smithi090.stdout:6/179: mkdir d8/d1d/d2f/d30 0 2024-04-24T08:24:48.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/123: mknod d0/dc/c22 0 2024-04-24T08:24:48.493 INFO:tasks.workunit.client.1.smithi149.stdout:0/434: dread d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:48.493 INFO:tasks.workunit.client.1.smithi149.stdout:9/522: symlink d1/d2/d33/d60/lb5 0 2024-04-24T08:24:48.495 INFO:tasks.workunit.client.0.smithi090.stdout:0/212: rename d1a/l34 to d1a/d2a/d3c/l46 0 2024-04-24T08:24:48.496 INFO:tasks.workunit.client.0.smithi090.stdout:1/152: mkdir d3/d25/d30 0 2024-04-24T08:24:48.498 INFO:tasks.workunit.client.0.smithi090.stdout:3/107: creat d2/f1e x:0 0 0 2024-04-24T08:24:48.500 INFO:tasks.workunit.client.1.smithi149.stdout:6/383: unlink d0/d1a/d28/d31/d4c/l6e 0 2024-04-24T08:24:48.503 INFO:tasks.workunit.client.0.smithi090.stdout:4/92: getdents d3 0 2024-04-24T08:24:48.505 INFO:tasks.workunit.client.1.smithi149.stdout:0/435: symlink d2/d9/df/d40/l86 0 2024-04-24T08:24:48.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/180: symlink d8/d1d/d2f/l31 0 2024-04-24T08:24:48.507 INFO:tasks.workunit.client.1.smithi149.stdout:8/414: creat d1/d1f/f8d x:0 0 0 2024-04-24T08:24:48.508 INFO:tasks.workunit.client.1.smithi149.stdout:3/528: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:48.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/153: dwrite d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:48.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/456: unlink d2/d18/d1e/d3c/d48/f92 0 2024-04-24T08:24:48.512 INFO:tasks.workunit.client.1.smithi149.stdout:9/523: mkdir d1/da/dd/d79/db6 0 2024-04-24T08:24:48.513 INFO:tasks.workunit.client.0.smithi090.stdout:0/213: chown c11 1560 1 2024-04-24T08:24:48.513 INFO:tasks.workunit.client.1.smithi149.stdout:7/485: truncate d9/dc/d2e/f6f 1815267 0 2024-04-24T08:24:48.515 INFO:tasks.workunit.client.1.smithi149.stdout:7/486: write d9/d43/f7e [13112888,71988] 0 2024-04-24T08:24:48.519 INFO:tasks.workunit.client.1.smithi149.stdout:3/529: chown d0/d26/l92 756607131 1 2024-04-24T08:24:48.521 INFO:tasks.workunit.client.0.smithi090.stdout:9/124: symlink d0/d19/d20/l23 0 2024-04-24T08:24:48.523 INFO:tasks.workunit.client.0.smithi090.stdout:1/154: mknod d3/d5/c31 0 2024-04-24T08:24:48.524 INFO:tasks.workunit.client.0.smithi090.stdout:9/125: dread - d0/d19/f1a zero size 2024-04-24T08:24:48.524 INFO:tasks.workunit.client.1.smithi149.stdout:4/457: mknod d2/d24/d2e/d38/c97 0 2024-04-24T08:24:48.525 INFO:tasks.workunit.client.0.smithi090.stdout:9/126: fdatasync d0/fb 0 2024-04-24T08:24:48.531 INFO:tasks.workunit.client.0.smithi090.stdout:2/158: fdatasync d0/d14/f15 0 2024-04-24T08:24:48.531 INFO:tasks.workunit.client.0.smithi090.stdout:6/181: mknod d8/d1d/d2f/d30/c32 0 2024-04-24T08:24:48.536 INFO:tasks.workunit.client.1.smithi149.stdout:3/530: creat d0/d5/da/d19/d1b/d22/fbc x:0 0 0 2024-04-24T08:24:48.537 INFO:tasks.workunit.client.1.smithi149.stdout:0/436: write d2/f7f [3559710,91942] 0 2024-04-24T08:24:48.539 INFO:tasks.workunit.client.1.smithi149.stdout:4/458: fsync d2/d10/d1a/f6f 0 2024-04-24T08:24:48.539 INFO:tasks.workunit.client.1.smithi149.stdout:3/531: stat d0/d5/da/d19/d1b/d22/cb8 0 2024-04-24T08:24:48.541 INFO:tasks.workunit.client.1.smithi149.stdout:3/532: write d0/d5/d53/f3f [1408121,114696] 0 2024-04-24T08:24:48.542 INFO:tasks.workunit.client.1.smithi149.stdout:7/487: creat d9/d43/d95/f9c x:0 0 0 2024-04-24T08:24:48.544 INFO:tasks.workunit.client.1.smithi149.stdout:7/488: write d9/dc/de/f16 [6403439,127761] 0 2024-04-24T08:24:48.545 INFO:tasks.workunit.client.1.smithi149.stdout:8/415: rename d1/d1f/d31/f79 to d1/f8e 0 2024-04-24T08:24:48.546 INFO:tasks.workunit.client.0.smithi090.stdout:3/108: dwrite d2/f9 [0,4194304] 0 2024-04-24T08:24:48.546 INFO:tasks.workunit.client.0.smithi090.stdout:9/127: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:48.548 INFO:tasks.workunit.client.1.smithi149.stdout:2/414: truncate d0/d6/db/d27/f37 1565704 0 2024-04-24T08:24:48.548 INFO:tasks.workunit.client.0.smithi090.stdout:6/182: mknod d8/d1d/d1f/c33 0 2024-04-24T08:24:48.551 INFO:tasks.workunit.client.1.smithi149.stdout:2/415: write d0/d6/db/d39/d3d/f51 [1334271,24060] 0 2024-04-24T08:24:48.553 INFO:tasks.workunit.client.1.smithi149.stdout:0/437: symlink d2/d17/d41/d68/l87 0 2024-04-24T08:24:48.554 INFO:tasks.workunit.client.0.smithi090.stdout:0/214: symlink d1a/d2a/d35/l47 0 2024-04-24T08:24:48.555 INFO:tasks.workunit.client.0.smithi090.stdout:2/159: rmdir d0/d7 39 2024-04-24T08:24:48.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/109: dread d2/df/f12 [0,4194304] 0 2024-04-24T08:24:48.560 INFO:tasks.workunit.client.0.smithi090.stdout:3/110: chown d2/lc 452 1 2024-04-24T08:24:48.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/128: rename d0/d19/d21 to d0/dc/d24 0 2024-04-24T08:24:48.562 INFO:tasks.workunit.client.0.smithi090.stdout:9/129: chown d0/c16 74336511 1 2024-04-24T08:24:48.563 INFO:tasks.workunit.client.0.smithi090.stdout:9/130: read - d0/d19/f1c zero size 2024-04-24T08:24:48.566 INFO:tasks.workunit.client.0.smithi090.stdout:1/155: truncate d3/d5/f16 1020161 0 2024-04-24T08:24:48.567 INFO:tasks.workunit.client.0.smithi090.stdout:6/183: creat d8/dc/d19/f34 x:0 0 0 2024-04-24T08:24:48.569 INFO:tasks.workunit.client.0.smithi090.stdout:1/156: write d3/d5/f1a [4328448,26452] 0 2024-04-24T08:24:48.572 INFO:tasks.workunit.client.1.smithi149.stdout:9/524: rename d1/da/dd/d8d to d1/da/d4a/dad/db7 0 2024-04-24T08:24:48.577 INFO:tasks.workunit.client.0.smithi090.stdout:3/111: rmdir d2 39 2024-04-24T08:24:48.582 INFO:tasks.workunit.client.0.smithi090.stdout:9/131: creat d0/d19/f25 x:0 0 0 2024-04-24T08:24:48.584 INFO:tasks.workunit.client.0.smithi090.stdout:9/132: chown d0/f18 63336 1 2024-04-24T08:24:48.584 INFO:tasks.workunit.client.0.smithi090.stdout:6/184: dwrite f6 [0,4194304] 0 2024-04-24T08:24:48.585 INFO:tasks.workunit.client.0.smithi090.stdout:9/133: read - d0/dc/f1f zero size 2024-04-24T08:24:48.585 INFO:tasks.workunit.client.0.smithi090.stdout:1/157: dwrite d3/d5/da/dc/f2b [0,4194304] 0 2024-04-24T08:24:48.586 INFO:tasks.workunit.client.1.smithi149.stdout:3/533: dread d0/d5/da/d19/d1b/d22/d38/f50 [4194304,4194304] 0 2024-04-24T08:24:48.589 INFO:tasks.workunit.client.0.smithi090.stdout:8/111: sync 2024-04-24T08:24:48.589 INFO:tasks.workunit.client.0.smithi090.stdout:5/221: sync 2024-04-24T08:24:48.593 INFO:tasks.workunit.client.0.smithi090.stdout:3/112: dread - d2/f17 zero size 2024-04-24T08:24:48.595 INFO:tasks.workunit.client.1.smithi149.stdout:9/525: symlink d1/da/dd/d55/lb8 0 2024-04-24T08:24:48.597 INFO:tasks.workunit.client.1.smithi149.stdout:3/534: mknod d0/d5/d53/d4c/d5f/cbd 0 2024-04-24T08:24:48.599 INFO:tasks.workunit.client.0.smithi090.stdout:9/134: mknod d0/dc/c26 0 2024-04-24T08:24:48.602 INFO:tasks.workunit.client.1.smithi149.stdout:4/459: rename d2/d18/d1e/d3b/f60 to d2/d24/d2e/f98 0 2024-04-24T08:24:48.604 INFO:tasks.workunit.client.0.smithi090.stdout:1/158: creat d3/d5/f32 x:0 0 0 2024-04-24T08:24:48.605 INFO:tasks.workunit.client.0.smithi090.stdout:2/160: dread d0/d14/d17/f18 [4194304,4194304] 0 2024-04-24T08:24:48.606 INFO:tasks.workunit.client.1.smithi149.stdout:4/460: read d2/f44 [814632,92180] 0 2024-04-24T08:24:48.607 INFO:tasks.workunit.client.1.smithi149.stdout:8/416: getdents d1/de/d18/d32/d63 0 2024-04-24T08:24:48.608 INFO:tasks.workunit.client.0.smithi090.stdout:0/215: truncate f12 2214601 0 2024-04-24T08:24:48.609 INFO:tasks.workunit.client.1.smithi149.stdout:3/535: creat d0/d88/fbe x:0 0 0 2024-04-24T08:24:48.611 INFO:tasks.workunit.client.0.smithi090.stdout:2/161: truncate d0/d14/f1f 4761307 0 2024-04-24T08:24:48.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/112: rename d2/da/d13/d15/f1e to d2/da/d13/f28 0 2024-04-24T08:24:48.612 INFO:tasks.workunit.client.0.smithi090.stdout:3/113: dwrite d2/df/f19 [0,4194304] 0 2024-04-24T08:24:48.614 INFO:tasks.workunit.client.1.smithi149.stdout:7/489: rename d9/d43 to d9/d43/d4f/d9d 22 2024-04-24T08:24:48.615 INFO:tasks.workunit.client.1.smithi149.stdout:7/490: readlink d9/d43/d4f/l85 0 2024-04-24T08:24:48.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/135: symlink d0/l27 0 2024-04-24T08:24:48.620 INFO:tasks.workunit.client.1.smithi149.stdout:9/526: creat d1/d4c/d91/fb9 x:0 0 0 2024-04-24T08:24:48.622 INFO:tasks.workunit.client.0.smithi090.stdout:1/159: creat d3/d5/f33 x:0 0 0 2024-04-24T08:24:48.623 INFO:tasks.workunit.client.1.smithi149.stdout:2/416: rename d0/d6/c29 to d0/d6/db/d28/c85 0 2024-04-24T08:24:48.624 INFO:tasks.workunit.client.1.smithi149.stdout:2/417: dread - d0/f77 zero size 2024-04-24T08:24:48.626 INFO:tasks.workunit.client.1.smithi149.stdout:2/418: fsync d0/f22 0 2024-04-24T08:24:48.627 INFO:tasks.workunit.client.1.smithi149.stdout:7/491: creat d9/d43/d4c/d67/f9e x:0 0 0 2024-04-24T08:24:48.627 INFO:tasks.workunit.client.0.smithi090.stdout:2/162: fdatasync d0/f21 0 2024-04-24T08:24:48.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/461: dwrite d2/d18/d1e/d3c/f43 [0,4194304] 0 2024-04-24T08:24:48.629 INFO:tasks.workunit.client.0.smithi090.stdout:5/222: rename d8/da/d3a/d2d/d37/f39 to d8/da/d14/d2f/d44/f49 0 2024-04-24T08:24:48.631 INFO:tasks.workunit.client.0.smithi090.stdout:8/113: symlink d2/l29 0 2024-04-24T08:24:48.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/160: dread f0 [4194304,4194304] 0 2024-04-24T08:24:48.633 INFO:tasks.workunit.client.1.smithi149.stdout:8/417: rename d1/de/f6d to d1/d1f/d25/f8f 0 2024-04-24T08:24:48.635 INFO:tasks.workunit.client.0.smithi090.stdout:3/114: mknod d2/df/d1a/c1f 0 2024-04-24T08:24:48.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/536: dwrite d0/d5/da/d19/faf [0,4194304] 0 2024-04-24T08:24:48.644 INFO:tasks.workunit.client.1.smithi149.stdout:7/492: symlink d9/dc/d11/d83/l9f 0 2024-04-24T08:24:48.645 INFO:tasks.workunit.client.1.smithi149.stdout:3/537: chown d0/d5/da/d19/l2e 14 1 2024-04-24T08:24:48.645 INFO:tasks.workunit.client.0.smithi090.stdout:7/220: sync 2024-04-24T08:24:48.647 INFO:tasks.workunit.client.1.smithi149.stdout:7/493: fdatasync d9/d43/d4f/d66/f6e 0 2024-04-24T08:24:48.649 INFO:tasks.workunit.client.0.smithi090.stdout:0/216: write d1a/f30 [3083538,64683] 0 2024-04-24T08:24:48.649 INFO:tasks.workunit.client.1.smithi149.stdout:2/419: dwrite d0/d6/db/d39/d3d/d44/f61 [0,4194304] 0 2024-04-24T08:24:48.650 INFO:tasks.workunit.client.0.smithi090.stdout:3/115: dread d2/f9 [0,4194304] 0 2024-04-24T08:24:48.650 INFO:tasks.workunit.client.0.smithi090.stdout:0/217: chown f18 199587237 1 2024-04-24T08:24:48.651 INFO:tasks.workunit.client.1.smithi149.stdout:7/494: fdatasync d9/dc/d11/d26/f28 0 2024-04-24T08:24:48.651 INFO:tasks.workunit.client.0.smithi090.stdout:3/116: stat d2/d15/f1d 0 2024-04-24T08:24:48.653 INFO:tasks.workunit.client.1.smithi149.stdout:8/418: mknod d1/d1f/d25/c90 0 2024-04-24T08:24:48.653 INFO:tasks.workunit.client.1.smithi149.stdout:7/495: chown d9/dc/de/d44/f91 12878680 1 2024-04-24T08:24:48.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/419: chown d1/de/d18/f6f 6 1 2024-04-24T08:24:48.655 INFO:tasks.workunit.client.1.smithi149.stdout:4/462: mknod d2/c99 0 2024-04-24T08:24:48.657 INFO:tasks.workunit.client.1.smithi149.stdout:8/420: fdatasync d1/d1f/d5e/f8c 0 2024-04-24T08:24:48.659 INFO:tasks.workunit.client.1.smithi149.stdout:8/421: write d1/d1f/d31/f84 [160676,80698] 0 2024-04-24T08:24:48.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/223: symlink d8/da/d3a/d2d/d37/l4a 0 2024-04-24T08:24:48.663 INFO:tasks.workunit.client.0.smithi090.stdout:8/114: symlink d2/l2a 0 2024-04-24T08:24:48.665 INFO:tasks.workunit.client.1.smithi149.stdout:3/538: rename d0/d5/d53/d34/d3a/f36 to d0/d5/da/d59/fbf 0 2024-04-24T08:24:48.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/161: mkdir d3/d1d/d34 0 2024-04-24T08:24:48.669 INFO:tasks.workunit.client.1.smithi149.stdout:9/527: creat d1/d2/fba x:0 0 0 2024-04-24T08:24:48.673 INFO:tasks.workunit.client.0.smithi090.stdout:6/185: getdents d8 0 2024-04-24T08:24:48.673 INFO:tasks.workunit.client.0.smithi090.stdout:9/136: write d0/f18 [895844,4787] 0 2024-04-24T08:24:48.677 INFO:tasks.workunit.client.0.smithi090.stdout:7/221: mknod db/d17/d29/d38/c52 0 2024-04-24T08:24:48.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/539: rmdir d0/d5/d53 39 2024-04-24T08:24:48.680 INFO:tasks.workunit.client.0.smithi090.stdout:3/117: mknod d2/df/d1a/c20 0 2024-04-24T08:24:48.683 INFO:tasks.workunit.client.0.smithi090.stdout:2/163: symlink d0/d7/l2f 0 2024-04-24T08:24:48.684 INFO:tasks.workunit.client.1.smithi149.stdout:2/420: symlink d0/d6/db/d39/l86 0 2024-04-24T08:24:48.686 INFO:tasks.workunit.client.1.smithi149.stdout:7/496: symlink d9/d43/d4f/d66/d96/la0 0 2024-04-24T08:24:48.686 INFO:tasks.workunit.client.1.smithi149.stdout:2/421: stat d0/d6/db/d39/d3d/d44/d75 0 2024-04-24T08:24:48.688 INFO:tasks.workunit.client.0.smithi090.stdout:8/115: chown d2/da/d13/d15/l1a 1909445 1 2024-04-24T08:24:48.689 INFO:tasks.workunit.client.1.smithi149.stdout:2/422: chown d0/d6/db/d39/d3d/d44/d75/f79 897268615 1 2024-04-24T08:24:48.690 INFO:tasks.workunit.client.1.smithi149.stdout:8/422: mkdir d1/de/d18/d91 0 2024-04-24T08:24:48.691 INFO:tasks.workunit.client.0.smithi090.stdout:8/116: write d2/da/d13/d15/f26 [3936496,68076] 0 2024-04-24T08:24:48.692 INFO:tasks.workunit.client.0.smithi090.stdout:1/162: creat d3/d5/da/dc/f35 x:0 0 0 2024-04-24T08:24:48.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/218: dwrite d1a/f1d [0,4194304] 0 2024-04-24T08:24:48.693 INFO:tasks.workunit.client.1.smithi149.stdout:2/423: read d0/f8 [358168,102852] 0 2024-04-24T08:24:48.694 INFO:tasks.workunit.client.0.smithi090.stdout:1/163: truncate d3/d5/f28 569830 0 2024-04-24T08:24:48.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/164: chown f0 97584 1 2024-04-24T08:24:48.695 INFO:tasks.workunit.client.1.smithi149.stdout:4/463: dread d2/d24/d2e/d38/f5e [0,4194304] 0 2024-04-24T08:24:48.696 INFO:tasks.workunit.client.0.smithi090.stdout:1/165: stat d3/d5/l6 0 2024-04-24T08:24:48.696 INFO:tasks.workunit.client.1.smithi149.stdout:4/464: stat d2/l7e 0 2024-04-24T08:24:48.698 INFO:tasks.workunit.client.1.smithi149.stdout:4/465: write d2/d24/d2e/d38/f7a [548505,29922] 0 2024-04-24T08:24:48.698 INFO:tasks.workunit.client.0.smithi090.stdout:6/186: symlink d8/l35 0 2024-04-24T08:24:48.701 INFO:tasks.workunit.client.1.smithi149.stdout:4/466: write d2/d18/d1e/d3c/f84 [586418,82856] 0 2024-04-24T08:24:48.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/222: creat db/d17/d29/d38/f53 x:0 0 0 2024-04-24T08:24:48.710 INFO:tasks.workunit.client.0.smithi090.stdout:3/118: mkdir d2/df/d21 0 2024-04-24T08:24:48.711 INFO:tasks.workunit.client.0.smithi090.stdout:2/164: mknod d0/d7/d24/c30 0 2024-04-24T08:24:48.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/576: sync 2024-04-24T08:24:48.715 INFO:tasks.workunit.client.1.smithi149.stdout:9/528: dwrite d1/d2/d15/d1c/d42/f4b [0,4194304] 0 2024-04-24T08:24:48.718 INFO:tasks.workunit.client.1.smithi149.stdout:2/424: creat d0/d6/db/d28/d7f/f87 x:0 0 0 2024-04-24T08:24:48.719 INFO:tasks.workunit.client.0.smithi090.stdout:0/219: dread f15 [0,4194304] 0 2024-04-24T08:24:48.720 INFO:tasks.workunit.client.0.smithi090.stdout:9/137: getdents d0/dc/d24 0 2024-04-24T08:24:48.724 INFO:tasks.workunit.client.1.smithi149.stdout:7/497: write d9/dc/d2e/d51/f59 [2947287,408] 0 2024-04-24T08:24:48.725 INFO:tasks.workunit.client.0.smithi090.stdout:7/223: mkdir db/d54 0 2024-04-24T08:24:48.726 INFO:tasks.workunit.client.0.smithi090.stdout:4/93: sync 2024-04-24T08:24:48.727 INFO:tasks.workunit.client.1.smithi149.stdout:4/467: dread d2/d10/f27 [0,4194304] 0 2024-04-24T08:24:48.727 INFO:tasks.workunit.client.1.smithi149.stdout:5/577: chown db/c3f 139467 1 2024-04-24T08:24:48.729 INFO:tasks.workunit.client.1.smithi149.stdout:2/425: rename d0/d6/l11 to d0/d6/l88 0 2024-04-24T08:24:48.731 INFO:tasks.workunit.client.0.smithi090.stdout:5/224: dwrite f2 [0,4194304] 0 2024-04-24T08:24:48.733 INFO:tasks.workunit.client.0.smithi090.stdout:9/138: dread d0/f15 [0,4194304] 0 2024-04-24T08:24:48.733 INFO:tasks.workunit.client.0.smithi090.stdout:5/225: read - d8/da/d18/d21/f41 zero size 2024-04-24T08:24:48.734 INFO:tasks.workunit.client.0.smithi090.stdout:1/166: symlink d3/d25/d30/l36 0 2024-04-24T08:24:48.735 INFO:tasks.workunit.client.1.smithi149.stdout:5/578: fsync db/d12/f84 0 2024-04-24T08:24:48.735 INFO:tasks.workunit.client.0.smithi090.stdout:0/220: mknod d1a/d2a/d35/c48 0 2024-04-24T08:24:48.736 INFO:tasks.workunit.client.1.smithi149.stdout:8/423: dwrite d1/de/f47 [0,4194304] 0 2024-04-24T08:24:48.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/139: chown d0/l9 271204 1 2024-04-24T08:24:48.737 INFO:tasks.workunit.client.1.smithi149.stdout:2/426: chown d0/d6/db/d27/f5f 0 1 2024-04-24T08:24:48.739 INFO:tasks.workunit.client.1.smithi149.stdout:3/540: dwrite d0/fc [0,4194304] 0 2024-04-24T08:24:48.740 INFO:tasks.workunit.client.0.smithi090.stdout:4/94: unlink d3/f10 0 2024-04-24T08:24:48.741 INFO:tasks.workunit.client.0.smithi090.stdout:2/165: mkdir d0/d14/d17/d2a/d31 0 2024-04-24T08:24:48.744 INFO:tasks.workunit.client.1.smithi149.stdout:3/541: dread - d0/d5/d1a/d70/da3/fb0 zero size 2024-04-24T08:24:48.745 INFO:tasks.workunit.client.0.smithi090.stdout:8/117: link d2/l2a d2/da/l2b 0 2024-04-24T08:24:48.747 INFO:tasks.workunit.client.1.smithi149.stdout:4/468: unlink d2/d18/d1e/d3c/c90 0 2024-04-24T08:24:48.752 INFO:tasks.workunit.client.1.smithi149.stdout:7/498: creat d9/dc/d72/fa1 x:0 0 0 2024-04-24T08:24:48.753 INFO:tasks.workunit.client.1.smithi149.stdout:8/424: fsync d1/d1f/f52 0 2024-04-24T08:24:48.758 INFO:tasks.workunit.client.1.smithi149.stdout:2/427: fdatasync d0/d6/db/d27/f3b 0 2024-04-24T08:24:48.758 INFO:tasks.workunit.client.0.smithi090.stdout:6/187: rename d8/dc/lf to d8/d1d/l36 0 2024-04-24T08:24:48.760 INFO:tasks.workunit.client.0.smithi090.stdout:3/119: truncate d2/df/f10 4169373 0 2024-04-24T08:24:48.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/167: creat d3/d1d/f37 x:0 0 0 2024-04-24T08:24:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:5/579: creat db/d1f/fc2 x:0 0 0 2024-04-24T08:24:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:2/428: write d0/d6/db/d27/f3b [240174,20439] 0 2024-04-24T08:24:48.763 INFO:tasks.workunit.client.0.smithi090.stdout:0/221: creat d1a/d2a/d35/f49 x:0 0 0 2024-04-24T08:24:48.763 INFO:tasks.workunit.client.1.smithi149.stdout:9/529: dwrite d1/da/f8 [0,4194304] 0 2024-04-24T08:24:48.765 INFO:tasks.workunit.client.0.smithi090.stdout:7/224: write db/f10 [1707146,8401] 0 2024-04-24T08:24:48.766 INFO:tasks.workunit.client.0.smithi090.stdout:5/226: creat d8/da/d18/d21/f4b x:0 0 0 2024-04-24T08:24:48.767 INFO:tasks.workunit.client.0.smithi090.stdout:6/188: write f0 [4235387,98682] 0 2024-04-24T08:24:48.767 INFO:tasks.workunit.client.0.smithi090.stdout:4/95: dwrite d3/f16 [0,4194304] 0 2024-04-24T08:24:48.768 INFO:tasks.workunit.client.1.smithi149.stdout:4/469: rename d2/d24/d2e/c57 to d2/d24/c9a 0 2024-04-24T08:24:48.772 INFO:tasks.workunit.client.1.smithi149.stdout:7/499: creat d9/dc/d7b/fa2 x:0 0 0 2024-04-24T08:24:48.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/580: dread db/d12/d60/d8f/da0/fb6 [0,4194304] 0 2024-04-24T08:24:48.775 INFO:tasks.workunit.client.0.smithi090.stdout:9/140: rename d0/fa to d0/d19/f28 0 2024-04-24T08:24:48.776 INFO:tasks.workunit.client.1.smithi149.stdout:5/581: write db/d12/d1e/fc1 [752796,34385] 0 2024-04-24T08:24:48.776 INFO:tasks.workunit.client.0.smithi090.stdout:8/118: dwrite d2/da/d13/d15/d24/f21 [0,4194304] 0 2024-04-24T08:24:48.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/166: dread d0/d14/d17/f18 [4194304,4194304] 0 2024-04-24T08:24:48.783 INFO:tasks.workunit.client.1.smithi149.stdout:8/425: mkdir d1/d1f/d5e/d92 0 2024-04-24T08:24:48.783 INFO:tasks.workunit.client.0.smithi090.stdout:2/167: read d0/d7/f11 [10352830,42108] 0 2024-04-24T08:24:48.784 INFO:tasks.workunit.client.0.smithi090.stdout:3/120: unlink d2/f1e 0 2024-04-24T08:24:48.785 INFO:tasks.workunit.client.0.smithi090.stdout:2/168: fsync d0/d14/d17/f1b 0 2024-04-24T08:24:48.785 INFO:tasks.workunit.client.1.smithi149.stdout:8/426: write d1/d1f/f8d [432548,25288] 0 2024-04-24T08:24:48.790 INFO:tasks.workunit.client.0.smithi090.stdout:1/168: mknod d3/c38 0 2024-04-24T08:24:48.791 INFO:tasks.workunit.client.1.smithi149.stdout:8/427: write d1/d1f/f8d [162030,103287] 0 2024-04-24T08:24:48.792 INFO:tasks.workunit.client.1.smithi149.stdout:3/542: mknod d0/d5/da/d19/d80/dac/cc0 0 2024-04-24T08:24:48.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/222: rmdir d1a/d2a/d3c 39 2024-04-24T08:24:48.796 INFO:tasks.workunit.client.0.smithi090.stdout:0/223: fsync ff 0 2024-04-24T08:24:48.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/429: unlink d0/d6/db/d27/d64/d66/f7a 0 2024-04-24T08:24:48.797 INFO:tasks.workunit.client.1.smithi149.stdout:4/470: dwrite d2/d18/f64 [0,4194304] 0 2024-04-24T08:24:48.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/189: fdatasync d8/dc/d19/f27 0 2024-04-24T08:24:48.800 INFO:tasks.workunit.client.0.smithi090.stdout:7/225: creat db/d17/d29/f55 x:0 0 0 2024-04-24T08:24:48.801 INFO:tasks.workunit.client.0.smithi090.stdout:3/121: dwrite d2/d15/f16 [0,4194304] 0 2024-04-24T08:24:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:7/500: symlink d9/d43/d4c/d67/la3 0 2024-04-24T08:24:48.802 INFO:tasks.workunit.client.1.smithi149.stdout:2/430: dread - d0/d6/db/d27/d64/f76 zero size 2024-04-24T08:24:48.802 INFO:tasks.workunit.client.0.smithi090.stdout:5/227: mkdir d8/da/d18/d21/d4c 0 2024-04-24T08:24:48.803 INFO:tasks.workunit.client.0.smithi090.stdout:5/228: chown d8/da/f20 1226634 1 2024-04-24T08:24:48.806 INFO:tasks.workunit.client.1.smithi149.stdout:5/582: creat db/d12/d2f/d95/fc3 x:0 0 0 2024-04-24T08:24:48.807 INFO:tasks.workunit.client.1.smithi149.stdout:5/583: fsync db/d12/d60/f67 0 2024-04-24T08:24:48.807 INFO:tasks.workunit.client.1.smithi149.stdout:2/431: dread - d0/d6/db/d39/d78/d7c/f82 zero size 2024-04-24T08:24:48.815 INFO:tasks.workunit.client.0.smithi090.stdout:8/119: fdatasync d2/f10 0 2024-04-24T08:24:48.816 INFO:tasks.workunit.client.0.smithi090.stdout:8/120: fsync d2/da/d13/f18 0 2024-04-24T08:24:48.820 INFO:tasks.workunit.client.0.smithi090.stdout:4/96: symlink d3/df/l17 0 2024-04-24T08:24:48.822 INFO:tasks.workunit.client.0.smithi090.stdout:4/97: write d3/f15 [299000,81771] 0 2024-04-24T08:24:48.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/98: truncate d3/f15 1377012 0 2024-04-24T08:24:48.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/99: chown d3/f16 789101922 1 2024-04-24T08:24:48.825 INFO:tasks.workunit.client.0.smithi090.stdout:4/100: read d3/f9 [2701017,96751] 0 2024-04-24T08:24:48.826 INFO:tasks.workunit.client.1.smithi149.stdout:8/428: unlink d1/de/d18/l49 0 2024-04-24T08:24:48.826 INFO:tasks.workunit.client.0.smithi090.stdout:4/101: truncate d3/f15 1792613 0 2024-04-24T08:24:48.827 INFO:tasks.workunit.client.1.smithi149.stdout:9/530: write d1/d2/d33/d76/f7c [87160,28580] 0 2024-04-24T08:24:48.830 INFO:tasks.workunit.client.0.smithi090.stdout:2/169: creat d0/d7/f32 x:0 0 0 2024-04-24T08:24:48.831 INFO:tasks.workunit.client.1.smithi149.stdout:7/501: rmdir d9/d43/d4f/d66/d96 39 2024-04-24T08:24:48.832 INFO:tasks.workunit.client.1.smithi149.stdout:7/502: stat d9/dc/d11/c55 0 2024-04-24T08:24:48.835 INFO:tasks.workunit.client.1.smithi149.stdout:4/471: rename d2/d24/d2e/d3f/f67 to d2/d24/f9b 0 2024-04-24T08:24:48.837 INFO:tasks.workunit.client.0.smithi090.stdout:1/169: creat d3/d12/f39 x:0 0 0 2024-04-24T08:24:48.843 INFO:tasks.workunit.client.1.smithi149.stdout:9/531: mkdir d1/da/d13/d74/dbb 0 2024-04-24T08:24:48.844 INFO:tasks.workunit.client.0.smithi090.stdout:6/190: rmdir d8/d1d 39 2024-04-24T08:24:48.845 INFO:tasks.workunit.client.1.smithi149.stdout:7/503: symlink d9/dc/d72/la4 0 2024-04-24T08:24:48.846 INFO:tasks.workunit.client.0.smithi090.stdout:9/141: dwrite d0/f5 [0,4194304] 0 2024-04-24T08:24:48.847 INFO:tasks.workunit.client.0.smithi090.stdout:9/142: chown d0/l9 28413 1 2024-04-24T08:24:48.847 INFO:tasks.workunit.client.1.smithi149.stdout:1/595: sync 2024-04-24T08:24:48.848 INFO:tasks.workunit.client.0.smithi090.stdout:9/143: readlink d0/l27 0 2024-04-24T08:24:48.848 INFO:tasks.workunit.client.1.smithi149.stdout:1/596: chown d0/d5/db 6466242 1 2024-04-24T08:24:48.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/122: creat d2/df/f22 x:0 0 0 2024-04-24T08:24:48.849 INFO:tasks.workunit.client.1.smithi149.stdout:8/429: rename d1/de/d18/l19 to d1/d1f/d31/l93 0 2024-04-24T08:24:48.851 INFO:tasks.workunit.client.1.smithi149.stdout:2/432: dread d0/f2 [4194304,4194304] 0 2024-04-24T08:24:48.853 INFO:tasks.workunit.client.1.smithi149.stdout:2/433: chown d0/f77 128529 1 2024-04-24T08:24:48.855 INFO:tasks.workunit.client.1.smithi149.stdout:4/472: unlink d2/c99 0 2024-04-24T08:24:48.857 INFO:tasks.workunit.client.0.smithi090.stdout:8/121: creat d2/f2c x:0 0 0 2024-04-24T08:24:48.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/102: mkdir d3/d18 0 2024-04-24T08:24:48.861 INFO:tasks.workunit.client.1.smithi149.stdout:9/532: dwrite d1/d2/d33/d76/f7c [0,4194304] 0 2024-04-24T08:24:48.863 INFO:tasks.workunit.client.0.smithi090.stdout:2/170: creat d0/d7/f33 x:0 0 0 2024-04-24T08:24:48.864 INFO:tasks.workunit.client.1.smithi149.stdout:3/543: write d0/d5/d73/fad [144604,74249] 0 2024-04-24T08:24:48.866 INFO:tasks.workunit.client.0.smithi090.stdout:2/171: write d0/d7/f32 [250307,71578] 0 2024-04-24T08:24:48.868 INFO:tasks.workunit.client.1.smithi149.stdout:1/597: symlink d0/d5/db/d63/d98/lba 0 2024-04-24T08:24:48.870 INFO:tasks.workunit.client.0.smithi090.stdout:2/172: read d0/d7/f32 [96218,124309] 0 2024-04-24T08:24:48.873 INFO:tasks.workunit.client.0.smithi090.stdout:9/144: creat d0/dc/f29 x:0 0 0 2024-04-24T08:24:48.875 INFO:tasks.workunit.client.0.smithi090.stdout:9/145: dread - d0/dc/f29 zero size 2024-04-24T08:24:48.875 INFO:tasks.workunit.client.1.smithi149.stdout:4/473: write d2/d24/d2e/d38/f5e [560902,27715] 0 2024-04-24T08:24:48.878 INFO:tasks.workunit.client.1.smithi149.stdout:9/533: creat d1/d2/d15/fbc x:0 0 0 2024-04-24T08:24:48.880 INFO:tasks.workunit.client.1.smithi149.stdout:7/504: mkdir d9/d43/d4f/da5 0 2024-04-24T08:24:48.887 INFO:tasks.workunit.client.1.smithi149.stdout:1/598: fsync d0/d5/db/f1e 0 2024-04-24T08:24:48.887 INFO:tasks.workunit.client.0.smithi090.stdout:3/123: dread d2/df/f12 [0,4194304] 0 2024-04-24T08:24:48.892 INFO:tasks.workunit.client.1.smithi149.stdout:5/584: getdents db/d12/d60/d8f/da0 0 2024-04-24T08:24:48.893 INFO:tasks.workunit.client.1.smithi149.stdout:8/430: dread d1/de/f1a [0,4194304] 0 2024-04-24T08:24:48.894 INFO:tasks.workunit.client.1.smithi149.stdout:4/474: creat d2/d24/d2e/d38/f9c x:0 0 0 2024-04-24T08:24:48.894 INFO:tasks.workunit.client.1.smithi149.stdout:5/585: chown db/d12/d2f/f57 14 1 2024-04-24T08:24:48.895 INFO:tasks.workunit.client.0.smithi090.stdout:9/146: mkdir d0/d2a 0 2024-04-24T08:24:48.897 INFO:tasks.workunit.client.1.smithi149.stdout:9/534: mknod d1/d2/d33/d5d/cbd 0 2024-04-24T08:24:48.899 INFO:tasks.workunit.client.0.smithi090.stdout:7/226: rmdir db/d17/d29/d38/d50 0 2024-04-24T08:24:48.903 INFO:tasks.workunit.client.0.smithi090.stdout:3/124: rmdir d2/d15 39 2024-04-24T08:24:48.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/147: dwrite d0/d19/f1e [0,4194304] 0 2024-04-24T08:24:48.909 INFO:tasks.workunit.client.0.smithi090.stdout:6/191: symlink d8/d1d/l37 0 2024-04-24T08:24:48.909 INFO:tasks.workunit.client.1.smithi149.stdout:2/434: dread d0/d6/db/f10 [0,4194304] 0 2024-04-24T08:24:48.909 INFO:tasks.workunit.client.1.smithi149.stdout:8/431: mknod d1/d1f/d22/c94 0 2024-04-24T08:24:48.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/148: read d0/f5 [3427552,39843] 0 2024-04-24T08:24:48.910 INFO:tasks.workunit.client.0.smithi090.stdout:6/192: chown d8/d1d/f1e 207567163 1 2024-04-24T08:24:48.911 INFO:tasks.workunit.client.1.smithi149.stdout:8/432: chown d1/d1f/d22/l24 521699 1 2024-04-24T08:24:48.911 INFO:tasks.workunit.client.0.smithi090.stdout:7/227: creat db/f56 x:0 0 0 2024-04-24T08:24:48.912 INFO:tasks.workunit.client.1.smithi149.stdout:2/435: write d0/d6/db/d27/f5f [1526098,81342] 0 2024-04-24T08:24:48.912 INFO:tasks.workunit.client.1.smithi149.stdout:0/438: sync 2024-04-24T08:24:48.913 INFO:tasks.workunit.client.1.smithi149.stdout:6/384: sync 2024-04-24T08:24:48.914 INFO:tasks.workunit.client.0.smithi090.stdout:4/103: dwrite d3/f9 [0,4194304] 0 2024-04-24T08:24:48.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/224: link d1a/c31 d1a/d2a/d3c/c4a 0 2024-04-24T08:24:48.916 INFO:tasks.workunit.client.1.smithi149.stdout:8/433: read d1/de/d18/d32/d65/d8b/f7c [1125681,76671] 0 2024-04-24T08:24:48.917 INFO:tasks.workunit.client.0.smithi090.stdout:4/104: truncate d3/f15 2812329 0 2024-04-24T08:24:48.923 INFO:tasks.workunit.client.0.smithi090.stdout:3/125: rmdir d2 39 2024-04-24T08:24:48.923 INFO:tasks.workunit.client.0.smithi090.stdout:8/122: write d2/f7 [214576,106520] 0 2024-04-24T08:24:48.925 INFO:tasks.workunit.client.0.smithi090.stdout:8/123: chown d2/da/d13/f28 2081062 1 2024-04-24T08:24:48.930 INFO:tasks.workunit.client.0.smithi090.stdout:2/173: creat d0/d14/f34 x:0 0 0 2024-04-24T08:24:48.936 INFO:tasks.workunit.client.1.smithi149.stdout:4/475: link d2/d18/d1e/d3c/f80 d2/d10/d91/f9d 0 2024-04-24T08:24:48.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/193: mkdir d8/d1d/d2f/d30/d38 0 2024-04-24T08:24:48.937 INFO:tasks.workunit.client.1.smithi149.stdout:1/599: dwrite d0/d39/d53/d61/f71 [0,4194304] 0 2024-04-24T08:24:48.940 INFO:tasks.workunit.client.0.smithi090.stdout:7/228: rename db/d17/d29/d32/f4a to db/d17/d29/d32/f57 0 2024-04-24T08:24:48.941 INFO:tasks.workunit.client.0.smithi090.stdout:4/105: rename d3/ce to d3/c19 0 2024-04-24T08:24:48.945 INFO:tasks.workunit.client.0.smithi090.stdout:2/174: dwrite d0/d7/f33 [0,4194304] 0 2024-04-24T08:24:48.946 INFO:tasks.workunit.client.1.smithi149.stdout:0/439: creat d2/d9/df/d40/f88 x:0 0 0 2024-04-24T08:24:48.949 INFO:tasks.workunit.client.1.smithi149.stdout:8/434: creat d1/d1f/d31/f95 x:0 0 0 2024-04-24T08:24:48.952 INFO:tasks.workunit.client.1.smithi149.stdout:8/435: chown d1/d1f/d5e/f73 85 1 2024-04-24T08:24:48.952 INFO:tasks.workunit.client.0.smithi090.stdout:6/194: dwrite d8/d1d/d29/f2c [0,4194304] 0 2024-04-24T08:24:48.960 INFO:tasks.workunit.client.1.smithi149.stdout:3/544: getdents d0/d5/da/d19/d80 0 2024-04-24T08:24:48.961 INFO:tasks.workunit.client.0.smithi090.stdout:7/229: creat db/d17/d29/d38/d3e/f58 x:0 0 0 2024-04-24T08:24:48.963 INFO:tasks.workunit.client.1.smithi149.stdout:3/545: chown d0/d5/da/d19/d1b/d72/fbb 388259 1 2024-04-24T08:24:48.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/230: truncate db/d17/f3a 992917 0 2024-04-24T08:24:48.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/175: rename d0/d7/d24/l2d to d0/d7/d28/l35 0 2024-04-24T08:24:48.968 INFO:tasks.workunit.client.0.smithi090.stdout:9/149: dwrite d0/d19/f1e [4194304,4194304] 0 2024-04-24T08:24:48.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/176: dread - d0/d7/d24/f27 zero size 2024-04-24T08:24:48.975 INFO:tasks.workunit.client.0.smithi090.stdout:2/177: chown d0/d14/d17/d2a/d31 144549609 1 2024-04-24T08:24:48.981 INFO:tasks.workunit.client.0.smithi090.stdout:4/106: mkdir d3/d18/d1a 0 2024-04-24T08:24:48.985 INFO:tasks.workunit.client.1.smithi149.stdout:9/535: link d1/d2/d15/f3c d1/d2/d34/fbe 0 2024-04-24T08:24:48.990 INFO:tasks.workunit.client.0.smithi090.stdout:3/126: dread d2/d15/f16 [0,4194304] 0 2024-04-24T08:24:48.991 INFO:tasks.workunit.client.1.smithi149.stdout:9/536: readlink d1/da/dd/d55/lb8 0 2024-04-24T08:24:48.994 INFO:tasks.workunit.client.1.smithi149.stdout:9/537: readlink d1/da/l4f 0 2024-04-24T08:24:48.995 INFO:tasks.workunit.client.0.smithi090.stdout:9/150: symlink d0/d19/d20/l2b 0 2024-04-24T08:24:48.995 INFO:tasks.workunit.client.0.smithi090.stdout:9/151: stat d0/c6 0 2024-04-24T08:24:48.996 INFO:tasks.workunit.client.1.smithi149.stdout:4/476: dwrite d2/d10/d1a/f36 [4194304,4194304] 0 2024-04-24T08:24:48.998 INFO:tasks.workunit.client.1.smithi149.stdout:8/436: rename d1/d1f/d5e/f87 to d1/de/d18/d32/d65/d8b/d7d/f96 0 2024-04-24T08:24:48.999 INFO:tasks.workunit.client.0.smithi090.stdout:6/195: write d8/dc/d19/f27 [1236257,89937] 0 2024-04-24T08:24:49.000 INFO:tasks.workunit.client.0.smithi090.stdout:6/196: chown d8/f1b 100983341 1 2024-04-24T08:24:49.001 INFO:tasks.workunit.client.1.smithi149.stdout:7/505: getdents d9/d43/d4c/d67 0 2024-04-24T08:24:49.002 INFO:tasks.workunit.client.1.smithi149.stdout:6/385: dwrite d0/f17 [0,4194304] 0 2024-04-24T08:24:49.004 INFO:tasks.workunit.client.1.smithi149.stdout:2/436: truncate d0/d6/db/f10 8189292 0 2024-04-24T08:24:49.007 INFO:tasks.workunit.client.0.smithi090.stdout:7/231: mkdir db/d17/d39/d59 0 2024-04-24T08:24:49.009 INFO:tasks.workunit.client.1.smithi149.stdout:5/586: link db/d12/d22/d45/f42 db/d12/d22/fc4 0 2024-04-24T08:24:49.009 INFO:tasks.workunit.client.1.smithi149.stdout:3/546: dread d0/f11 [0,4194304] 0 2024-04-24T08:24:49.012 INFO:tasks.workunit.client.0.smithi090.stdout:3/127: creat d2/df/f23 x:0 0 0 2024-04-24T08:24:49.013 INFO:tasks.workunit.client.0.smithi090.stdout:9/152: creat d0/dc/f2c x:0 0 0 2024-04-24T08:24:49.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/197: dwrite d8/d1d/f24 [4194304,4194304] 0 2024-04-24T08:24:49.015 INFO:tasks.workunit.client.1.smithi149.stdout:3/547: chown d0/d5/da/d19/d1b/d72/fbb 1272906 1 2024-04-24T08:24:49.018 INFO:tasks.workunit.client.0.smithi090.stdout:6/198: write f0 [1201312,106357] 0 2024-04-24T08:24:49.019 INFO:tasks.workunit.client.1.smithi149.stdout:1/600: dwrite d0/f84 [4194304,4194304] 0 2024-04-24T08:24:49.019 INFO:tasks.workunit.client.0.smithi090.stdout:7/232: mkdir db/d17/d29/d32/d5a 0 2024-04-24T08:24:49.020 INFO:tasks.workunit.client.1.smithi149.stdout:8/437: dread - d1/d1f/d25/f34 zero size 2024-04-24T08:24:49.021 INFO:tasks.workunit.client.1.smithi149.stdout:4/477: symlink d2/d24/l9e 0 2024-04-24T08:24:49.023 INFO:tasks.workunit.client.1.smithi149.stdout:6/386: readlink d0/l22 0 2024-04-24T08:24:49.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/438: chown d1/d1f/d25/c44 3 1 2024-04-24T08:24:49.024 INFO:tasks.workunit.client.1.smithi149.stdout:4/478: fsync d2/d10/d1a/d34/f37 0 2024-04-24T08:24:49.024 INFO:tasks.workunit.client.0.smithi090.stdout:4/107: unlink d3/df/f11 0 2024-04-24T08:24:49.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/199: write d8/d1d/f1e [230503,113670] 0 2024-04-24T08:24:49.026 INFO:tasks.workunit.client.1.smithi149.stdout:7/506: rename d9/d43/d4c to d9/dc/d11/d83/da6 0 2024-04-24T08:24:49.029 INFO:tasks.workunit.client.1.smithi149.stdout:2/437: mknod d0/d6/c89 0 2024-04-24T08:24:49.032 INFO:tasks.workunit.client.1.smithi149.stdout:0/440: rmdir d2/d9/df/d40/d4a/d4e/d6f/d7a 0 2024-04-24T08:24:49.034 INFO:tasks.workunit.client.1.smithi149.stdout:7/507: write d9/dc/d7b/fa2 [103616,90772] 0 2024-04-24T08:24:49.034 INFO:tasks.workunit.client.1.smithi149.stdout:4/479: truncate d2/d18/d1e/d3c/f84 790755 0 2024-04-24T08:24:49.035 INFO:tasks.workunit.client.0.smithi090.stdout:7/233: mknod db/c5b 0 2024-04-24T08:24:49.035 INFO:tasks.workunit.client.0.smithi090.stdout:6/200: write d8/fa [6548593,106454] 0 2024-04-24T08:24:49.035 INFO:tasks.workunit.client.1.smithi149.stdout:5/587: mkdir db/d12/d60/db2/d83/dc5 0 2024-04-24T08:24:49.035 INFO:tasks.workunit.client.1.smithi149.stdout:9/538: mkdir d1/d2/d33/d5d/d83/dbf 0 2024-04-24T08:24:49.037 INFO:tasks.workunit.client.0.smithi090.stdout:4/108: chown d3/df/c14 49629775 1 2024-04-24T08:24:49.037 INFO:tasks.workunit.client.1.smithi149.stdout:9/539: dread - d1/d2/fba zero size 2024-04-24T08:24:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:4/480: write d2/d18/d1e/d3b/f7c [167450,6033] 0 2024-04-24T08:24:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:3/548: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:1/601: mknod d0/d39/da0/cbb 0 2024-04-24T08:24:49.045 INFO:tasks.workunit.client.0.smithi090.stdout:9/153: dwrite d0/dc/fd [0,4194304] 0 2024-04-24T08:24:49.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/481: dread - d2/d24/d2e/d38/f5c zero size 2024-04-24T08:24:49.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/201: rename f0 to d8/d1d/d2f/f39 0 2024-04-24T08:24:49.054 INFO:tasks.workunit.client.0.smithi090.stdout:7/234: creat db/d17/d29/d38/f5c x:0 0 0 2024-04-24T08:24:49.054 INFO:tasks.workunit.client.1.smithi149.stdout:8/439: truncate d1/de/d18/d43/f88 5229707 0 2024-04-24T08:24:49.055 INFO:tasks.workunit.client.1.smithi149.stdout:8/440: stat d1/de 0 2024-04-24T08:24:49.056 INFO:tasks.workunit.client.0.smithi090.stdout:9/154: mkdir d0/dc/d24/d2d 0 2024-04-24T08:24:49.059 INFO:tasks.workunit.client.1.smithi149.stdout:5/588: dwrite db/d35/f9d [0,4194304] 0 2024-04-24T08:24:49.061 INFO:tasks.workunit.client.0.smithi090.stdout:3/128: write d2/f9 [2140240,90767] 0 2024-04-24T08:24:49.066 INFO:tasks.workunit.client.1.smithi149.stdout:2/438: dwrite d0/d6/db/d27/f31 [0,4194304] 0 2024-04-24T08:24:49.066 INFO:tasks.workunit.client.1.smithi149.stdout:5/589: write db/d12/d22/d45/f93 [2336378,78590] 0 2024-04-24T08:24:49.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/235: mknod db/d17/d39/c5d 0 2024-04-24T08:24:49.069 INFO:tasks.workunit.client.1.smithi149.stdout:5/590: chown db/l51 43 1 2024-04-24T08:24:49.072 INFO:tasks.workunit.client.1.smithi149.stdout:1/602: mknod d0/d5/db/d18/cbc 0 2024-04-24T08:24:49.074 INFO:tasks.workunit.client.1.smithi149.stdout:1/603: write d0/d39/f7c [2236383,17786] 0 2024-04-24T08:24:49.079 INFO:tasks.workunit.client.0.smithi090.stdout:4/109: creat d3/df/f1b x:0 0 0 2024-04-24T08:24:49.083 INFO:tasks.workunit.client.1.smithi149.stdout:8/441: write d1/d1f/f50 [5759952,780] 0 2024-04-24T08:24:49.084 INFO:tasks.workunit.client.0.smithi090.stdout:4/110: write d3/f9 [901586,117131] 0 2024-04-24T08:24:49.084 INFO:tasks.workunit.client.0.smithi090.stdout:7/236: creat db/d17/d29/d38/f5e x:0 0 0 2024-04-24T08:24:49.089 INFO:tasks.workunit.client.0.smithi090.stdout:4/111: fdatasync d3/f13 0 2024-04-24T08:24:49.089 INFO:tasks.workunit.client.0.smithi090.stdout:6/202: dread d8/f16 [0,4194304] 0 2024-04-24T08:24:49.091 INFO:tasks.workunit.client.0.smithi090.stdout:4/112: write d3/f13 [861646,20179] 0 2024-04-24T08:24:49.091 INFO:tasks.workunit.client.1.smithi149.stdout:6/387: dread d0/fc [0,4194304] 0 2024-04-24T08:24:49.091 INFO:tasks.workunit.client.0.smithi090.stdout:7/237: fdatasync db/d17/d29/f2f 0 2024-04-24T08:24:49.093 INFO:tasks.workunit.client.1.smithi149.stdout:6/388: dread - d0/d1a/d28/d31/f5c zero size 2024-04-24T08:24:49.093 INFO:tasks.workunit.client.1.smithi149.stdout:2/439: unlink d0/d6/db/d27/c68 0 2024-04-24T08:24:49.093 INFO:tasks.workunit.client.0.smithi090.stdout:9/155: dwrite d0/d19/f28 [0,4194304] 0 2024-04-24T08:24:49.094 INFO:tasks.workunit.client.1.smithi149.stdout:5/591: mkdir db/d12/d2f/d74/dc6 0 2024-04-24T08:24:49.094 INFO:tasks.workunit.client.0.smithi090.stdout:7/238: write db/d17/d29/d38/f53 [956519,12097] 0 2024-04-24T08:24:49.096 INFO:tasks.workunit.client.0.smithi090.stdout:7/239: chown db/d17/d1b 429227088 1 2024-04-24T08:24:49.098 INFO:tasks.workunit.client.1.smithi149.stdout:4/482: mkdir d2/d10/d91/d9f 0 2024-04-24T08:24:49.102 INFO:tasks.workunit.client.0.smithi090.stdout:6/203: dread d8/dc/f10 [0,4194304] 0 2024-04-24T08:24:49.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/592: rmdir db/d12/d2f 39 2024-04-24T08:24:49.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/204: readlink d8/d1d/l36 0 2024-04-24T08:24:49.106 INFO:tasks.workunit.client.0.smithi090.stdout:4/113: dwrite d3/f15 [0,4194304] 0 2024-04-24T08:24:49.107 INFO:tasks.workunit.client.1.smithi149.stdout:7/508: dread d9/dc/d11/f42 [0,4194304] 0 2024-04-24T08:24:49.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/156: dwrite d0/d19/f1a [0,4194304] 0 2024-04-24T08:24:49.110 INFO:tasks.workunit.client.0.smithi090.stdout:6/205: creat d8/dc/f3a x:0 0 0 2024-04-24T08:24:49.111 INFO:tasks.workunit.client.1.smithi149.stdout:8/442: symlink d1/de/l97 0 2024-04-24T08:24:49.111 INFO:tasks.workunit.client.1.smithi149.stdout:8/443: read - d1/de/d18/f7e zero size 2024-04-24T08:24:49.114 INFO:tasks.workunit.client.1.smithi149.stdout:1/604: creat d0/fbd x:0 0 0 2024-04-24T08:24:49.114 INFO:tasks.workunit.client.1.smithi149.stdout:2/440: dwrite d0/d6/db/d27/d64/f74 [0,4194304] 0 2024-04-24T08:24:49.117 INFO:tasks.workunit.client.1.smithi149.stdout:3/549: getdents d0/d26/d3c/db1 0 2024-04-24T08:24:49.118 INFO:tasks.workunit.client.1.smithi149.stdout:3/550: readlink d0/d5/da/d19/l2e 0 2024-04-24T08:24:49.119 INFO:tasks.workunit.client.0.smithi090.stdout:6/206: creat d8/d1d/d29/f3b x:0 0 0 2024-04-24T08:24:49.119 INFO:tasks.workunit.client.0.smithi090.stdout:6/207: stat d8/fa 0 2024-04-24T08:24:49.121 INFO:tasks.workunit.client.0.smithi090.stdout:9/157: mknod d0/dc/d24/d2d/c2e 0 2024-04-24T08:24:49.124 INFO:tasks.workunit.client.0.smithi090.stdout:9/158: fdatasync d0/d19/f25 0 2024-04-24T08:24:49.125 INFO:tasks.workunit.client.1.smithi149.stdout:2/441: mkdir d0/d6/db/d39/d78/d8a 0 2024-04-24T08:24:49.125 INFO:tasks.workunit.client.0.smithi090.stdout:4/114: rmdir d3/d18/d1a 0 2024-04-24T08:24:49.127 INFO:tasks.workunit.client.1.smithi149.stdout:5/593: mknod db/d12/d22/d45/da7/cc7 0 2024-04-24T08:24:49.128 INFO:tasks.workunit.client.1.smithi149.stdout:3/551: symlink d0/d5/d1a/d70/lc1 0 2024-04-24T08:24:49.129 INFO:tasks.workunit.client.1.smithi149.stdout:7/509: mkdir d9/dc/d11/d84/da7 0 2024-04-24T08:24:49.129 INFO:tasks.workunit.client.1.smithi149.stdout:3/552: readlink d0/d5/d1a/lb4 0 2024-04-24T08:24:49.130 INFO:tasks.workunit.client.1.smithi149.stdout:5/594: write db/d12/d22/d45/f21 [7109840,94135] 0 2024-04-24T08:24:49.131 INFO:tasks.workunit.client.0.smithi090.stdout:4/115: write d3/f9 [687940,72703] 0 2024-04-24T08:24:49.132 INFO:tasks.workunit.client.1.smithi149.stdout:7/510: write d9/dc/de/f3a [363973,86593] 0 2024-04-24T08:24:49.134 INFO:tasks.workunit.client.0.smithi090.stdout:4/116: chown d3/f16 110600782 1 2024-04-24T08:24:49.136 INFO:tasks.workunit.client.1.smithi149.stdout:7/511: dread - d9/dc/d7b/f7d zero size 2024-04-24T08:24:49.136 INFO:tasks.workunit.client.0.smithi090.stdout:9/159: truncate d0/f15 3885862 0 2024-04-24T08:24:49.138 INFO:tasks.workunit.client.1.smithi149.stdout:2/442: fsync d0/d6/db/f3e 0 2024-04-24T08:24:49.139 INFO:tasks.workunit.client.1.smithi149.stdout:8/444: creat d1/de/d18/f98 x:0 0 0 2024-04-24T08:24:49.143 INFO:tasks.workunit.client.0.smithi090.stdout:9/160: write d0/d19/f1a [2781427,37882] 0 2024-04-24T08:24:49.143 INFO:tasks.workunit.client.1.smithi149.stdout:7/512: creat d9/dc/d11/d83/da6/d67/fa8 x:0 0 0 2024-04-24T08:24:49.146 INFO:tasks.workunit.client.0.smithi090.stdout:6/208: dwrite d8/dc/d19/f34 [0,4194304] 0 2024-04-24T08:24:49.146 INFO:tasks.workunit.client.1.smithi149.stdout:8/445: chown d1/de/d18/d43/f88 0 1 2024-04-24T08:24:49.148 INFO:tasks.workunit.client.0.smithi090.stdout:7/240: rmdir db 39 2024-04-24T08:24:49.149 INFO:tasks.workunit.client.0.smithi090.stdout:3/129: truncate d2/df/f19 4101548 0 2024-04-24T08:24:49.151 INFO:tasks.workunit.client.0.smithi090.stdout:6/209: read d8/dc/f11 [3716006,115838] 0 2024-04-24T08:24:49.153 INFO:tasks.workunit.client.0.smithi090.stdout:3/130: write d2/d15/f1d [407309,90220] 0 2024-04-24T08:24:49.155 INFO:tasks.workunit.client.1.smithi149.stdout:3/553: dwrite d0/d5/da/d19/d1b/d72/fbb [0,4194304] 0 2024-04-24T08:24:49.156 INFO:tasks.workunit.client.0.smithi090.stdout:9/161: symlink d0/d19/l2f 0 2024-04-24T08:24:49.157 INFO:tasks.workunit.client.1.smithi149.stdout:8/446: write d1/de/d18/f45 [2106298,59386] 0 2024-04-24T08:24:49.158 INFO:tasks.workunit.client.0.smithi090.stdout:3/131: write d2/df/f23 [900534,4647] 0 2024-04-24T08:24:49.161 INFO:tasks.workunit.client.0.smithi090.stdout:9/162: chown d0/f2 551384 1 2024-04-24T08:24:49.161 INFO:tasks.workunit.client.1.smithi149.stdout:3/554: write d0/d5/d1a/f8f [240702,107074] 0 2024-04-24T08:24:49.162 INFO:tasks.workunit.client.0.smithi090.stdout:4/117: dwrite d3/f13 [0,4194304] 0 2024-04-24T08:24:49.163 INFO:tasks.workunit.client.1.smithi149.stdout:8/447: mknod d1/c99 0 2024-04-24T08:24:49.165 INFO:tasks.workunit.client.0.smithi090.stdout:7/241: mknod db/d17/d29/d32/c5f 0 2024-04-24T08:24:49.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/132: truncate d2/f14 1380266 0 2024-04-24T08:24:49.173 INFO:tasks.workunit.client.0.smithi090.stdout:9/163: fdatasync d0/d19/f1d 0 2024-04-24T08:24:49.174 INFO:tasks.workunit.client.1.smithi149.stdout:3/555: creat d0/d5/d53/d4c/fc2 x:0 0 0 2024-04-24T08:24:49.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/540: dwrite d1/da/d13/f45 [0,4194304] 0 2024-04-24T08:24:49.180 INFO:tasks.workunit.client.1.smithi149.stdout:3/556: read - d0/d5/d1a/d70/fba zero size 2024-04-24T08:24:49.185 INFO:tasks.workunit.client.1.smithi149.stdout:3/557: dread - d0/d5/d53/d4c/fc2 zero size 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/210: dwrite d8/d1d/f1e [0,4194304] 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/133: rename d2/df/f12 to d2/f24 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/211: creat d8/d1d/d2f/f3c x:0 0 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/242: rename db/d17/d29/d38/f5c to db/d17/d29/d38/d3e/d46/f60 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:9/164: dwrite d0/f18 [0,4194304] 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/212: dwrite d8/d1d/f24 [4194304,4194304] 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/134: mkdir d2/d25 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/135: fsync d2/f9 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:9/165: mknod d0/c30 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/136: readlink d2/lc 0 2024-04-24T08:24:49.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/137: chown d2/l18 1652 1 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/213: rmdir d8/d1d/d1f 39 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/214: fdatasync f6 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/215: dwrite d8/dc/f3a [0,4194304] 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/216: chown d8/d1d/f24 290360 1 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:7/243: link db/d17/d29/f2f db/d17/d29/d32/f61 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:7/244: write db/d17/f2d [1207992,68196] 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/217: creat d8/dc/d19/f3d x:0 0 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:3/138: dwrite d2/df/f19 [0,4194304] 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/218: unlink d8/d1d/d1f/f23 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/219: read d8/d1d/d29/f2c [1074638,126231] 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:3/139: rename d2/df/c11 to d2/df/c26 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:5/229: sync 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:3/140: creat d2/df/d21/f27 x:0 0 0 2024-04-24T08:24:49.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/220: dread d8/dc/f3a [0,4194304] 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/230: creat d8/da/d3a/d2d/f4d x:0 0 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/231: readlink d8/da/d18/d21/l29 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/221: mkdir d8/d1d/d1f/d3e 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:3/141: truncate d2/df/f10 363362 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:3/142: chown d2/df/f23 2724 1 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:3/143: creat d2/df/d1a/f28 x:0 0 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/222: dread d8/d1d/d29/f2c [0,4194304] 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/245: truncate db/fc 1475667 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/232: dread d8/f9 [8388608,4194304] 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/223: dwrite d8/f15 [0,4194304] 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/233: creat d8/da/d3a/d36/f4e x:0 0 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:3/144: rename d2/df/c26 to d2/d15/c29 0 2024-04-24T08:24:49.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/224: getdents d8/d1d 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:7/513: dwrite d9/dc/d11/d83/f93 [0,4194304] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/441: truncate d2/d9/df/d2f/f34 3428051 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:4/483: write d2/d18/f20 [892764,75248] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:7/514: truncate d9/dc/d7b/fa2 1149317 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:9/541: mkdir d1/da/d13/d74/dbb/dc0 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:5/595: rename db/d12/d22/d45/da7 to db/d12/d60/db2/d83/dc8 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:1/605: truncate d0/d39/d53/d61/f70 2661420 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:6/389: dwrite d0/d1a/f25 [0,4194304] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/442: creat d2/d9/f89 x:0 0 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/443: read d2/d9/df/d48/f4f [3099206,71067] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:2/443: write d0/d6/db/d39/d3d/d44/f53 [360495,2187] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:1/606: dread d0/f25 [0,4194304] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:4/484: dread d2/d18/d1e/d3c/d48/d69/f70 [0,4194304] 0 2024-04-24T08:24:49.368 INFO:tasks.workunit.client.1.smithi149.stdout:9/542: creat d1/d2/d33/d5d/d83/dbf/fc1 x:0 0 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:9/543: chown d1/da/dd/d55 697 1 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:9/544: write d1/da/dd/d55/fb1 [120081,68826] 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:9/545: chown d1/d2/d15/d1c 328379618 1 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:6/390: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f81 x:0 0 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:4/485: write d2/d24/d2e/f98 [2239726,12560] 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:4/486: readlink d2/d24/d2e/d38/l73 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/444: getdents d0/d6/db/d27/d81 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:0/444: dread d2/d9/df/d40/d4a/f79 [0,4194304] 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:4/487: creat d2/d18/d1e/d3b/fa0 x:0 0 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:5/596: link db/l51 db/d12/d2f/d74/dc6/lc9 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:9/546: link d1/d2/d33/d76/f7c d1/da/d4a/dad/fc2 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:1/607: link d0/d5/db/d63/d98/l9f d0/d5/db/d18/d4d/d50/daf/lbe 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:0/445: dwrite d2/d17/d41/d68/f82 [0,4194304] 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:5/597: mknod db/d1f/cca 0 2024-04-24T08:24:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/445: mkdir d0/d6/db/d39/d3d/d44/d8b 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:3/558: write d0/d5/d53/d4c/d5f/f8b [771173,80901] 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/446: creat d2/d9/df/d40/d4a/d4e/d6f/f8a x:0 0 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/447: fdatasync d2/d9/d1a/f3f 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:3/559: dread - d0/d5/da/d19/d1b/d22/fbc zero size 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:2/446: fdatasync d0/d6/db/d27/f47 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/448: read d2/d9/d1a/d36/f47 [350570,89267] 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:2/447: truncate d0/d6/db/d27/d64/f76 205407 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/449: dread d2/d9/d1a/d36/f74 [0,4194304] 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/450: dread d2/f7f [0,4194304] 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:6/391: write d0/f3f [99531,62997] 0 2024-04-24T08:24:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/488: write d2/d10/f17 [3880882,57595] 0 2024-04-24T08:24:49.371 INFO:tasks.workunit.client.0.smithi090.stdout:7/246: write db/d17/d29/d32/f36 [759904,32255] 0 2024-04-24T08:24:49.373 INFO:tasks.workunit.client.1.smithi149.stdout:1/608: creat d0/d5/df/fbf x:0 0 0 2024-04-24T08:24:49.373 INFO:tasks.workunit.client.1.smithi149.stdout:5/598: creat db/d12/d22/fcb x:0 0 0 2024-04-24T08:24:49.374 INFO:tasks.workunit.client.0.smithi090.stdout:6/225: creat d8/f3f x:0 0 0 2024-04-24T08:24:49.378 INFO:tasks.workunit.client.1.smithi149.stdout:4/489: dread d2/d10/d1a/f36 [0,4194304] 0 2024-04-24T08:24:49.378 INFO:tasks.workunit.client.1.smithi149.stdout:3/560: symlink d0/d5/da/d19/d80/dac/lc3 0 2024-04-24T08:24:49.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/451: dread d2/d9/df/d48/f4c [0,4194304] 0 2024-04-24T08:24:49.381 INFO:tasks.workunit.client.1.smithi149.stdout:4/490: dread - d2/d18/d1e/d3b/f78 zero size 2024-04-24T08:24:49.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/452: truncate d2/d9/df/d40/f67 107755 0 2024-04-24T08:24:49.382 INFO:tasks.workunit.client.1.smithi149.stdout:6/392: creat d0/d1a/d28/d31/f82 x:0 0 0 2024-04-24T08:24:49.384 INFO:tasks.workunit.client.1.smithi149.stdout:6/393: chown d0/d1a/d28/d31/d4c/d80/d6f 318004 1 2024-04-24T08:24:49.385 INFO:tasks.workunit.client.0.smithi090.stdout:6/226: dwrite d8/f15 [0,4194304] 0 2024-04-24T08:24:49.386 INFO:tasks.workunit.client.1.smithi149.stdout:1/609: write d0/d5/db/d63/f95 [283941,63861] 0 2024-04-24T08:24:49.387 INFO:tasks.workunit.client.1.smithi149.stdout:6/394: write d0/d1a/d28/d31/d4c/d80/f68 [417689,85205] 0 2024-04-24T08:24:49.388 INFO:tasks.workunit.client.0.smithi090.stdout:6/227: write d8/d1d/f24 [4200218,88474] 0 2024-04-24T08:24:49.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/599: rename db/d1f/fc2 to db/d12/d1e/d85/d92/fcc 0 2024-04-24T08:24:49.389 INFO:tasks.workunit.client.0.smithi090.stdout:5/234: dwrite d8/f31 [4194304,4194304] 0 2024-04-24T08:24:49.391 INFO:tasks.workunit.client.1.smithi149.stdout:6/395: readlink d0/ld 0 2024-04-24T08:24:49.393 INFO:tasks.workunit.client.0.smithi090.stdout:3/145: truncate d2/d15/f16 3946990 0 2024-04-24T08:24:49.394 INFO:tasks.workunit.client.1.smithi149.stdout:1/610: write d0/d1b/d5d/f85 [479403,69919] 0 2024-04-24T08:24:49.395 INFO:tasks.workunit.client.1.smithi149.stdout:6/396: dread - d0/d1a/d28/d4a/f72 zero size 2024-04-24T08:24:49.401 INFO:tasks.workunit.client.1.smithi149.stdout:2/448: unlink d0/d6/db/d39/l3f 0 2024-04-24T08:24:49.403 INFO:tasks.workunit.client.0.smithi090.stdout:3/146: dread d2/df/f23 [0,4194304] 0 2024-04-24T08:24:49.404 INFO:tasks.workunit.client.0.smithi090.stdout:5/235: symlink d8/da/d18/d21/l4f 0 2024-04-24T08:24:49.405 INFO:tasks.workunit.client.0.smithi090.stdout:3/147: stat d2/df/f23 0 2024-04-24T08:24:49.406 INFO:tasks.workunit.client.0.smithi090.stdout:5/236: chown f2 1 1 2024-04-24T08:24:49.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/148: chown d2/d15/l1c 31913 1 2024-04-24T08:24:49.407 INFO:tasks.workunit.client.0.smithi090.stdout:5/237: chown d8/da/d3a 7036635 1 2024-04-24T08:24:49.407 INFO:tasks.workunit.client.0.smithi090.stdout:3/149: fdatasync d2/df/f19 0 2024-04-24T08:24:49.413 INFO:tasks.workunit.client.1.smithi149.stdout:3/561: dread d0/d5/ff [0,4194304] 0 2024-04-24T08:24:49.415 INFO:tasks.workunit.client.0.smithi090.stdout:8/124: sync 2024-04-24T08:24:49.415 INFO:tasks.workunit.client.0.smithi090.stdout:0/225: sync 2024-04-24T08:24:49.416 INFO:tasks.workunit.client.0.smithi090.stdout:1/170: sync 2024-04-24T08:24:49.416 INFO:tasks.workunit.client.0.smithi090.stdout:3/150: mknod d2/df/d1a/c2a 0 2024-04-24T08:24:49.419 INFO:tasks.workunit.client.0.smithi090.stdout:3/151: write d2/df/f22 [376181,38002] 0 2024-04-24T08:24:49.420 INFO:tasks.workunit.client.0.smithi090.stdout:0/226: chown d1a/l37 174034 1 2024-04-24T08:24:49.421 INFO:tasks.workunit.client.1.smithi149.stdout:5/600: creat db/d12/d2f/d74/dc6/fcd x:0 0 0 2024-04-24T08:24:49.421 INFO:tasks.workunit.client.0.smithi090.stdout:5/238: dwrite d8/da/d14/f27 [0,4194304] 0 2024-04-24T08:24:49.424 INFO:tasks.workunit.client.1.smithi149.stdout:3/562: write d0/d5/da/fa7 [808906,24003] 0 2024-04-24T08:24:49.424 INFO:tasks.workunit.client.0.smithi090.stdout:1/171: creat d3/d5/da/dc/f3a x:0 0 0 2024-04-24T08:24:49.425 INFO:tasks.workunit.client.0.smithi090.stdout:6/228: getdents d8/d1d 0 2024-04-24T08:24:49.428 INFO:tasks.workunit.client.1.smithi149.stdout:5/601: dread db/d12/d2f/d74/f97 [0,4194304] 0 2024-04-24T08:24:49.431 INFO:tasks.workunit.client.1.smithi149.stdout:2/449: getdents d0/d6/db/d39/d3d/d44/d8b 0 2024-04-24T08:24:49.433 INFO:tasks.workunit.client.0.smithi090.stdout:5/239: mkdir d8/da/d3a/d36/d50 0 2024-04-24T08:24:49.435 INFO:tasks.workunit.client.0.smithi090.stdout:8/125: dwrite d2/da/d13/f28 [0,4194304] 0 2024-04-24T08:24:49.436 INFO:tasks.workunit.client.1.smithi149.stdout:4/491: rename d2/d18/l74 to d2/d24/d2e/la1 0 2024-04-24T08:24:49.436 INFO:tasks.workunit.client.0.smithi090.stdout:3/152: dwrite d2/df/d1a/f28 [0,4194304] 0 2024-04-24T08:24:49.437 INFO:tasks.workunit.client.0.smithi090.stdout:8/126: fsync d2/da/d13/d15/f26 0 2024-04-24T08:24:49.437 INFO:tasks.workunit.client.0.smithi090.stdout:6/229: creat d8/d1d/d29/f40 x:0 0 0 2024-04-24T08:24:49.446 INFO:tasks.workunit.client.1.smithi149.stdout:3/563: unlink d0/d5/d53/d4c/d5f/la1 0 2024-04-24T08:24:49.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/227: mkdir d1a/d4b 0 2024-04-24T08:24:49.448 INFO:tasks.workunit.client.1.smithi149.stdout:3/564: chown d0/d5/f6f 89 1 2024-04-24T08:24:49.449 INFO:tasks.workunit.client.0.smithi090.stdout:0/228: truncate d1a/f3d 363086 0 2024-04-24T08:24:49.453 INFO:tasks.workunit.client.0.smithi090.stdout:6/230: dwrite f6 [0,4194304] 0 2024-04-24T08:24:49.454 INFO:tasks.workunit.client.1.smithi149.stdout:5/602: creat db/d12/d1e/d85/d92/da6/fce x:0 0 0 2024-04-24T08:24:49.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/450: rmdir d0/d6/db/d39/d3d/d44/d75 39 2024-04-24T08:24:49.462 INFO:tasks.workunit.client.1.smithi149.stdout:5/603: dread db/d12/d1e/d85/fa5 [0,4194304] 0 2024-04-24T08:24:49.468 INFO:tasks.workunit.client.0.smithi090.stdout:8/127: rmdir d2/da/d13/d15/d24 39 2024-04-24T08:24:49.468 INFO:tasks.workunit.client.1.smithi149.stdout:4/492: fsync d2/d24/d2e/f4a 0 2024-04-24T08:24:49.469 INFO:tasks.workunit.client.0.smithi090.stdout:1/172: creat d3/d1d/d34/f3b x:0 0 0 2024-04-24T08:24:49.469 INFO:tasks.workunit.client.1.smithi149.stdout:5/604: read db/f1a [3653631,13537] 0 2024-04-24T08:24:49.469 INFO:tasks.workunit.client.1.smithi149.stdout:0/453: dwrite d2/d9/d1a/f6b [0,4194304] 0 2024-04-24T08:24:49.470 INFO:tasks.workunit.client.1.smithi149.stdout:6/397: dwrite d0/f26 [0,4194304] 0 2024-04-24T08:24:49.471 INFO:tasks.workunit.client.1.smithi149.stdout:1/611: getdents d0/db2 0 2024-04-24T08:24:49.474 INFO:tasks.workunit.client.0.smithi090.stdout:0/229: stat d1a/f1d 0 2024-04-24T08:24:49.474 INFO:tasks.workunit.client.1.smithi149.stdout:3/565: fsync d0/d5/da/f6e 0 2024-04-24T08:24:49.475 INFO:tasks.workunit.client.1.smithi149.stdout:2/451: dread d0/f35 [0,4194304] 0 2024-04-24T08:24:49.476 INFO:tasks.workunit.client.0.smithi090.stdout:3/153: fdatasync d2/f24 0 2024-04-24T08:24:49.479 INFO:tasks.workunit.client.1.smithi149.stdout:6/398: stat d0/d1a/c23 0 2024-04-24T08:24:49.483 INFO:tasks.workunit.client.1.smithi149.stdout:5/605: rename db/d12/d22/fc4 to db/d12/d1e/d85/d92/da6/fcf 0 2024-04-24T08:24:49.488 INFO:tasks.workunit.client.1.smithi149.stdout:0/454: mknod d2/d9/df/d48/c8b 0 2024-04-24T08:24:49.488 INFO:tasks.workunit.client.1.smithi149.stdout:5/606: dread - db/d12/d1e/d85/d92/da6/fce zero size 2024-04-24T08:24:49.489 INFO:tasks.workunit.client.0.smithi090.stdout:8/128: dwrite d2/f10 [0,4194304] 0 2024-04-24T08:24:49.489 INFO:tasks.workunit.client.0.smithi090.stdout:5/240: getdents d8/da/d3a/d2d/d37 0 2024-04-24T08:24:49.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/154: unlink d2/df/f23 0 2024-04-24T08:24:49.493 INFO:tasks.workunit.client.0.smithi090.stdout:5/241: chown d8/da/d14 7047893 1 2024-04-24T08:24:49.493 INFO:tasks.workunit.client.0.smithi090.stdout:1/173: dwrite d3/d5/da/dc/f3a [0,4194304] 0 2024-04-24T08:24:49.494 INFO:tasks.workunit.client.0.smithi090.stdout:0/230: mknod d1a/d2a/d3c/c4c 0 2024-04-24T08:24:49.495 INFO:tasks.workunit.client.1.smithi149.stdout:4/493: dread d2/ff [0,4194304] 0 2024-04-24T08:24:49.497 INFO:tasks.workunit.client.1.smithi149.stdout:6/399: write d0/f1 [5705890,114584] 0 2024-04-24T08:24:49.498 INFO:tasks.workunit.client.1.smithi149.stdout:4/494: read - d2/d18/d1e/d3b/f78 zero size 2024-04-24T08:24:49.500 INFO:tasks.workunit.client.1.smithi149.stdout:3/566: rename d0/d5/d53/d39/f56 to d0/d5/d1a/d70/da0/fc4 0 2024-04-24T08:24:49.507 INFO:tasks.workunit.client.0.smithi090.stdout:5/242: dread d8/da/f1d [0,4194304] 0 2024-04-24T08:24:49.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/243: write f7 [2193306,69056] 0 2024-04-24T08:24:49.514 INFO:tasks.workunit.client.0.smithi090.stdout:3/155: unlink d2/df/d1a/c20 0 2024-04-24T08:24:49.517 INFO:tasks.workunit.client.0.smithi090.stdout:8/129: dread d2/f7 [0,4194304] 0 2024-04-24T08:24:49.519 INFO:tasks.workunit.client.1.smithi149.stdout:5/607: rmdir db 39 2024-04-24T08:24:49.520 INFO:tasks.workunit.client.1.smithi149.stdout:0/455: symlink d2/d9/df/d40/l8c 0 2024-04-24T08:24:49.521 INFO:tasks.workunit.client.1.smithi149.stdout:2/452: mkdir d0/d6/db/d39/d3d/d8c 0 2024-04-24T08:24:49.522 INFO:tasks.workunit.client.1.smithi149.stdout:3/567: dwrite d0/d5/f6f [0,4194304] 0 2024-04-24T08:24:49.523 INFO:tasks.workunit.client.0.smithi090.stdout:1/174: mknod d3/d5/da/c3c 0 2024-04-24T08:24:49.524 INFO:tasks.workunit.client.0.smithi090.stdout:5/244: dwrite d8/da/d18/f22 [4194304,4194304] 0 2024-04-24T08:24:49.524 INFO:tasks.workunit.client.1.smithi149.stdout:2/453: chown d0/d6/l19 36 1 2024-04-24T08:24:49.530 INFO:tasks.workunit.client.0.smithi090.stdout:6/231: creat d8/dc/d19/f41 x:0 0 0 2024-04-24T08:24:49.531 INFO:tasks.workunit.client.1.smithi149.stdout:1/612: creat d0/d39/fc0 x:0 0 0 2024-04-24T08:24:49.533 INFO:tasks.workunit.client.1.smithi149.stdout:4/495: mkdir d2/d10/d1a/da2 0 2024-04-24T08:24:49.534 INFO:tasks.workunit.client.0.smithi090.stdout:3/156: creat d2/df/d1a/f2b x:0 0 0 2024-04-24T08:24:49.535 INFO:tasks.workunit.client.0.smithi090.stdout:3/157: readlink d2/l18 0 2024-04-24T08:24:49.537 INFO:tasks.workunit.client.0.smithi090.stdout:4/118: sync 2024-04-24T08:24:49.537 INFO:tasks.workunit.client.0.smithi090.stdout:2/178: sync 2024-04-24T08:24:49.537 INFO:tasks.workunit.client.0.smithi090.stdout:9/166: sync 2024-04-24T08:24:49.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/167: readlink d0/l9 0 2024-04-24T08:24:49.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/130: read d2/da/d13/d15/d24/f21 [2058493,35950] 0 2024-04-24T08:24:49.541 INFO:tasks.workunit.client.1.smithi149.stdout:3/568: creat d0/d5/da/d19/d80/dac/fc5 x:0 0 0 2024-04-24T08:24:49.542 INFO:tasks.workunit.client.0.smithi090.stdout:5/245: dread - d8/f10 zero size 2024-04-24T08:24:49.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/131: fdatasync d2/f12 0 2024-04-24T08:24:49.543 INFO:tasks.workunit.client.0.smithi090.stdout:0/231: creat d1a/d4b/f4d x:0 0 0 2024-04-24T08:24:49.543 INFO:tasks.workunit.client.1.smithi149.stdout:6/400: symlink d0/d1a/l83 0 2024-04-24T08:24:49.544 INFO:tasks.workunit.client.0.smithi090.stdout:5/246: truncate d8/da/d3a/d2d/f4d 153637 0 2024-04-24T08:24:49.544 INFO:tasks.workunit.client.1.smithi149.stdout:1/613: creat d0/d5/db/d18/fc1 x:0 0 0 2024-04-24T08:24:49.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/232: symlink d8/d1d/l42 0 2024-04-24T08:24:49.547 INFO:tasks.workunit.client.1.smithi149.stdout:5/608: creat db/d1f/d7f/fd0 x:0 0 0 2024-04-24T08:24:49.548 INFO:tasks.workunit.client.1.smithi149.stdout:1/614: write d0/d5/d4f/d60/f78 [956577,121943] 0 2024-04-24T08:24:49.549 INFO:tasks.workunit.client.1.smithi149.stdout:1/615: readlink d0/da/l1d 0 2024-04-24T08:24:49.550 INFO:tasks.workunit.client.0.smithi090.stdout:9/168: symlink d0/d19/l31 0 2024-04-24T08:24:49.552 INFO:tasks.workunit.client.1.smithi149.stdout:3/569: mkdir d0/d5/da/d19/d1b/d22/dc6 0 2024-04-24T08:24:49.553 INFO:tasks.workunit.client.0.smithi090.stdout:4/119: dwrite d3/df/f1b [0,4194304] 0 2024-04-24T08:24:49.554 INFO:tasks.workunit.client.0.smithi090.stdout:8/132: write d2/da/f1c [756468,63632] 0 2024-04-24T08:24:49.556 INFO:tasks.workunit.client.0.smithi090.stdout:0/232: readlink d1a/l2d 0 2024-04-24T08:24:49.556 INFO:tasks.workunit.client.1.smithi149.stdout:4/496: creat d2/d10/d1a/da2/fa3 x:0 0 0 2024-04-24T08:24:49.556 INFO:tasks.workunit.client.0.smithi090.stdout:8/133: chown d2/f7 28968 1 2024-04-24T08:24:49.559 INFO:tasks.workunit.client.0.smithi090.stdout:5/247: unlink f4 0 2024-04-24T08:24:49.560 INFO:tasks.workunit.client.1.smithi149.stdout:5/609: symlink db/d12/d2f/d78/ld1 0 2024-04-24T08:24:49.561 INFO:tasks.workunit.client.0.smithi090.stdout:5/248: fdatasync d8/da/d18/d21/f4b 0 2024-04-24T08:24:49.562 INFO:tasks.workunit.client.0.smithi090.stdout:3/158: mkdir d2/d25/d2c 0 2024-04-24T08:24:49.565 INFO:tasks.workunit.client.1.smithi149.stdout:6/401: symlink d0/d1a/l84 0 2024-04-24T08:24:49.565 INFO:tasks.workunit.client.1.smithi149.stdout:7/515: sync 2024-04-24T08:24:49.565 INFO:tasks.workunit.client.1.smithi149.stdout:9/547: sync 2024-04-24T08:24:49.567 INFO:tasks.workunit.client.0.smithi090.stdout:6/233: symlink d8/d1d/d2f/l43 0 2024-04-24T08:24:49.568 INFO:tasks.workunit.client.0.smithi090.stdout:6/234: write d8/f15 [1141246,102498] 0 2024-04-24T08:24:49.571 INFO:tasks.workunit.client.0.smithi090.stdout:2/179: mkdir d0/d14/d36 0 2024-04-24T08:24:49.571 INFO:tasks.workunit.client.0.smithi090.stdout:8/134: dread d2/f10 [0,4194304] 0 2024-04-24T08:24:49.571 INFO:tasks.workunit.client.1.smithi149.stdout:0/456: write d2/d9/f2d [3043183,95665] 0 2024-04-24T08:24:49.573 INFO:tasks.workunit.client.1.smithi149.stdout:4/497: mkdir d2/d18/da4 0 2024-04-24T08:24:49.573 INFO:tasks.workunit.client.1.smithi149.stdout:1/616: dwrite d0/fbd [0,4194304] 0 2024-04-24T08:24:49.573 INFO:tasks.workunit.client.0.smithi090.stdout:4/120: dwrite d3/f9 [0,4194304] 0 2024-04-24T08:24:49.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/169: fsync d0/d19/f1d 0 2024-04-24T08:24:49.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/135: write d2/da/d13/d15/f16 [2267932,96734] 0 2024-04-24T08:24:49.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/610: unlink db/d12/f87 0 2024-04-24T08:24:49.582 INFO:tasks.workunit.client.0.smithi090.stdout:7/247: sync 2024-04-24T08:24:49.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/402: symlink d0/d1a/d28/d4a/l85 0 2024-04-24T08:24:49.585 INFO:tasks.workunit.client.0.smithi090.stdout:7/248: dread - db/d17/d29/d32/f4c zero size 2024-04-24T08:24:49.585 INFO:tasks.workunit.client.1.smithi149.stdout:2/454: dwrite d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:49.589 INFO:tasks.workunit.client.0.smithi090.stdout:1/175: truncate d3/d5/f14 2647841 0 2024-04-24T08:24:49.594 INFO:tasks.workunit.client.0.smithi090.stdout:3/159: symlink d2/df/l2d 0 2024-04-24T08:24:49.603 INFO:tasks.workunit.client.1.smithi149.stdout:7/516: dwrite d9/d43/f7e [4194304,4194304] 0 2024-04-24T08:24:49.605 INFO:tasks.workunit.client.1.smithi149.stdout:3/570: creat d0/d5/fc7 x:0 0 0 2024-04-24T08:24:49.605 INFO:tasks.workunit.client.0.smithi090.stdout:1/176: dwrite d3/d5/f28 [0,4194304] 0 2024-04-24T08:24:49.605 INFO:tasks.workunit.client.0.smithi090.stdout:6/235: rmdir d8/dc 39 2024-04-24T08:24:49.610 INFO:tasks.workunit.client.1.smithi149.stdout:3/571: stat d0/d5/d1a/d70/fba 0 2024-04-24T08:24:49.611 INFO:tasks.workunit.client.1.smithi149.stdout:3/572: dread - d0/d5/d53/d4c/fc2 zero size 2024-04-24T08:24:49.614 INFO:tasks.workunit.client.1.smithi149.stdout:6/403: mknod d0/d1a/d28/d31/d33/c86 0 2024-04-24T08:24:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/455: mknod d0/d6/db/d28/d7f/c8d 0 2024-04-24T08:24:49.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/170: mkdir d0/d19/d20/d32 0 2024-04-24T08:24:49.626 INFO:tasks.workunit.client.1.smithi149.stdout:7/517: chown d9/dc/d11/d26/c94 163569 1 2024-04-24T08:24:49.632 INFO:tasks.workunit.client.1.smithi149.stdout:5/611: fdatasync db/d12/d1e/f4f 0 2024-04-24T08:24:49.637 INFO:tasks.workunit.client.1.smithi149.stdout:3/573: rename d0/d5/da/d59/d79/c89 to d0/d5/d53/d4c/cc8 0 2024-04-24T08:24:49.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/498: dwrite d2/d24/d2e/f51 [4194304,4194304] 0 2024-04-24T08:24:49.642 INFO:tasks.workunit.client.1.smithi149.stdout:0/457: dwrite d2/d9/df/f21 [0,4194304] 0 2024-04-24T08:24:49.642 INFO:tasks.workunit.client.1.smithi149.stdout:3/574: fdatasync d0/d5/d1a/d70/fba 0 2024-04-24T08:24:49.643 INFO:tasks.workunit.client.0.smithi090.stdout:3/160: creat d2/d15/f2e x:0 0 0 2024-04-24T08:24:49.646 INFO:tasks.workunit.client.0.smithi090.stdout:0/233: dwrite d1a/f1e [0,4194304] 0 2024-04-24T08:24:49.647 INFO:tasks.workunit.client.0.smithi090.stdout:1/177: symlink d3/d25/d30/l3d 0 2024-04-24T08:24:49.650 INFO:tasks.workunit.client.1.smithi149.stdout:0/458: fdatasync d2/d9/df/d40/d4a/d4e/d6f/f76 0 2024-04-24T08:24:49.650 INFO:tasks.workunit.client.1.smithi149.stdout:6/404: dwrite d0/d1a/d28/d31/d38/f3a [0,4194304] 0 2024-04-24T08:24:49.652 INFO:tasks.workunit.client.0.smithi090.stdout:7/249: dwrite db/d17/d39/f49 [0,4194304] 0 2024-04-24T08:24:49.652 INFO:tasks.workunit.client.0.smithi090.stdout:6/236: rmdir d8 39 2024-04-24T08:24:49.652 INFO:tasks.workunit.client.0.smithi090.stdout:2/180: creat d0/d2e/f37 x:0 0 0 2024-04-24T08:24:49.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/121: mknod d3/df/c1c 0 2024-04-24T08:24:49.658 INFO:tasks.workunit.client.1.smithi149.stdout:4/499: dread d2/ff [0,4194304] 0 2024-04-24T08:24:49.658 INFO:tasks.workunit.client.1.smithi149.stdout:3/575: dread - d0/d5/da/f14 zero size 2024-04-24T08:24:49.659 INFO:tasks.workunit.client.0.smithi090.stdout:2/181: read d0/d14/f15 [3927939,22692] 0 2024-04-24T08:24:49.662 INFO:tasks.workunit.client.1.smithi149.stdout:4/500: write d2/d18/f64 [3526895,52324] 0 2024-04-24T08:24:49.662 INFO:tasks.workunit.client.0.smithi090.stdout:9/171: mkdir d0/d19/d33 0 2024-04-24T08:24:49.672 INFO:tasks.workunit.client.1.smithi149.stdout:8/448: sync 2024-04-24T08:24:49.672 INFO:tasks.workunit.client.1.smithi149.stdout:1/617: link d0/d5/db/f87 d0/d39/fc2 0 2024-04-24T08:24:49.674 INFO:tasks.workunit.client.1.smithi149.stdout:7/518: symlink d9/d43/d4f/d66/la9 0 2024-04-24T08:24:49.680 INFO:tasks.workunit.client.0.smithi090.stdout:7/250: dread db/fd [0,4194304] 0 2024-04-24T08:24:49.681 INFO:tasks.workunit.client.0.smithi090.stdout:7/251: write f7 [4356560,122122] 0 2024-04-24T08:24:49.683 INFO:tasks.workunit.client.0.smithi090.stdout:5/249: link d8/da/f1d d8/d25/f51 0 2024-04-24T08:24:49.688 INFO:tasks.workunit.client.1.smithi149.stdout:6/405: rmdir d0/d1a/d28/d31/d4c/d4e/d62 39 2024-04-24T08:24:49.689 INFO:tasks.workunit.client.1.smithi149.stdout:5/612: write db/d12/d2f/f46 [1571996,25996] 0 2024-04-24T08:24:49.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/136: truncate d2/da/d13/f28 4136804 0 2024-04-24T08:24:49.695 INFO:tasks.workunit.client.0.smithi090.stdout:8/137: write d2/f2c [403421,157] 0 2024-04-24T08:24:49.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/178: rename d3/d12 to d3/d5/d3e 0 2024-04-24T08:24:49.701 INFO:tasks.workunit.client.1.smithi149.stdout:1/618: creat d0/d39/d53/fc3 x:0 0 0 2024-04-24T08:24:49.707 INFO:tasks.workunit.client.1.smithi149.stdout:1/619: readlink d0/d1b/db5/l3e 0 2024-04-24T08:24:49.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/172: creat d0/f34 x:0 0 0 2024-04-24T08:24:49.709 INFO:tasks.workunit.client.1.smithi149.stdout:2/456: link d0/f21 d0/d6/db/d39/f8e 0 2024-04-24T08:24:49.709 INFO:tasks.workunit.client.1.smithi149.stdout:3/576: dwrite d0/d5/da/d19/d1b/d72/fbb [0,4194304] 0 2024-04-24T08:24:49.710 INFO:tasks.workunit.client.0.smithi090.stdout:9/173: chown d0/f1b 418 1 2024-04-24T08:24:49.711 INFO:tasks.workunit.client.1.smithi149.stdout:7/519: dwrite d9/dc/d11/f37 [0,4194304] 0 2024-04-24T08:24:49.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/449: dwrite d1/f9 [4194304,4194304] 0 2024-04-24T08:24:49.712 INFO:tasks.workunit.client.1.smithi149.stdout:6/406: write d0/d1a/f75 [2436247,23466] 0 2024-04-24T08:24:49.714 INFO:tasks.workunit.client.1.smithi149.stdout:0/459: dwrite d2/d9/d1a/f25 [0,4194304] 0 2024-04-24T08:24:49.715 INFO:tasks.workunit.client.1.smithi149.stdout:5/613: creat db/d12/d2f/fd2 x:0 0 0 2024-04-24T08:24:49.715 INFO:tasks.workunit.client.0.smithi090.stdout:5/250: mknod d8/c52 0 2024-04-24T08:24:49.718 INFO:tasks.workunit.client.1.smithi149.stdout:8/450: chown d1/d1f/d22/l2a 1767304 1 2024-04-24T08:24:49.719 INFO:tasks.workunit.client.1.smithi149.stdout:2/457: chown d0/d6/db/d39/d3d/d44/d8b 397 1 2024-04-24T08:24:49.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/138: dread d2/da/d13/d15/f16 [0,4194304] 0 2024-04-24T08:24:49.725 INFO:tasks.workunit.client.1.smithi149.stdout:7/520: symlink d9/dc/d7b/laa 0 2024-04-24T08:24:49.728 INFO:tasks.workunit.client.1.smithi149.stdout:5/614: dread - db/d12/d1e/f4f zero size 2024-04-24T08:24:49.730 INFO:tasks.workunit.client.1.smithi149.stdout:0/460: read - d2/d9/f5a zero size 2024-04-24T08:24:49.731 INFO:tasks.workunit.client.0.smithi090.stdout:1/179: creat d3/d5/f3f x:0 0 0 2024-04-24T08:24:49.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/458: fsync d0/d6/db/d28/f3a 0 2024-04-24T08:24:49.735 INFO:tasks.workunit.client.0.smithi090.stdout:7/252: creat db/d17/d39/d59/f62 x:0 0 0 2024-04-24T08:24:49.735 INFO:tasks.workunit.client.1.smithi149.stdout:2/459: chown d0/d6/f12 7339517 1 2024-04-24T08:24:49.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/174: rename d0/d19/l2f to d0/d19/d20/l35 0 2024-04-24T08:24:49.738 INFO:tasks.workunit.client.1.smithi149.stdout:6/407: mkdir d0/d1a/d28/d31/d38/d54/d87 0 2024-04-24T08:24:49.740 INFO:tasks.workunit.client.1.smithi149.stdout:7/521: creat d9/dc/de/d44/fab x:0 0 0 2024-04-24T08:24:49.741 INFO:tasks.workunit.client.1.smithi149.stdout:5/615: creat db/d23/fd3 x:0 0 0 2024-04-24T08:24:49.742 INFO:tasks.workunit.client.0.smithi090.stdout:8/139: mknod d2/da/d13/d15/d24/c2d 0 2024-04-24T08:24:49.743 INFO:tasks.workunit.client.1.smithi149.stdout:0/461: chown d2/d9/df/d48/c55 48 1 2024-04-24T08:24:49.744 INFO:tasks.workunit.client.1.smithi149.stdout:5/616: dread db/d35/f7c [0,4194304] 0 2024-04-24T08:24:49.744 INFO:tasks.workunit.client.0.smithi090.stdout:6/237: mkdir d8/d1d/d1f/d3e/d44 0 2024-04-24T08:24:49.745 INFO:tasks.workunit.client.1.smithi149.stdout:3/577: link d0/d5/da/f97 d0/d5/da/d59/fc9 0 2024-04-24T08:24:49.748 INFO:tasks.workunit.client.1.smithi149.stdout:2/460: mkdir d0/d6/db/d28/d7f/d8f 0 2024-04-24T08:24:49.751 INFO:tasks.workunit.client.0.smithi090.stdout:1/180: creat d3/d5/da/dc/f40 x:0 0 0 2024-04-24T08:24:49.752 INFO:tasks.workunit.client.1.smithi149.stdout:7/522: mkdir d9/dc/d11/d38/dac 0 2024-04-24T08:24:49.754 INFO:tasks.workunit.client.0.smithi090.stdout:3/161: getdents d2/d15 0 2024-04-24T08:24:49.757 INFO:tasks.workunit.client.0.smithi090.stdout:0/234: getdents d1a/d2a 0 2024-04-24T08:24:49.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/617: fsync db/d12/d1e/f4f 0 2024-04-24T08:24:49.759 INFO:tasks.workunit.client.0.smithi090.stdout:5/251: link f7 d8/da/d3a/d36/d50/f53 0 2024-04-24T08:24:49.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/578: mkdir d0/d5/d53/d4c/dca 0 2024-04-24T08:24:49.762 INFO:tasks.workunit.client.0.smithi090.stdout:8/140: write d2/da/d13/d15/d24/f21 [3406261,70752] 0 2024-04-24T08:24:49.764 INFO:tasks.workunit.client.1.smithi149.stdout:1/620: link d0/d1b/d5d/f6c d0/d1b/d4b/da1/fc4 0 2024-04-24T08:24:49.764 INFO:tasks.workunit.client.1.smithi149.stdout:3/579: dread - d0/d5/da/d19/d1b/d22/fbc zero size 2024-04-24T08:24:49.765 INFO:tasks.workunit.client.0.smithi090.stdout:6/238: rename d8/lb to d8/d1d/d29/l45 0 2024-04-24T08:24:49.767 INFO:tasks.workunit.client.1.smithi149.stdout:3/580: write d0/fc [885084,103325] 0 2024-04-24T08:24:49.768 INFO:tasks.workunit.client.0.smithi090.stdout:4/122: getdents d3/df 0 2024-04-24T08:24:49.768 INFO:tasks.workunit.client.1.smithi149.stdout:1/621: write d0/d39/d53/d61/f71 [1213245,700] 0 2024-04-24T08:24:49.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/523: dwrite d9/dc/d7b/fa2 [0,4194304] 0 2024-04-24T08:24:49.771 INFO:tasks.workunit.client.1.smithi149.stdout:6/408: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f88 x:0 0 0 2024-04-24T08:24:49.772 INFO:tasks.workunit.client.0.smithi090.stdout:2/182: link d0/d14/d17/f18 d0/d14/f38 0 2024-04-24T08:24:49.775 INFO:tasks.workunit.client.0.smithi090.stdout:2/183: write d0/d7/f1c [378001,115305] 0 2024-04-24T08:24:49.776 INFO:tasks.workunit.client.0.smithi090.stdout:8/141: dwrite d2/da/d13/d15/d24/f21 [0,4194304] 0 2024-04-24T08:24:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:49 smithi149 ceph-mon[31282]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 MiB/s rd, 183 MiB/s wr, 337 op/s 2024-04-24T08:24:49.778 INFO:tasks.workunit.client.1.smithi149.stdout:8/451: dread d1/d1f/d25/f78 [0,4194304] 0 2024-04-24T08:24:49.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/409: dread - d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 zero size 2024-04-24T08:24:49.781 INFO:tasks.workunit.client.1.smithi149.stdout:8/452: chown d1/de/d18/d32/d65 126 1 2024-04-24T08:24:49.782 INFO:tasks.workunit.client.0.smithi090.stdout:9/175: creat d0/d19/d33/f36 x:0 0 0 2024-04-24T08:24:49.789 INFO:tasks.workunit.client.1.smithi149.stdout:5/618: creat db/d12/d2f/d78/fd4 x:0 0 0 2024-04-24T08:24:49.791 INFO:tasks.workunit.client.0.smithi090.stdout:5/252: symlink d8/da/d3a/d2d/d37/l54 0 2024-04-24T08:24:49.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/181: dread d3/d5/f14 [0,4194304] 0 2024-04-24T08:24:49.791 INFO:tasks.workunit.client.1.smithi149.stdout:5/619: fsync db/d12/fb0 0 2024-04-24T08:24:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:49 smithi090 ceph-mon[25016]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 MiB/s rd, 183 MiB/s wr, 337 op/s 2024-04-24T08:24:49.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/182: dread - d3/d5/d3e/f39 zero size 2024-04-24T08:24:49.794 INFO:tasks.workunit.client.1.smithi149.stdout:0/462: write d2/d9/df/f5d [697415,99510] 0 2024-04-24T08:24:49.796 INFO:tasks.workunit.client.0.smithi090.stdout:4/123: mkdir d3/d18/d1d 0 2024-04-24T08:24:49.797 INFO:tasks.workunit.client.1.smithi149.stdout:6/410: dread d0/f17 [0,4194304] 0 2024-04-24T08:24:49.799 INFO:tasks.workunit.client.1.smithi149.stdout:3/581: mkdir d0/d5/da/d19/dcb 0 2024-04-24T08:24:49.803 INFO:tasks.workunit.client.0.smithi090.stdout:8/142: symlink d2/l2e 0 2024-04-24T08:24:49.805 INFO:tasks.workunit.client.1.smithi149.stdout:8/453: creat d1/de/d18/d32/d63/d76/f9a x:0 0 0 2024-04-24T08:24:49.807 INFO:tasks.workunit.client.0.smithi090.stdout:9/176: mknod d0/c37 0 2024-04-24T08:24:49.807 INFO:tasks.workunit.client.1.smithi149.stdout:7/524: dread d9/dc/d11/d83/da6/f52 [0,4194304] 0 2024-04-24T08:24:49.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/620: mknod db/d23/cd5 0 2024-04-24T08:24:49.811 INFO:tasks.workunit.client.1.smithi149.stdout:2/461: creat d0/d6/db/d39/f90 x:0 0 0 2024-04-24T08:24:49.811 INFO:tasks.workunit.client.1.smithi149.stdout:5/621: write db/d12/f82 [2124410,45969] 0 2024-04-24T08:24:49.813 INFO:tasks.workunit.client.1.smithi149.stdout:0/463: symlink d2/d9/d1a/d36/l8d 0 2024-04-24T08:24:49.814 INFO:tasks.workunit.client.0.smithi090.stdout:1/183: creat d3/f41 x:0 0 0 2024-04-24T08:24:49.815 INFO:tasks.workunit.client.0.smithi090.stdout:1/184: read d3/f2e [242601,49664] 0 2024-04-24T08:24:49.817 INFO:tasks.workunit.client.0.smithi090.stdout:5/253: rename d8/da/d18/d21/f4b to d8/da/d3a/f55 0 2024-04-24T08:24:49.821 INFO:tasks.workunit.client.1.smithi149.stdout:3/582: rename d0/d5/da/d19/f48 to d0/d5/d1a/fcc 0 2024-04-24T08:24:49.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/184: mknod d0/d14/d36/c39 0 2024-04-24T08:24:49.823 INFO:tasks.workunit.client.1.smithi149.stdout:8/454: symlink d1/de/d18/d32/d65/d8b/l9b 0 2024-04-24T08:24:49.824 INFO:tasks.workunit.client.1.smithi149.stdout:3/583: write d0/d5/fc7 [141514,105400] 0 2024-04-24T08:24:49.824 INFO:tasks.workunit.client.0.smithi090.stdout:2/185: write d0/d7/fd [91727,124171] 0 2024-04-24T08:24:49.825 INFO:tasks.workunit.client.0.smithi090.stdout:8/143: creat d2/da/d13/d15/d24/f2f x:0 0 0 2024-04-24T08:24:49.826 INFO:tasks.workunit.client.1.smithi149.stdout:7/525: rmdir d9/dc/d11/d83/da6/d67/d69 39 2024-04-24T08:24:49.827 INFO:tasks.workunit.client.0.smithi090.stdout:8/144: chown d2/da/lf 12604 1 2024-04-24T08:24:49.827 INFO:tasks.workunit.client.1.smithi149.stdout:5/622: dwrite db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:49.830 INFO:tasks.workunit.client.1.smithi149.stdout:5/623: write db/d12/fb0 [15122,86061] 0 2024-04-24T08:24:49.830 INFO:tasks.workunit.client.1.smithi149.stdout:6/411: mknod d0/d1a/d7e/c89 0 2024-04-24T08:24:49.831 INFO:tasks.workunit.client.1.smithi149.stdout:2/462: rmdir d0/d6/db/d39/d3d/d58 39 2024-04-24T08:24:49.832 INFO:tasks.workunit.client.1.smithi149.stdout:5/624: write db/d1f/d7f/fd0 [913379,79509] 0 2024-04-24T08:24:49.834 INFO:tasks.workunit.client.1.smithi149.stdout:0/464: symlink d2/d9/df/d2f/d6c/l8e 0 2024-04-24T08:24:49.836 INFO:tasks.workunit.client.0.smithi090.stdout:5/254: fsync d8/f10 0 2024-04-24T08:24:49.846 INFO:tasks.workunit.client.1.smithi149.stdout:8/455: dwrite d1/f64 [4194304,4194304] 0 2024-04-24T08:24:49.847 INFO:tasks.workunit.client.1.smithi149.stdout:3/584: creat d0/d5/da/d19/d1b/d72/fcd x:0 0 0 2024-04-24T08:24:49.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/177: creat d0/d2a/f38 x:0 0 0 2024-04-24T08:24:49.851 INFO:tasks.workunit.client.1.smithi149.stdout:7/526: chown d9/dc/d11/c32 1 1 2024-04-24T08:24:49.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/162: getdents d2/df/d21 0 2024-04-24T08:24:49.855 INFO:tasks.workunit.client.1.smithi149.stdout:6/412: dwrite d0/d1a/d28/d31/d38/f3a [0,4194304] 0 2024-04-24T08:24:49.857 INFO:tasks.workunit.client.1.smithi149.stdout:3/585: chown d0/d5/da/d59/d79/c87 948911139 1 2024-04-24T08:24:49.857 INFO:tasks.workunit.client.1.smithi149.stdout:2/463: dwrite d0/f22 [0,4194304] 0 2024-04-24T08:24:49.859 INFO:tasks.workunit.client.1.smithi149.stdout:3/586: stat d0/d5/da/d19/d80/dac/lc3 0 2024-04-24T08:24:49.860 INFO:tasks.workunit.client.0.smithi090.stdout:8/145: mkdir d2/da/d13/d30 0 2024-04-24T08:24:49.861 INFO:tasks.workunit.client.0.smithi090.stdout:0/235: getdents d1a/d4b 0 2024-04-24T08:24:49.862 INFO:tasks.workunit.client.0.smithi090.stdout:4/124: truncate d3/f9 3699701 0 2024-04-24T08:24:49.863 INFO:tasks.workunit.client.1.smithi149.stdout:0/465: link d2/d9/df/d40/f88 d2/d9/df/d2f/f8f 0 2024-04-24T08:24:49.864 INFO:tasks.workunit.client.0.smithi090.stdout:5/255: mkdir d8/d56 0 2024-04-24T08:24:49.864 INFO:tasks.workunit.client.1.smithi149.stdout:3/587: truncate d0/d5/da/fa7 1047671 0 2024-04-24T08:24:49.865 INFO:tasks.workunit.client.0.smithi090.stdout:0/236: chown ff 16 1 2024-04-24T08:24:49.868 INFO:tasks.workunit.client.0.smithi090.stdout:4/125: chown d3/df 67 1 2024-04-24T08:24:49.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/466: mknod d2/c90 0 2024-04-24T08:24:49.871 INFO:tasks.workunit.client.1.smithi149.stdout:7/527: mkdir d9/dc/d11/d83/da6/d67/d69/dad 0 2024-04-24T08:24:49.872 INFO:tasks.workunit.client.0.smithi090.stdout:1/185: dwrite d3/d5/f14 [0,4194304] 0 2024-04-24T08:24:49.873 INFO:tasks.workunit.client.1.smithi149.stdout:7/528: write d9/d43/f7e [10554943,59029] 0 2024-04-24T08:24:49.874 INFO:tasks.workunit.client.0.smithi090.stdout:0/237: symlink d1a/d4b/l4e 0 2024-04-24T08:24:49.877 INFO:tasks.workunit.client.0.smithi090.stdout:1/186: stat d3/d5/da/c3c 0 2024-04-24T08:24:49.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/456: dwrite d1/de/d18/d32/d65/f81 [0,4194304] 0 2024-04-24T08:24:49.881 INFO:tasks.workunit.client.1.smithi149.stdout:8/457: write d1/d1f/d5e/f73 [2880616,72313] 0 2024-04-24T08:24:49.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/529: creat d9/dc/d11/d26/fae x:0 0 0 2024-04-24T08:24:49.886 INFO:tasks.workunit.client.1.smithi149.stdout:0/467: mkdir d2/d9/df/d2f/d6c/d72/d91 0 2024-04-24T08:24:49.887 INFO:tasks.workunit.client.0.smithi090.stdout:3/163: dwrite d2/f17 [0,4194304] 0 2024-04-24T08:24:49.888 INFO:tasks.workunit.client.0.smithi090.stdout:9/178: dwrite d0/f34 [0,4194304] 0 2024-04-24T08:24:49.890 INFO:tasks.workunit.client.0.smithi090.stdout:8/146: rmdir d2/da/d13/d30 0 2024-04-24T08:24:49.893 INFO:tasks.workunit.client.0.smithi090.stdout:4/126: rmdir d3 39 2024-04-24T08:24:49.895 INFO:tasks.workunit.client.0.smithi090.stdout:1/187: mkdir d3/d5/d3e/d42 0 2024-04-24T08:24:49.896 INFO:tasks.workunit.client.1.smithi149.stdout:8/458: mkdir d1/d1f/d5e/d9c 0 2024-04-24T08:24:49.897 INFO:tasks.workunit.client.1.smithi149.stdout:6/413: write d0/f4 [2481053,87179] 0 2024-04-24T08:24:49.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/256: creat d8/da/d3a/d36/d50/f57 x:0 0 0 2024-04-24T08:24:49.903 INFO:tasks.workunit.client.1.smithi149.stdout:3/588: getdents d0/d5/da/d19/d80/dac 0 2024-04-24T08:24:49.906 INFO:tasks.workunit.client.1.smithi149.stdout:0/468: mkdir d2/d9/df/d92 0 2024-04-24T08:24:49.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/127: chown d3 3976735 1 2024-04-24T08:24:49.910 INFO:tasks.workunit.client.1.smithi149.stdout:0/469: write d2/d9/d1a/f25 [4245425,32948] 0 2024-04-24T08:24:49.912 INFO:tasks.workunit.client.0.smithi090.stdout:9/179: dwrite d0/d19/f28 [4194304,4194304] 0 2024-04-24T08:24:49.912 INFO:tasks.workunit.client.0.smithi090.stdout:8/147: symlink d2/da/l31 0 2024-04-24T08:24:49.914 INFO:tasks.workunit.client.1.smithi149.stdout:0/470: chown d2/d9/df/d40/l8c 23631936 1 2024-04-24T08:24:49.914 INFO:tasks.workunit.client.0.smithi090.stdout:4/128: write d3/f15 [2241663,39949] 0 2024-04-24T08:24:49.914 INFO:tasks.workunit.client.0.smithi090.stdout:2/186: dwrite d0/ff [0,4194304] 0 2024-04-24T08:24:49.917 INFO:tasks.workunit.client.1.smithi149.stdout:7/530: truncate d9/dc/d11/d26/f28 3386176 0 2024-04-24T08:24:49.918 INFO:tasks.workunit.client.0.smithi090.stdout:5/257: fsync d8/da/d14/d2f/f47 0 2024-04-24T08:24:49.919 INFO:tasks.workunit.client.0.smithi090.stdout:4/129: write d3/f16 [1012651,118538] 0 2024-04-24T08:24:49.919 INFO:tasks.workunit.client.1.smithi149.stdout:3/589: mknod d0/d5/da/d5a/d9c/cce 0 2024-04-24T08:24:49.924 INFO:tasks.workunit.client.0.smithi090.stdout:3/164: creat d2/f2f x:0 0 0 2024-04-24T08:24:49.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/471: truncate d2/d9/df/d48/f4c 803672 0 2024-04-24T08:24:49.926 INFO:tasks.workunit.client.0.smithi090.stdout:5/258: stat d8/d25/f26 0 2024-04-24T08:24:49.927 INFO:tasks.workunit.client.0.smithi090.stdout:9/180: unlink d0/f10 0 2024-04-24T08:24:49.928 INFO:tasks.workunit.client.0.smithi090.stdout:9/181: chown d0/c37 105 1 2024-04-24T08:24:49.930 INFO:tasks.workunit.client.1.smithi149.stdout:9/548: sync 2024-04-24T08:24:49.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/501: sync 2024-04-24T08:24:49.936 INFO:tasks.workunit.client.0.smithi090.stdout:8/148: dwrite d2/da/d13/d15/d24/f2f [0,4194304] 0 2024-04-24T08:24:49.937 INFO:tasks.workunit.client.1.smithi149.stdout:3/590: creat d0/d5/da/d59/fcf x:0 0 0 2024-04-24T08:24:49.940 INFO:tasks.workunit.client.0.smithi090.stdout:2/187: creat d0/d7/d28/f3a x:0 0 0 2024-04-24T08:24:49.943 INFO:tasks.workunit.client.1.smithi149.stdout:0/472: mknod d2/d3c/c93 0 2024-04-24T08:24:49.944 INFO:tasks.workunit.client.0.smithi090.stdout:1/188: rmdir d3/d5/d3e/d42 0 2024-04-24T08:24:49.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/473: stat d2/d9/df/d2f/c31 0 2024-04-24T08:24:49.948 INFO:tasks.workunit.client.0.smithi090.stdout:5/259: dwrite d8/da/d18/f22 [0,4194304] 0 2024-04-24T08:24:49.949 INFO:tasks.workunit.client.1.smithi149.stdout:7/531: mknod d9/d43/d4f/d66/caf 0 2024-04-24T08:24:49.949 INFO:tasks.workunit.client.1.smithi149.stdout:0/474: chown d2/d9/df/d40/c45 33607 1 2024-04-24T08:24:49.949 INFO:tasks.workunit.client.1.smithi149.stdout:0/475: chown d2 68904 1 2024-04-24T08:24:49.950 INFO:tasks.workunit.client.1.smithi149.stdout:8/459: write d1/d1f/d31/f67 [897601,88419] 0 2024-04-24T08:24:49.953 INFO:tasks.workunit.client.1.smithi149.stdout:9/549: fsync d1/da/d4a/dad/fc2 0 2024-04-24T08:24:49.954 INFO:tasks.workunit.client.0.smithi090.stdout:3/165: creat d2/df/d21/f30 x:0 0 0 2024-04-24T08:24:49.958 INFO:tasks.workunit.client.0.smithi090.stdout:0/238: dwrite f12 [0,4194304] 0 2024-04-24T08:24:49.959 INFO:tasks.workunit.client.1.smithi149.stdout:3/591: fsync d0/f11 0 2024-04-24T08:24:49.960 INFO:tasks.workunit.client.0.smithi090.stdout:9/182: symlink d0/d19/d20/l39 0 2024-04-24T08:24:49.965 INFO:tasks.workunit.client.1.smithi149.stdout:0/476: truncate d2/d9/df/d40/d4a/d4e/f6d 275664 0 2024-04-24T08:24:49.966 INFO:tasks.workunit.client.1.smithi149.stdout:2/464: dread d0/f4d [0,4194304] 0 2024-04-24T08:24:49.975 INFO:tasks.workunit.client.1.smithi149.stdout:4/502: dwrite d2/d24/d2e/f51 [4194304,4194304] 0 2024-04-24T08:24:49.976 INFO:tasks.workunit.client.1.smithi149.stdout:9/550: symlink d1/da/d13/d74/lc3 0 2024-04-24T08:24:49.980 INFO:tasks.workunit.client.1.smithi149.stdout:9/551: readlink d1/da/d4a/l90 0 2024-04-24T08:24:49.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/189: mknod d3/d5/da/c43 0 2024-04-24T08:24:49.985 INFO:tasks.workunit.client.0.smithi090.stdout:1/190: dread - d3/d5/da/dc/f40 zero size 2024-04-24T08:24:49.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/532: symlink d9/dc/d11/lb0 0 2024-04-24T08:24:49.987 INFO:tasks.workunit.client.1.smithi149.stdout:7/533: fdatasync d9/dc/f17 0 2024-04-24T08:24:49.991 INFO:tasks.workunit.client.1.smithi149.stdout:7/534: read - d9/dc/de/d44/fab zero size 2024-04-24T08:24:49.992 INFO:tasks.workunit.client.1.smithi149.stdout:7/535: stat d9/dc/d11/c55 0 2024-04-24T08:24:49.993 INFO:tasks.workunit.client.0.smithi090.stdout:3/166: creat d2/d15/f31 x:0 0 0 2024-04-24T08:24:49.994 INFO:tasks.workunit.client.1.smithi149.stdout:6/414: dread d0/d1a/f27 [4194304,4194304] 0 2024-04-24T08:24:49.994 INFO:tasks.workunit.client.0.smithi090.stdout:2/188: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:49.995 INFO:tasks.workunit.client.1.smithi149.stdout:2/465: unlink d0/d6/db/d28/l33 0 2024-04-24T08:24:49.995 INFO:tasks.workunit.client.1.smithi149.stdout:6/415: readlink d0/d1a/l83 0 2024-04-24T08:24:49.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/239: symlink d1a/d2a/d3c/l4f 0 2024-04-24T08:24:49.998 INFO:tasks.workunit.client.1.smithi149.stdout:6/416: write d0/fa [1017847,42805] 0 2024-04-24T08:24:50.000 INFO:tasks.workunit.client.1.smithi149.stdout:4/503: dread d2/d18/f20 [0,4194304] 0 2024-04-24T08:24:50.000 INFO:tasks.workunit.client.1.smithi149.stdout:3/592: dread d0/d5/d53/f3f [0,4194304] 0 2024-04-24T08:24:50.002 INFO:tasks.workunit.client.1.smithi149.stdout:4/504: dread - d2/d24/d2e/d38/f9c zero size 2024-04-24T08:24:50.004 INFO:tasks.workunit.client.0.smithi090.stdout:9/183: mkdir d0/d19/d20/d3a 0 2024-04-24T08:24:50.004 INFO:tasks.workunit.client.1.smithi149.stdout:3/593: dread - d0/d5/da/d19/f9f zero size 2024-04-24T08:24:50.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/184: chown d0/d19/d20/d3a 2355 1 2024-04-24T08:24:50.005 INFO:tasks.workunit.client.0.smithi090.stdout:2/189: dread d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:50.007 INFO:tasks.workunit.client.1.smithi149.stdout:8/460: rename d1/d1f/d31/l93 to d1/de/d18/d32/d65/d8b/d7d/l9d 0 2024-04-24T08:24:50.011 INFO:tasks.workunit.client.1.smithi149.stdout:3/594: chown d0/d5/d53/d34/d3a/l54 16884070 1 2024-04-24T08:24:50.011 INFO:tasks.workunit.client.1.smithi149.stdout:9/552: fdatasync d1/d2/d34/fbe 0 2024-04-24T08:24:50.015 INFO:tasks.workunit.client.1.smithi149.stdout:7/536: dwrite d9/dc/f35 [0,4194304] 0 2024-04-24T08:24:50.018 INFO:tasks.workunit.client.0.smithi090.stdout:3/167: rmdir d2/df 39 2024-04-24T08:24:50.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/260: dread d8/da/f20 [0,4194304] 0 2024-04-24T08:24:50.024 INFO:tasks.workunit.client.0.smithi090.stdout:0/240: symlink d1a/d2a/d35/l50 0 2024-04-24T08:24:50.026 INFO:tasks.workunit.client.0.smithi090.stdout:1/191: dread d3/d5/f16 [0,4194304] 0 2024-04-24T08:24:50.028 INFO:tasks.workunit.client.1.smithi149.stdout:8/461: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:24:50.031 INFO:tasks.workunit.client.0.smithi090.stdout:8/149: creat d2/da/d13/d15/f32 x:0 0 0 2024-04-24T08:24:50.037 INFO:tasks.workunit.client.0.smithi090.stdout:4/130: getdents d3/d18 0 2024-04-24T08:24:50.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/462: mkdir d1/de/d18/d32/d63/d9e 0 2024-04-24T08:24:50.045 INFO:tasks.workunit.client.1.smithi149.stdout:2/466: mknod d0/d6/db/d39/d3d/d58/c91 0 2024-04-24T08:24:50.055 INFO:tasks.workunit.client.1.smithi149.stdout:2/467: symlink d0/d6/db/d39/d78/d7c/l92 0 2024-04-24T08:24:50.056 INFO:tasks.workunit.client.1.smithi149.stdout:9/553: dread d1/da/dd/f69 [0,4194304] 0 2024-04-24T08:24:50.058 INFO:tasks.workunit.client.1.smithi149.stdout:8/463: mkdir d1/de/d18/d91/d9f 0 2024-04-24T08:24:50.061 INFO:tasks.workunit.client.1.smithi149.stdout:7/537: rename d9/d43/l5a to d9/dc/d11/d83/da6/d67/lb1 0 2024-04-24T08:24:50.063 INFO:tasks.workunit.client.1.smithi149.stdout:9/554: mkdir d1/d2/d33/d5d/d83/dbf/dc4 0 2024-04-24T08:24:50.066 INFO:tasks.workunit.client.1.smithi149.stdout:9/555: read d1/d2/fa0 [6545933,42779] 0 2024-04-24T08:24:50.067 INFO:tasks.workunit.client.1.smithi149.stdout:2/468: rename d0/d6/db/f7e to d0/d6/db/d27/d64/d66/f93 0 2024-04-24T08:24:50.069 INFO:tasks.workunit.client.1.smithi149.stdout:9/556: truncate d1/da/dd/fac 327051 0 2024-04-24T08:24:50.071 INFO:tasks.workunit.client.1.smithi149.stdout:2/469: mknod d0/d6/db/d39/d78/c94 0 2024-04-24T08:24:50.074 INFO:tasks.workunit.client.1.smithi149.stdout:9/557: creat d1/da/d13/d74/dbb/fc5 x:0 0 0 2024-04-24T08:24:50.075 INFO:tasks.workunit.client.1.smithi149.stdout:8/464: dwrite d1/d1f/f8d [0,4194304] 0 2024-04-24T08:24:50.075 INFO:tasks.workunit.client.1.smithi149.stdout:2/470: unlink d0/d6/db/d39/d3d/c54 0 2024-04-24T08:24:50.078 INFO:tasks.workunit.client.1.smithi149.stdout:9/558: truncate d1/d2/d34/f78 944076 0 2024-04-24T08:24:50.078 INFO:tasks.workunit.client.1.smithi149.stdout:8/465: write d1/de/d18/d32/d65/f81 [3072498,44583] 0 2024-04-24T08:24:50.081 INFO:tasks.workunit.client.1.smithi149.stdout:9/559: symlink d1/d2/lc6 0 2024-04-24T08:24:50.082 INFO:tasks.workunit.client.1.smithi149.stdout:2/471: chown d0/d6/db/f72 4123866 1 2024-04-24T08:24:50.083 INFO:tasks.workunit.client.1.smithi149.stdout:8/466: creat d1/d1f/d5e/fa0 x:0 0 0 2024-04-24T08:24:50.086 INFO:tasks.workunit.client.1.smithi149.stdout:9/560: mknod d1/da/d4a/cc7 0 2024-04-24T08:24:50.088 INFO:tasks.workunit.client.1.smithi149.stdout:2/472: symlink d0/d6/db/d39/d3d/l95 0 2024-04-24T08:24:50.096 INFO:tasks.workunit.client.1.smithi149.stdout:2/473: truncate d0/d6/db/d39/d3d/d44/d57/f71 674489 0 2024-04-24T08:24:50.103 INFO:tasks.workunit.client.1.smithi149.stdout:9/561: dwrite d1/d2/f2b [4194304,4194304] 0 2024-04-24T08:24:50.105 INFO:tasks.workunit.client.1.smithi149.stdout:9/562: write d1/da/f8 [1071012,123261] 0 2024-04-24T08:24:50.107 INFO:tasks.workunit.client.1.smithi149.stdout:9/563: readlink d1/da/d13/d74/lc3 0 2024-04-24T08:24:50.109 INFO:tasks.workunit.client.1.smithi149.stdout:2/474: dread d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:24:50.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/192: rmdir d3 39 2024-04-24T08:24:50.117 INFO:tasks.workunit.client.0.smithi090.stdout:4/131: creat d3/df/f1e x:0 0 0 2024-04-24T08:24:50.117 INFO:tasks.workunit.client.0.smithi090.stdout:3/168: creat d2/df/d21/f32 x:0 0 0 2024-04-24T08:24:50.124 INFO:tasks.workunit.client.0.smithi090.stdout:5/261: dread d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:24:50.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/262: truncate d8/d25/f3d 600988 0 2024-04-24T08:24:50.129 INFO:tasks.workunit.client.0.smithi090.stdout:8/150: dread d2/f10 [0,4194304] 0 2024-04-24T08:24:50.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/263: dread d8/da/f1b [0,4194304] 0 2024-04-24T08:24:50.143 INFO:tasks.workunit.client.1.smithi149.stdout:9/564: dread d1/de/f22 [0,4194304] 0 2024-04-24T08:24:50.145 INFO:tasks.workunit.client.1.smithi149.stdout:9/565: fdatasync d1/da/dd/d79/fa6 0 2024-04-24T08:24:50.161 INFO:tasks.workunit.client.1.smithi149.stdout:5/625: sync 2024-04-24T08:24:50.161 INFO:tasks.workunit.client.1.smithi149.stdout:1/622: sync 2024-04-24T08:24:50.162 INFO:tasks.workunit.client.1.smithi149.stdout:5/626: readlink db/d12/d22/l2e 0 2024-04-24T08:24:50.194 INFO:tasks.workunit.client.1.smithi149.stdout:2/475: creat d0/d6/db/d39/d3d/d44/f96 x:0 0 0 2024-04-24T08:24:50.197 INFO:tasks.workunit.client.1.smithi149.stdout:2/476: read - d0/d6/db/d39/d3d/d44/f96 zero size 2024-04-24T08:24:50.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/241: write d1a/f1b [2264580,33023] 0 2024-04-24T08:24:50.199 INFO:tasks.workunit.client.1.smithi149.stdout:9/566: unlink d1/de/d12/f96 0 2024-04-24T08:24:50.200 INFO:tasks.workunit.client.0.smithi090.stdout:0/242: write d1a/f3d [1394848,474] 0 2024-04-24T08:24:50.201 INFO:tasks.workunit.client.0.smithi090.stdout:9/185: rmdir d0/d19/d20/d3a 0 2024-04-24T08:24:50.201 INFO:tasks.workunit.client.1.smithi149.stdout:2/477: fsync d0/d6/db/d39/d78/d7c/f82 0 2024-04-24T08:24:50.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/623: unlink d0/d1b/l38 0 2024-04-24T08:24:50.204 INFO:tasks.workunit.client.1.smithi149.stdout:5/627: mknod db/d12/d60/db2/d83/dc8/cd6 0 2024-04-24T08:24:50.204 INFO:tasks.workunit.client.1.smithi149.stdout:1/624: fdatasync d0/d5/f8 0 2024-04-24T08:24:50.204 INFO:tasks.workunit.client.0.smithi090.stdout:4/132: rmdir d3 39 2024-04-24T08:24:50.205 INFO:tasks.workunit.client.0.smithi090.stdout:9/186: write d0/dc/fd [2885613,14402] 0 2024-04-24T08:24:50.206 INFO:tasks.workunit.client.0.smithi090.stdout:3/169: mkdir d2/d25/d33 0 2024-04-24T08:24:50.207 INFO:tasks.workunit.client.1.smithi149.stdout:9/567: mkdir d1/d2/dc8 0 2024-04-24T08:24:50.210 INFO:tasks.workunit.client.0.smithi090.stdout:5/264: symlink d8/da/d14/d2f/l58 0 2024-04-24T08:24:50.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/170: stat d2/d25/d2c 0 2024-04-24T08:24:50.212 INFO:tasks.workunit.client.1.smithi149.stdout:1/625: fdatasync d0/d5/db/d18/d4d/d50/daf/f56 0 2024-04-24T08:24:50.212 INFO:tasks.workunit.client.1.smithi149.stdout:2/478: rename d0/d6/db/d39/d3d/d44/f4c to d0/d6/db/d39/d78/d7c/f97 0 2024-04-24T08:24:50.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/243: rmdir d1a/d2a/d35 39 2024-04-24T08:24:50.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/568: mkdir d1/d2/dc8/dc9 0 2024-04-24T08:24:50.220 INFO:tasks.workunit.client.0.smithi090.stdout:4/133: write d3/f16 [1409830,111628] 0 2024-04-24T08:24:50.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/193: symlink d3/d1d/d34/l44 0 2024-04-24T08:24:50.220 INFO:tasks.workunit.client.1.smithi149.stdout:9/569: stat d1/d2/d15/d1c/f62 0 2024-04-24T08:24:50.223 INFO:tasks.workunit.client.1.smithi149.stdout:5/628: dwrite db/d12/d2f/f46 [0,4194304] 0 2024-04-24T08:24:50.227 INFO:tasks.workunit.client.1.smithi149.stdout:1/626: mkdir d0/d39/da0/db8/dc5 0 2024-04-24T08:24:50.228 INFO:tasks.workunit.client.0.smithi090.stdout:8/151: dread d2/da/d13/f28 [0,4194304] 0 2024-04-24T08:24:50.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/171: rename d2/l13 to d2/df/d1a/l34 0 2024-04-24T08:24:50.231 INFO:tasks.workunit.client.0.smithi090.stdout:9/187: dread d0/d19/f28 [4194304,4194304] 0 2024-04-24T08:24:50.233 INFO:tasks.workunit.client.0.smithi090.stdout:3/172: fdatasync d2/d15/f31 0 2024-04-24T08:24:50.233 INFO:tasks.workunit.client.0.smithi090.stdout:0/244: fsync d1a/f3f 0 2024-04-24T08:24:50.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/152: write d2/da/d13/d15/d24/f2f [5193398,128115] 0 2024-04-24T08:24:50.235 INFO:tasks.workunit.client.1.smithi149.stdout:1/627: symlink d0/d5/db/d82/lc6 0 2024-04-24T08:24:50.236 INFO:tasks.workunit.client.1.smithi149.stdout:9/570: dread d1/da/dd/d79/f63 [0,4194304] 0 2024-04-24T08:24:50.237 INFO:tasks.workunit.client.0.smithi090.stdout:3/173: write d2/d15/f2e [949339,54946] 0 2024-04-24T08:24:50.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/194: symlink d3/l45 0 2024-04-24T08:24:50.239 INFO:tasks.workunit.client.1.smithi149.stdout:9/571: dread - d1/d4c/d91/fb9 zero size 2024-04-24T08:24:50.242 INFO:tasks.workunit.client.1.smithi149.stdout:1/628: creat d0/d39/d53/d61/fc7 x:0 0 0 2024-04-24T08:24:50.243 INFO:tasks.workunit.client.0.smithi090.stdout:8/153: read d2/da/f1c [3886985,92397] 0 2024-04-24T08:24:50.245 INFO:tasks.workunit.client.1.smithi149.stdout:5/629: dwrite db/d12/d22/d45/d5d/f63 [0,4194304] 0 2024-04-24T08:24:50.245 INFO:tasks.workunit.client.0.smithi090.stdout:1/195: fdatasync d3/d5/da/dc/f3a 0 2024-04-24T08:24:50.246 INFO:tasks.workunit.client.0.smithi090.stdout:5/265: write d8/f9 [4460235,96499] 0 2024-04-24T08:24:50.248 INFO:tasks.workunit.client.0.smithi090.stdout:9/188: dread - d0/dc/f1f zero size 2024-04-24T08:24:50.250 INFO:tasks.workunit.client.0.smithi090.stdout:4/134: creat d3/d18/d1d/f1f x:0 0 0 2024-04-24T08:24:50.253 INFO:tasks.workunit.client.0.smithi090.stdout:8/154: chown d2/da/l2b 26 1 2024-04-24T08:24:50.254 INFO:tasks.workunit.client.0.smithi090.stdout:8/155: dread - d2/da/d13/f18 zero size 2024-04-24T08:24:50.255 INFO:tasks.workunit.client.1.smithi149.stdout:1/629: rename d0/d1b/f64 to d0/d5/d4f/d60/fc8 0 2024-04-24T08:24:50.259 INFO:tasks.workunit.client.0.smithi090.stdout:7/253: sync 2024-04-24T08:24:50.259 INFO:tasks.workunit.client.0.smithi090.stdout:6/239: sync 2024-04-24T08:24:50.260 INFO:tasks.workunit.client.0.smithi090.stdout:3/174: mknod d2/d25/c35 0 2024-04-24T08:24:50.261 INFO:tasks.workunit.client.1.smithi149.stdout:1/630: unlink d0/d1b/d5d/f85 0 2024-04-24T08:24:50.263 INFO:tasks.workunit.client.0.smithi090.stdout:1/196: mknod d3/d1d/c46 0 2024-04-24T08:24:50.264 INFO:tasks.workunit.client.1.smithi149.stdout:5/630: dwrite db/d12/d1e/d65/fb4 [0,4194304] 0 2024-04-24T08:24:50.265 INFO:tasks.workunit.client.0.smithi090.stdout:5/266: mkdir d8/d59 0 2024-04-24T08:24:50.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/572: write d1/d2/d34/fbe [1010019,43497] 0 2024-04-24T08:24:50.267 INFO:tasks.workunit.client.0.smithi090.stdout:5/267: chown d8/d25/f26 1 1 2024-04-24T08:24:50.268 INFO:tasks.workunit.client.1.smithi149.stdout:9/573: stat d1/d2/d15 0 2024-04-24T08:24:50.271 INFO:tasks.workunit.client.1.smithi149.stdout:1/631: rename d0/d5/c17 to d0/d39/da0/db8/dc5/cc9 0 2024-04-24T08:24:50.271 INFO:tasks.workunit.client.1.smithi149.stdout:5/631: creat db/d12/d60/fd7 x:0 0 0 2024-04-24T08:24:50.275 INFO:tasks.workunit.client.0.smithi090.stdout:6/240: dread d8/fa [0,4194304] 0 2024-04-24T08:24:50.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/189: rename d0/c17 to d0/c3b 0 2024-04-24T08:24:50.279 INFO:tasks.workunit.client.0.smithi090.stdout:0/245: dread d1a/f1d [0,4194304] 0 2024-04-24T08:24:50.280 INFO:tasks.workunit.client.0.smithi090.stdout:8/156: rename d2/da/le to d2/da/d13/l33 0 2024-04-24T08:24:50.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/175: rmdir d2/df 39 2024-04-24T08:24:50.282 INFO:tasks.workunit.client.0.smithi090.stdout:3/176: chown d2/d15/f31 55900619 1 2024-04-24T08:24:50.283 INFO:tasks.workunit.client.1.smithi149.stdout:9/574: mkdir d1/d2/d15/d1c/dca 0 2024-04-24T08:24:50.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/268: mknod d8/da/d3a/d2d/c5a 0 2024-04-24T08:24:50.286 INFO:tasks.workunit.client.1.smithi149.stdout:5/632: symlink db/d12/d1e/ld8 0 2024-04-24T08:24:50.289 INFO:tasks.workunit.client.1.smithi149.stdout:1/632: creat d0/d5/df/fca x:0 0 0 2024-04-24T08:24:50.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/197: rename d3/d5/d3e/f39 to d3/d5/da/dc/f47 0 2024-04-24T08:24:50.292 INFO:tasks.workunit.client.0.smithi090.stdout:7/254: symlink db/d54/l63 0 2024-04-24T08:24:50.293 INFO:tasks.workunit.client.1.smithi149.stdout:5/633: symlink db/d12/d22/ld9 0 2024-04-24T08:24:50.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/157: mkdir d2/da/d13/d34 0 2024-04-24T08:24:50.294 INFO:tasks.workunit.client.1.smithi149.stdout:5/634: fdatasync db/d12/d2f/f57 0 2024-04-24T08:24:50.295 INFO:tasks.workunit.client.1.smithi149.stdout:5/635: stat db/f14 0 2024-04-24T08:24:50.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/177: mkdir d2/d15/d36 0 2024-04-24T08:24:50.298 INFO:tasks.workunit.client.0.smithi090.stdout:4/135: symlink d3/df/l20 0 2024-04-24T08:24:50.298 INFO:tasks.workunit.client.0.smithi090.stdout:3/178: chown d2/ce 924 1 2024-04-24T08:24:50.299 INFO:tasks.workunit.client.0.smithi090.stdout:3/179: fdatasync d2/f17 0 2024-04-24T08:24:50.299 INFO:tasks.workunit.client.0.smithi090.stdout:6/241: fdatasync d8/dc/f11 0 2024-04-24T08:24:50.302 INFO:tasks.workunit.client.0.smithi090.stdout:4/136: truncate d3/d18/d1d/f1f 236114 0 2024-04-24T08:24:50.303 INFO:tasks.workunit.client.1.smithi149.stdout:0/477: sync 2024-04-24T08:24:50.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/575: dwrite d1/de/d12/f3d [0,4194304] 0 2024-04-24T08:24:50.307 INFO:tasks.workunit.client.0.smithi090.stdout:9/190: creat d0/d19/d20/d32/f3c x:0 0 0 2024-04-24T08:24:50.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/198: dwrite d3/d5/d3e/f18 [8388608,4194304] 0 2024-04-24T08:24:50.309 INFO:tasks.workunit.client.0.smithi090.stdout:0/246: mknod d1a/c51 0 2024-04-24T08:24:50.315 INFO:tasks.workunit.client.0.smithi090.stdout:5/269: rename f2 to d8/da/d14/d2f/d44/f5b 0 2024-04-24T08:24:50.318 INFO:tasks.workunit.client.0.smithi090.stdout:6/242: dwrite d8/d1d/f1e [0,4194304] 0 2024-04-24T08:24:50.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/255: rmdir db/d17/d29 39 2024-04-24T08:24:50.321 INFO:tasks.workunit.client.1.smithi149.stdout:9/576: symlink d1/da/d4a/dad/lcb 0 2024-04-24T08:24:50.322 INFO:tasks.workunit.client.1.smithi149.stdout:5/636: dwrite db/d1f/d7f/fd0 [0,4194304] 0 2024-04-24T08:24:50.328 INFO:tasks.workunit.client.1.smithi149.stdout:9/577: write d1/d2/d15/f81 [1796256,88305] 0 2024-04-24T08:24:50.328 INFO:tasks.workunit.client.1.smithi149.stdout:1/633: dread d0/d5/db/d63/f6e [0,4194304] 0 2024-04-24T08:24:50.328 INFO:tasks.workunit.client.0.smithi090.stdout:8/158: creat d2/da/f35 x:0 0 0 2024-04-24T08:24:50.331 INFO:tasks.workunit.client.1.smithi149.stdout:9/578: fdatasync d1/d2/d33/d5d/d83/dbf/fc1 0 2024-04-24T08:24:50.332 INFO:tasks.workunit.client.0.smithi090.stdout:2/190: sync 2024-04-24T08:24:50.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/191: truncate d0/d7/d24/f27 724751 0 2024-04-24T08:24:50.338 INFO:tasks.workunit.client.1.smithi149.stdout:0/478: dread d2/f7f [0,4194304] 0 2024-04-24T08:24:50.338 INFO:tasks.workunit.client.0.smithi090.stdout:2/192: truncate d0/d14/f34 859611 0 2024-04-24T08:24:50.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/137: chown d3/df/f1b 1592 1 2024-04-24T08:24:50.339 INFO:tasks.workunit.client.0.smithi090.stdout:9/191: fsync d0/d19/f28 0 2024-04-24T08:24:50.340 INFO:tasks.workunit.client.0.smithi090.stdout:1/199: symlink d3/d1d/d34/l48 0 2024-04-24T08:24:50.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/200: readlink d3/l23 0 2024-04-24T08:24:50.347 INFO:tasks.workunit.client.0.smithi090.stdout:5/270: symlink d8/da/d3a/l5c 0 2024-04-24T08:24:50.348 INFO:tasks.workunit.client.0.smithi090.stdout:5/271: readlink d8/da/d14/d2f/l58 0 2024-04-24T08:24:50.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/272: readlink d8/da/l11 0 2024-04-24T08:24:50.351 INFO:tasks.workunit.client.0.smithi090.stdout:7/256: mknod db/d17/d1b/c64 0 2024-04-24T08:24:50.355 INFO:tasks.workunit.client.0.smithi090.stdout:3/180: mkdir d2/df/d1a/d37 0 2024-04-24T08:24:50.357 INFO:tasks.workunit.client.0.smithi090.stdout:2/193: rmdir d0/d2e 39 2024-04-24T08:24:50.359 INFO:tasks.workunit.client.0.smithi090.stdout:4/138: mknod d3/c21 0 2024-04-24T08:24:50.360 INFO:tasks.workunit.client.0.smithi090.stdout:9/192: mkdir d0/dc/d24/d3d 0 2024-04-24T08:24:50.362 INFO:tasks.workunit.client.0.smithi090.stdout:9/193: stat d0/dc/f29 0 2024-04-24T08:24:50.362 INFO:tasks.workunit.client.0.smithi090.stdout:1/201: symlink d3/d25/d30/l49 0 2024-04-24T08:24:50.363 INFO:tasks.workunit.client.0.smithi090.stdout:1/202: fdatasync d3/f26 0 2024-04-24T08:24:50.363 INFO:tasks.workunit.client.0.smithi090.stdout:6/243: dwrite d8/dc/f3a [0,4194304] 0 2024-04-24T08:24:50.363 INFO:tasks.workunit.client.0.smithi090.stdout:9/194: fsync d0/d19/f25 0 2024-04-24T08:24:50.364 INFO:tasks.workunit.client.0.smithi090.stdout:9/195: chown d0/dc/d24 13349725 1 2024-04-24T08:24:50.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/244: truncate d8/d1d/d29/f3b 661597 0 2024-04-24T08:24:50.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/257: creat db/d17/d39/f65 x:0 0 0 2024-04-24T08:24:50.369 INFO:tasks.workunit.client.0.smithi090.stdout:8/159: link d2/da/d13/d15/f32 d2/f36 0 2024-04-24T08:24:50.374 INFO:tasks.workunit.client.0.smithi090.stdout:4/139: mkdir d3/d18/d1d/d22 0 2024-04-24T08:24:50.375 INFO:tasks.workunit.client.1.smithi149.stdout:5/637: fsync db/d12/f72 0 2024-04-24T08:24:50.377 INFO:tasks.workunit.client.0.smithi090.stdout:0/247: rename f15 to d1a/f52 0 2024-04-24T08:24:50.378 INFO:tasks.workunit.client.1.smithi149.stdout:1/634: unlink d0/d39/l55 0 2024-04-24T08:24:50.379 INFO:tasks.workunit.client.0.smithi090.stdout:0/248: readlink d1a/l2d 0 2024-04-24T08:24:50.379 INFO:tasks.workunit.client.1.smithi149.stdout:9/579: read d1/d2/d33/f5f [290016,110776] 0 2024-04-24T08:24:50.380 INFO:tasks.workunit.client.0.smithi090.stdout:1/203: readlink d3/d5/l1c 0 2024-04-24T08:24:50.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/479: rename d2/d3c/c7b to d2/d9/df/d2f/d6c/d72/d91/c94 0 2024-04-24T08:24:50.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/196: chown d0/d19/d20/l35 244793 1 2024-04-24T08:24:50.385 INFO:tasks.workunit.client.1.smithi149.stdout:5/638: creat db/d12/d60/d8f/da0/fda x:0 0 0 2024-04-24T08:24:50.385 INFO:tasks.workunit.client.0.smithi090.stdout:9/197: stat d0/dc/d24 0 2024-04-24T08:24:50.386 INFO:tasks.workunit.client.0.smithi090.stdout:6/245: fdatasync d8/f16 0 2024-04-24T08:24:50.388 INFO:tasks.workunit.client.1.smithi149.stdout:1/635: read d0/d1b/db5/f57 [2517150,81035] 0 2024-04-24T08:24:50.389 INFO:tasks.workunit.client.0.smithi090.stdout:7/258: unlink db/d17/d39/d59/f62 0 2024-04-24T08:24:50.390 INFO:tasks.workunit.client.1.smithi149.stdout:0/480: dread d2/d9/fe [0,4194304] 0 2024-04-24T08:24:50.391 INFO:tasks.workunit.client.0.smithi090.stdout:7/259: read db/f11 [2150983,49050] 0 2024-04-24T08:24:50.391 INFO:tasks.workunit.client.0.smithi090.stdout:8/160: fsync d2/da/d13/d15/f16 0 2024-04-24T08:24:50.392 INFO:tasks.workunit.client.1.smithi149.stdout:1/636: truncate d0/d5/db/d18/fc1 1016427 0 2024-04-24T08:24:50.393 INFO:tasks.workunit.client.1.smithi149.stdout:5/639: mknod db/d35/cdb 0 2024-04-24T08:24:50.393 INFO:tasks.workunit.client.0.smithi090.stdout:0/249: dwrite d1a/f1e [0,4194304] 0 2024-04-24T08:24:50.399 INFO:tasks.workunit.client.1.smithi149.stdout:5/640: creat db/d12/d2f/d95/fdc x:0 0 0 2024-04-24T08:24:50.403 INFO:tasks.workunit.client.1.smithi149.stdout:5/641: write db/d12/d2f/d78/fd4 [140058,98930] 0 2024-04-24T08:24:50.403 INFO:tasks.workunit.client.1.smithi149.stdout:1/637: dread d0/d1b/d4b/f58 [0,4194304] 0 2024-04-24T08:24:50.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/204: creat d3/d25/f4a x:0 0 0 2024-04-24T08:24:50.409 INFO:tasks.workunit.client.0.smithi090.stdout:5/273: truncate d8/f31 8071803 0 2024-04-24T08:24:50.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/246: rmdir d8/d1d/d1f 39 2024-04-24T08:24:50.411 INFO:tasks.workunit.client.0.smithi090.stdout:5/274: dread - d8/da/d3a/d36/d50/f57 zero size 2024-04-24T08:24:50.414 INFO:tasks.workunit.client.1.smithi149.stdout:5/642: symlink db/d12/d60/db2/d83/dc8/ldd 0 2024-04-24T08:24:50.416 INFO:tasks.workunit.client.1.smithi149.stdout:0/481: getdents d2/d9 0 2024-04-24T08:24:50.418 INFO:tasks.workunit.client.0.smithi090.stdout:7/260: read - db/d17/d29/d32/f40 zero size 2024-04-24T08:24:50.419 INFO:tasks.workunit.client.1.smithi149.stdout:5/643: mknod db/d35/cde 0 2024-04-24T08:24:50.420 INFO:tasks.workunit.client.0.smithi090.stdout:8/161: mknod d2/da/c37 0 2024-04-24T08:24:50.422 INFO:tasks.workunit.client.0.smithi090.stdout:3/181: truncate d2/f9 2694952 0 2024-04-24T08:24:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:0/482: rename d2/d9/df/d2f/f30 to d2/d9/df/d2f/d6c/d72/d91/f95 0 2024-04-24T08:24:50.425 INFO:tasks.workunit.client.1.smithi149.stdout:1/638: write d0/d5/db/f87 [176859,81661] 0 2024-04-24T08:24:50.427 INFO:tasks.workunit.client.0.smithi090.stdout:1/205: symlink d3/d5/da/dc/l4b 0 2024-04-24T08:24:50.428 INFO:tasks.workunit.client.1.smithi149.stdout:5/644: mknod db/d12/d22/d45/cdf 0 2024-04-24T08:24:50.428 INFO:tasks.workunit.client.1.smithi149.stdout:1/639: readlink d0/d5/db/d63/d98/lba 0 2024-04-24T08:24:50.429 INFO:tasks.workunit.client.1.smithi149.stdout:0/483: rename d2/d3c/c62 to d2/d3c/d57/c96 0 2024-04-24T08:24:50.431 INFO:tasks.workunit.client.1.smithi149.stdout:5/645: mknod db/d12/d60/d8f/ce0 0 2024-04-24T08:24:50.432 INFO:tasks.workunit.client.1.smithi149.stdout:5/646: fsync db/f94 0 2024-04-24T08:24:50.432 INFO:tasks.workunit.client.1.smithi149.stdout:1/640: rmdir d0/d39/da0 39 2024-04-24T08:24:50.433 INFO:tasks.workunit.client.1.smithi149.stdout:5/647: write db/f14 [1400154,80215] 0 2024-04-24T08:24:50.435 INFO:tasks.workunit.client.0.smithi090.stdout:0/250: dwrite d1a/f24 [4194304,4194304] 0 2024-04-24T08:24:50.437 INFO:tasks.workunit.client.1.smithi149.stdout:1/641: symlink d0/d5/db/d82/lcb 0 2024-04-24T08:24:50.441 INFO:tasks.workunit.client.1.smithi149.stdout:5/648: creat db/d12/d1e/d65/fe1 x:0 0 0 2024-04-24T08:24:50.444 INFO:tasks.workunit.client.0.smithi090.stdout:2/194: rename d0/d7/f1c to d0/f3b 0 2024-04-24T08:24:50.446 INFO:tasks.workunit.client.0.smithi090.stdout:2/195: dread - d0/d14/f2b zero size 2024-04-24T08:24:50.447 INFO:tasks.workunit.client.0.smithi090.stdout:3/182: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:50.449 INFO:tasks.workunit.client.1.smithi149.stdout:1/642: unlink d0/d1b/db5/l3e 0 2024-04-24T08:24:50.451 INFO:tasks.workunit.client.1.smithi149.stdout:5/649: mkdir db/d12/d60/db2/de2 0 2024-04-24T08:24:50.454 INFO:tasks.workunit.client.1.smithi149.stdout:1/643: creat d0/d5/db/d18/fcc x:0 0 0 2024-04-24T08:24:50.455 INFO:tasks.workunit.client.1.smithi149.stdout:1/644: chown d0/d5/d4f/dac 99145715 1 2024-04-24T08:24:50.456 INFO:tasks.workunit.client.1.smithi149.stdout:0/484: rename d2/d9/df/d2f/d6c/d72/d91/c94 to d2/d9/d1a/c97 0 2024-04-24T08:24:50.458 INFO:tasks.workunit.client.1.smithi149.stdout:5/650: mknod db/da4/ce3 0 2024-04-24T08:24:50.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/140: getdents d3 0 2024-04-24T08:24:50.461 INFO:tasks.workunit.client.1.smithi149.stdout:1/645: creat d0/d5/db/fcd x:0 0 0 2024-04-24T08:24:50.462 INFO:tasks.workunit.client.1.smithi149.stdout:1/646: chown d0/d1b/d4b/c89 114435 1 2024-04-24T08:24:50.464 INFO:tasks.workunit.client.0.smithi090.stdout:7/261: rename db/d17/d29/d38/d3e/d46/f60 to db/d17/d39/f66 0 2024-04-24T08:24:50.466 INFO:tasks.workunit.client.0.smithi090.stdout:1/206: dwrite d3/f2e [0,4194304] 0 2024-04-24T08:24:50.468 INFO:tasks.workunit.client.0.smithi090.stdout:1/207: stat d3/d1d/c2c 0 2024-04-24T08:24:50.468 INFO:tasks.workunit.client.0.smithi090.stdout:7/262: fsync db/d17/d39/f65 0 2024-04-24T08:24:50.468 INFO:tasks.workunit.client.0.smithi090.stdout:9/198: getdents d0/d19/d20 0 2024-04-24T08:24:50.470 INFO:tasks.workunit.client.0.smithi090.stdout:1/208: write d3/d5/f3f [904411,111444] 0 2024-04-24T08:24:50.473 INFO:tasks.workunit.client.1.smithi149.stdout:0/485: rename d2/d9/df/f21 to d2/d17/f98 0 2024-04-24T08:24:50.474 INFO:tasks.workunit.client.1.smithi149.stdout:1/647: mkdir d0/d5/dce 0 2024-04-24T08:24:50.475 INFO:tasks.workunit.client.0.smithi090.stdout:5/275: write d8/f31 [3898522,71121] 0 2024-04-24T08:24:50.477 INFO:tasks.workunit.client.1.smithi149.stdout:0/486: creat d2/d9/df/d2f/d6c/d72/f99 x:0 0 0 2024-04-24T08:24:50.479 INFO:tasks.workunit.client.1.smithi149.stdout:1/648: rename d0/f44 to d0/d39/d53/fcf 0 2024-04-24T08:24:50.480 INFO:tasks.workunit.client.0.smithi090.stdout:8/162: dwrite d2/da/d13/d15/f32 [0,4194304] 0 2024-04-24T08:24:50.480 INFO:tasks.workunit.client.0.smithi090.stdout:0/251: truncate d1a/d2a/d35/f49 347845 0 2024-04-24T08:24:50.480 INFO:tasks.workunit.client.0.smithi090.stdout:4/141: mknod d3/d18/c23 0 2024-04-24T08:24:50.483 INFO:tasks.workunit.client.1.smithi149.stdout:1/649: mknod d0/d5/cd0 0 2024-04-24T08:24:50.485 INFO:tasks.workunit.client.0.smithi090.stdout:0/252: dread - d1a/d2a/d35/f3e zero size 2024-04-24T08:24:50.485 INFO:tasks.workunit.client.0.smithi090.stdout:4/142: write d3/df/f1e [448925,109895] 0 2024-04-24T08:24:50.488 INFO:tasks.workunit.client.0.smithi090.stdout:0/253: write d1a/d4b/f4d [824512,95252] 0 2024-04-24T08:24:50.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/196: symlink d0/d14/l3c 0 2024-04-24T08:24:50.489 INFO:tasks.workunit.client.0.smithi090.stdout:1/209: mkdir d3/d1d/d4c 0 2024-04-24T08:24:50.490 INFO:tasks.workunit.client.1.smithi149.stdout:1/650: creat d0/d5/db/d18/d2f/fd1 x:0 0 0 2024-04-24T08:24:50.493 INFO:tasks.workunit.client.0.smithi090.stdout:7/263: dread db/d17/f2d [0,4194304] 0 2024-04-24T08:24:50.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/651: fdatasync d0/d5/d4f/d60/f78 0 2024-04-24T08:24:50.493 INFO:tasks.workunit.client.1.smithi149.stdout:5/651: write db/d12/d60/db2/f6d [3345262,77970] 0 2024-04-24T08:24:50.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/163: creat d2/da/f38 x:0 0 0 2024-04-24T08:24:50.494 INFO:tasks.workunit.client.0.smithi090.stdout:3/183: mknod d2/d25/d33/c38 0 2024-04-24T08:24:50.498 INFO:tasks.workunit.client.1.smithi149.stdout:6/417: sync 2024-04-24T08:24:50.499 INFO:tasks.workunit.client.0.smithi090.stdout:8/164: chown d2/da/d13/d15/f16 2 1 2024-04-24T08:24:50.500 INFO:tasks.workunit.client.0.smithi090.stdout:3/184: truncate d2/f17 4330908 0 2024-04-24T08:24:50.501 INFO:tasks.workunit.client.1.smithi149.stdout:1/652: fdatasync d0/d5/d4f/d60/fc8 0 2024-04-24T08:24:50.501 INFO:tasks.workunit.client.0.smithi090.stdout:9/199: unlink d0/f15 0 2024-04-24T08:24:50.502 INFO:tasks.workunit.client.0.smithi090.stdout:8/165: truncate d2/da/f35 928006 0 2024-04-24T08:24:50.503 INFO:tasks.workunit.client.0.smithi090.stdout:9/200: fdatasync d0/fb 0 2024-04-24T08:24:50.504 INFO:tasks.workunit.client.1.smithi149.stdout:1/653: write d0/fe [1800109,100755] 0 2024-04-24T08:24:50.505 INFO:tasks.workunit.client.1.smithi149.stdout:5/652: rename db/d12/d2f/l39 to db/d12/d2f/d78/le4 0 2024-04-24T08:24:50.507 INFO:tasks.workunit.client.0.smithi090.stdout:3/185: write d2/d15/f31 [93795,25804] 0 2024-04-24T08:24:50.508 INFO:tasks.workunit.client.0.smithi090.stdout:4/143: dread d3/f15 [0,4194304] 0 2024-04-24T08:24:50.508 INFO:tasks.workunit.client.0.smithi090.stdout:1/210: creat d3/d5/da/f4d x:0 0 0 2024-04-24T08:24:50.509 INFO:tasks.workunit.client.1.smithi149.stdout:6/418: mkdir d0/d1a/d28/d31/d4c/d8a 0 2024-04-24T08:24:50.509 INFO:tasks.workunit.client.1.smithi149.stdout:0/487: dread d2/fa [0,4194304] 0 2024-04-24T08:24:50.511 INFO:tasks.workunit.client.0.smithi090.stdout:7/264: symlink db/d17/d29/d32/l67 0 2024-04-24T08:24:50.514 INFO:tasks.workunit.client.1.smithi149.stdout:1/654: symlink d0/d5/db/d18/d2f/ld2 0 2024-04-24T08:24:50.518 INFO:tasks.workunit.client.1.smithi149.stdout:0/488: unlink d2/d9/f5a 0 2024-04-24T08:24:50.519 INFO:tasks.workunit.client.0.smithi090.stdout:6/247: rename d8/dc/d19/f41 to d8/f46 0 2024-04-24T08:24:50.520 INFO:tasks.workunit.client.0.smithi090.stdout:7/265: dread - db/d17/d29/d32/f57 zero size 2024-04-24T08:24:50.520 INFO:tasks.workunit.client.1.smithi149.stdout:6/419: mknod d0/d1a/d28/d31/d4c/d4e/d62/c8b 0 2024-04-24T08:24:50.521 INFO:tasks.workunit.client.1.smithi149.stdout:6/420: readlink d0/d1a/d28/l36 0 2024-04-24T08:24:50.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/655: mkdir d0/d5/db/d63/d98/dd3 0 2024-04-24T08:24:50.523 INFO:tasks.workunit.client.0.smithi090.stdout:8/166: unlink d2/f7 0 2024-04-24T08:24:50.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/167: fsync d2/da/f35 0 2024-04-24T08:24:50.525 INFO:tasks.workunit.client.0.smithi090.stdout:2/197: dwrite d0/d14/f1f [0,4194304] 0 2024-04-24T08:24:50.527 INFO:tasks.workunit.client.0.smithi090.stdout:9/201: creat d0/dc/f3e x:0 0 0 2024-04-24T08:24:50.529 INFO:tasks.workunit.client.0.smithi090.stdout:3/186: rmdir d2/df 39 2024-04-24T08:24:50.530 INFO:tasks.workunit.client.1.smithi149.stdout:6/421: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f8c x:0 0 0 2024-04-24T08:24:50.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/276: link d8/da/d3a/d36/l45 d8/d59/l5d 0 2024-04-24T08:24:50.535 INFO:tasks.workunit.client.1.smithi149.stdout:1/656: read d0/d1b/db5/f2b [2306604,127065] 0 2024-04-24T08:24:50.539 INFO:tasks.workunit.client.0.smithi090.stdout:2/198: mknod d0/d14/d17/d2a/c3d 0 2024-04-24T08:24:50.539 INFO:tasks.workunit.client.1.smithi149.stdout:5/653: dwrite db/d35/f7c [0,4194304] 0 2024-04-24T08:24:50.541 INFO:tasks.workunit.client.1.smithi149.stdout:4/505: sync 2024-04-24T08:24:50.550 INFO:tasks.workunit.client.1.smithi149.stdout:5/654: dread db/d23/f43 [0,4194304] 0 2024-04-24T08:24:50.552 INFO:tasks.workunit.client.1.smithi149.stdout:4/506: mknod d2/d10/d91/ca5 0 2024-04-24T08:24:50.554 INFO:tasks.workunit.client.0.smithi090.stdout:1/211: dread d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:50.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/248: symlink d8/l47 0 2024-04-24T08:24:50.555 INFO:tasks.workunit.client.1.smithi149.stdout:5/655: write db/d12/d2f/d95/fc3 [252775,53173] 0 2024-04-24T08:24:50.556 INFO:tasks.workunit.client.1.smithi149.stdout:6/422: creat d0/d1a/d28/d31/d4c/d4e/f8d x:0 0 0 2024-04-24T08:24:50.557 INFO:tasks.workunit.client.0.smithi090.stdout:1/212: readlink d3/d5/da/dc/l20 0 2024-04-24T08:24:50.560 INFO:tasks.workunit.client.1.smithi149.stdout:1/657: dwrite d0/d5/db/d18/fcc [0,4194304] 0 2024-04-24T08:24:50.561 INFO:tasks.workunit.client.0.smithi090.stdout:4/144: truncate d3/f16 1944420 0 2024-04-24T08:24:50.563 INFO:tasks.workunit.client.0.smithi090.stdout:5/277: rename d8/d25/f51 to d8/da/d3a/d36/d50/f5e 0 2024-04-24T08:24:50.567 INFO:tasks.workunit.client.0.smithi090.stdout:2/199: fsync d0/d7/f11 0 2024-04-24T08:24:50.570 INFO:tasks.workunit.client.1.smithi149.stdout:0/489: creat d2/d9/df/d2f/d6c/f9a x:0 0 0 2024-04-24T08:24:50.573 INFO:tasks.workunit.client.0.smithi090.stdout:1/213: symlink d3/d5/l4e 0 2024-04-24T08:24:50.577 INFO:tasks.workunit.client.1.smithi149.stdout:5/656: dwrite db/da4/fb5 [0,4194304] 0 2024-04-24T08:24:50.580 INFO:tasks.workunit.client.1.smithi149.stdout:6/423: symlink d0/d1a/d28/d31/d4c/d76/l8e 0 2024-04-24T08:24:50.582 INFO:tasks.workunit.client.0.smithi090.stdout:6/249: rename d8/dc/l17 to d8/d1d/d2f/l48 0 2024-04-24T08:24:50.583 INFO:tasks.workunit.client.1.smithi149.stdout:0/490: rmdir d2/d9/df/d40/d4a/d4e/d6f 39 2024-04-24T08:24:50.584 INFO:tasks.workunit.client.0.smithi090.stdout:7/266: dwrite db/f1a [0,4194304] 0 2024-04-24T08:24:50.584 INFO:tasks.workunit.client.0.smithi090.stdout:6/250: write d8/d1d/d29/f2a [2920462,44895] 0 2024-04-24T08:24:50.584 INFO:tasks.workunit.client.0.smithi090.stdout:3/187: truncate d2/f17 750009 0 2024-04-24T08:24:50.592 INFO:tasks.workunit.client.1.smithi149.stdout:1/658: link d0/d5/f8 d0/d5/db/d18/d4d/d50/daf/fd4 0 2024-04-24T08:24:50.593 INFO:tasks.workunit.client.1.smithi149.stdout:1/659: chown d0/d39 542 1 2024-04-24T08:24:50.595 INFO:tasks.workunit.client.0.smithi090.stdout:2/200: write d0/f3b [290883,103490] 0 2024-04-24T08:24:50.598 INFO:tasks.workunit.client.1.smithi149.stdout:6/424: unlink d0/fe 0 2024-04-24T08:24:50.599 INFO:tasks.workunit.client.1.smithi149.stdout:7/538: sync 2024-04-24T08:24:50.599 INFO:tasks.workunit.client.1.smithi149.stdout:2/479: sync 2024-04-24T08:24:50.599 INFO:tasks.workunit.client.1.smithi149.stdout:3/595: sync 2024-04-24T08:24:50.599 INFO:tasks.workunit.client.1.smithi149.stdout:5/657: read db/d12/d1e/f41 [12847,83347] 0 2024-04-24T08:24:50.600 INFO:tasks.workunit.client.1.smithi149.stdout:9/580: sync 2024-04-24T08:24:50.600 INFO:tasks.workunit.client.1.smithi149.stdout:8/467: sync 2024-04-24T08:24:50.600 INFO:tasks.workunit.client.0.smithi090.stdout:1/214: mknod d3/d5/d3e/c4f 0 2024-04-24T08:24:50.604 INFO:tasks.workunit.client.1.smithi149.stdout:1/660: creat d0/d5/d4f/d60/fd5 x:0 0 0 2024-04-24T08:24:50.605 INFO:tasks.workunit.client.0.smithi090.stdout:9/202: getdents d0 0 2024-04-24T08:24:50.608 INFO:tasks.workunit.client.0.smithi090.stdout:4/145: fdatasync d3/f16 0 2024-04-24T08:24:50.611 INFO:tasks.workunit.client.1.smithi149.stdout:0/491: write d2/d9/d1a/f25 [2255046,73191] 0 2024-04-24T08:24:50.611 INFO:tasks.workunit.client.1.smithi149.stdout:9/581: chown d1/d2/c9 3640737 1 2024-04-24T08:24:50.614 INFO:tasks.workunit.client.0.smithi090.stdout:4/146: read d3/df/f1e [347769,124016] 0 2024-04-24T08:24:50.614 INFO:tasks.workunit.client.1.smithi149.stdout:9/582: dread - d1/d4c/d91/fb9 zero size 2024-04-24T08:24:50.615 INFO:tasks.workunit.client.1.smithi149.stdout:0/492: chown d2/d3c/d57/f78 51 1 2024-04-24T08:24:50.619 INFO:tasks.workunit.client.1.smithi149.stdout:4/507: dread d2/d18/d1e/d3c/f80 [4194304,4194304] 0 2024-04-24T08:24:50.624 INFO:tasks.workunit.client.1.smithi149.stdout:7/539: creat d9/dc/d7b/fb2 x:0 0 0 2024-04-24T08:24:50.624 INFO:tasks.workunit.client.0.smithi090.stdout:3/188: write d2/df/d21/f27 [853417,7319] 0 2024-04-24T08:24:50.624 INFO:tasks.workunit.client.1.smithi149.stdout:1/661: mkdir d0/d1b/d5d/dd6 0 2024-04-24T08:24:50.626 INFO:tasks.workunit.client.0.smithi090.stdout:8/168: getdents d2/da/d13/d15 0 2024-04-24T08:24:50.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/267: rename db/f56 to db/d17/d1b/f68 0 2024-04-24T08:24:50.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/480: dwrite d0/d6/db/f72 [0,4194304] 0 2024-04-24T08:24:50.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/425: dread d0/fb [0,4194304] 0 2024-04-24T08:24:50.634 INFO:tasks.workunit.client.0.smithi090.stdout:3/189: dread d2/df/f19 [0,4194304] 0 2024-04-24T08:24:50.635 INFO:tasks.workunit.client.1.smithi149.stdout:3/596: rename d0/d5/d53/d34/fa9 to d0/d26/d3c/db1/fd0 0 2024-04-24T08:24:50.640 INFO:tasks.workunit.client.1.smithi149.stdout:2/481: truncate d0/d6/db/d27/f3b 530106 0 2024-04-24T08:24:50.642 INFO:tasks.workunit.client.1.smithi149.stdout:3/597: fdatasync d0/d5/fc7 0 2024-04-24T08:24:50.644 INFO:tasks.workunit.client.1.smithi149.stdout:2/482: stat d0/d6/db/d27/d64/d66 0 2024-04-24T08:24:50.644 INFO:tasks.workunit.client.1.smithi149.stdout:8/468: creat d1/de/d18/d32/d63/d9e/fa1 x:0 0 0 2024-04-24T08:24:50.646 INFO:tasks.workunit.client.0.smithi090.stdout:5/278: creat d8/d56/f5f x:0 0 0 2024-04-24T08:24:50.647 INFO:tasks.workunit.client.1.smithi149.stdout:4/508: dwrite d2/d24/d2e/d38/f7a [0,4194304] 0 2024-04-24T08:24:50.648 INFO:tasks.workunit.client.0.smithi090.stdout:1/215: rename d3/d5/da/dc/fe to d3/d5/da/dc/f50 0 2024-04-24T08:24:50.651 INFO:tasks.workunit.client.0.smithi090.stdout:1/216: write d3/d5/da/dc/f3a [1519081,106157] 0 2024-04-24T08:24:50.652 INFO:tasks.workunit.client.0.smithi090.stdout:4/147: mkdir d3/d24 0 2024-04-24T08:24:50.653 INFO:tasks.workunit.client.1.smithi149.stdout:2/483: dread d0/f35 [0,4194304] 0 2024-04-24T08:24:50.656 INFO:tasks.workunit.client.1.smithi149.stdout:7/540: symlink d9/dc/de/d44/lb3 0 2024-04-24T08:24:50.658 INFO:tasks.workunit.client.0.smithi090.stdout:8/169: symlink d2/da/l39 0 2024-04-24T08:24:50.660 INFO:tasks.workunit.client.1.smithi149.stdout:1/662: write d0/d5/db/d18/d2f/fa4 [5045747,44327] 0 2024-04-24T08:24:50.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/268: rmdir db/d17/d29/d32 39 2024-04-24T08:24:50.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/269: chown f4 36 1 2024-04-24T08:24:50.663 INFO:tasks.workunit.client.1.smithi149.stdout:6/426: symlink d0/d1a/d28/d31/d38/l8f 0 2024-04-24T08:24:50.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/251: link d8/f16 d8/d1d/d2f/d30/d38/f49 0 2024-04-24T08:24:50.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/190: readlink d2/df/d1a/l34 0 2024-04-24T08:24:50.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/201: symlink d0/d2e/l3e 0 2024-04-24T08:24:50.671 INFO:tasks.workunit.client.1.smithi149.stdout:9/583: dwrite d1/d4c/f54 [4194304,4194304] 0 2024-04-24T08:24:50.672 INFO:tasks.workunit.client.0.smithi090.stdout:2/202: read d0/d14/f1d [3044962,111115] 0 2024-04-24T08:24:50.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/170: dread d2/f36 [0,4194304] 0 2024-04-24T08:24:50.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/279: creat d8/da/d18/f60 x:0 0 0 2024-04-24T08:24:50.674 INFO:tasks.workunit.client.0.smithi090.stdout:2/203: write d0/d7/d24/f29 [2451505,19971] 0 2024-04-24T08:24:50.675 INFO:tasks.workunit.client.1.smithi149.stdout:3/598: mknod d0/d5/d53/d4c/d5f/cd1 0 2024-04-24T08:24:50.676 INFO:tasks.workunit.client.0.smithi090.stdout:1/217: rmdir d3/d1d/d34 39 2024-04-24T08:24:50.677 INFO:tasks.workunit.client.0.smithi090.stdout:4/148: mkdir d3/d25 0 2024-04-24T08:24:50.679 INFO:tasks.workunit.client.1.smithi149.stdout:7/541: creat d9/dc/d11/d38/fb4 x:0 0 0 2024-04-24T08:24:50.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/658: link db/d12/d1e/d85/fa5 db/d12/d2f/d95/fe5 0 2024-04-24T08:24:50.686 INFO:tasks.workunit.client.1.smithi149.stdout:6/427: readlink d0/d1a/d28/d31/d4c/d80/l5a 0 2024-04-24T08:24:50.687 INFO:tasks.workunit.client.0.smithi090.stdout:3/191: chown d2/f9 12853483 1 2024-04-24T08:24:50.687 INFO:tasks.workunit.client.1.smithi149.stdout:6/428: fdatasync d0/d1a/d28/d31/d38/f3a 0 2024-04-24T08:24:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/584: readlink d1/da/l8e 0 2024-04-24T08:24:50.694 INFO:tasks.workunit.client.0.smithi090.stdout:2/204: symlink d0/d14/d17/l3f 0 2024-04-24T08:24:50.696 INFO:tasks.workunit.client.0.smithi090.stdout:5/280: rename d8/da/d14/f27 to d8/d59/f61 0 2024-04-24T08:24:50.697 INFO:tasks.workunit.client.0.smithi090.stdout:1/218: fdatasync d3/d1d/d34/f3b 0 2024-04-24T08:24:50.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/219: chown d3/d5/f33 1452 1 2024-04-24T08:24:50.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/149: fsync d3/f9 0 2024-04-24T08:24:50.700 INFO:tasks.workunit.client.1.smithi149.stdout:2/484: write d0/d6/db/f1f [8158957,56178] 0 2024-04-24T08:24:50.702 INFO:tasks.workunit.client.1.smithi149.stdout:7/542: mkdir d9/d43/d95/db5 0 2024-04-24T08:24:50.703 INFO:tasks.workunit.client.1.smithi149.stdout:1/663: truncate d0/d39/d53/fcf 23041 0 2024-04-24T08:24:50.704 INFO:tasks.workunit.client.0.smithi090.stdout:6/252: mknod d8/d1d/c4a 0 2024-04-24T08:24:50.709 INFO:tasks.workunit.client.1.smithi149.stdout:0/493: getdents d2/d17/d41 0 2024-04-24T08:24:50.710 INFO:tasks.workunit.client.0.smithi090.stdout:2/205: mknod d0/d14/d17/d2a/c40 0 2024-04-24T08:24:50.710 INFO:tasks.workunit.client.1.smithi149.stdout:0/494: chown d2/d3c/c53 752 1 2024-04-24T08:24:50.711 INFO:tasks.workunit.client.0.smithi090.stdout:2/206: readlink d0/d2e/l3e 0 2024-04-24T08:24:50.712 INFO:tasks.workunit.client.1.smithi149.stdout:6/429: rename d0/d1a/f75 to d0/d1a/d28/d31/d33/f90 0 2024-04-24T08:24:50.712 INFO:tasks.workunit.client.0.smithi090.stdout:8/171: dwrite d2/da/f1c [0,4194304] 0 2024-04-24T08:24:50.713 INFO:tasks.workunit.client.1.smithi149.stdout:8/469: dwrite d1/de/d18/f48 [0,4194304] 0 2024-04-24T08:24:50.716 INFO:tasks.workunit.client.1.smithi149.stdout:4/509: creat d2/d10/fa6 x:0 0 0 2024-04-24T08:24:50.717 INFO:tasks.workunit.client.1.smithi149.stdout:3/599: creat d0/d5/da/d19/dcb/fd2 x:0 0 0 2024-04-24T08:24:50.718 INFO:tasks.workunit.client.0.smithi090.stdout:4/150: symlink d3/l26 0 2024-04-24T08:24:50.720 INFO:tasks.workunit.client.1.smithi149.stdout:2/485: chown d0/d6/db/d39/d78/d7c/f97 18814 1 2024-04-24T08:24:50.722 INFO:tasks.workunit.client.1.smithi149.stdout:2/486: chown d0/d6 4976582 1 2024-04-24T08:24:50.722 INFO:tasks.workunit.client.1.smithi149.stdout:7/543: rmdir d9/dc/d11/d83/da6/d67/d69 39 2024-04-24T08:24:50.723 INFO:tasks.workunit.client.0.smithi090.stdout:1/220: creat d3/d5/da/dc/f51 x:0 0 0 2024-04-24T08:24:50.724 INFO:tasks.workunit.client.0.smithi090.stdout:7/270: creat db/d17/d29/f69 x:0 0 0 2024-04-24T08:24:50.724 INFO:tasks.workunit.client.1.smithi149.stdout:1/664: dread - d0/d5/db/f8c zero size 2024-04-24T08:24:50.725 INFO:tasks.workunit.client.1.smithi149.stdout:2/487: chown d0/d6/db/d39/d3d/d58/c5c 214383521 1 2024-04-24T08:24:50.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/192: mknod d2/df/d1a/d37/c39 0 2024-04-24T08:24:50.726 INFO:tasks.workunit.client.0.smithi090.stdout:1/221: fdatasync d3/d1d/f37 0 2024-04-24T08:24:50.727 INFO:tasks.workunit.client.1.smithi149.stdout:2/488: chown d0/d6/f16 272956 1 2024-04-24T08:24:50.727 INFO:tasks.workunit.client.1.smithi149.stdout:1/665: chown d0/d5/ca9 1389 1 2024-04-24T08:24:50.728 INFO:tasks.workunit.client.0.smithi090.stdout:1/222: fsync d3/d5/f17 0 2024-04-24T08:24:50.728 INFO:tasks.workunit.client.1.smithi149.stdout:7/544: write d9/dc/d11/d26/fae [796889,68045] 0 2024-04-24T08:24:50.730 INFO:tasks.workunit.client.1.smithi149.stdout:1/666: chown d0/d5/df/f94 0 1 2024-04-24T08:24:50.731 INFO:tasks.workunit.client.1.smithi149.stdout:7/545: readlink d9/dc/d2e/d51/l68 0 2024-04-24T08:24:50.731 INFO:tasks.workunit.client.1.smithi149.stdout:1/667: stat d0/d5/df 0 2024-04-24T08:24:50.732 INFO:tasks.workunit.client.0.smithi090.stdout:8/172: creat d2/da/d13/f3a x:0 0 0 2024-04-24T08:24:50.734 INFO:tasks.workunit.client.1.smithi149.stdout:9/585: mknod d1/d2/ccc 0 2024-04-24T08:24:50.734 INFO:tasks.workunit.client.0.smithi090.stdout:8/173: chown d2/f22 3 1 2024-04-24T08:24:50.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/470: rename d1/de/d18/l3a to d1/de/d18/d91/la2 0 2024-04-24T08:24:50.740 INFO:tasks.workunit.client.1.smithi149.stdout:3/600: mknod d0/d5/da/d59/cd3 0 2024-04-24T08:24:50.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/151: dwrite d3/f16 [0,4194304] 0 2024-04-24T08:24:50.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/601: fsync d0/d5/d1a/d70/fba 0 2024-04-24T08:24:50.743 INFO:tasks.workunit.client.0.smithi090.stdout:1/223: creat d3/d1d/d34/f52 x:0 0 0 2024-04-24T08:24:50.746 INFO:tasks.workunit.client.0.smithi090.stdout:3/193: dwrite d2/df/d1a/f2b [0,4194304] 0 2024-04-24T08:24:50.747 INFO:tasks.workunit.client.1.smithi149.stdout:2/489: mkdir d0/d6/db/d39/d3d/d58/d98 0 2024-04-24T08:24:50.748 INFO:tasks.workunit.client.0.smithi090.stdout:8/174: symlink d2/da/d13/d15/d24/l3b 0 2024-04-24T08:24:50.749 INFO:tasks.workunit.client.1.smithi149.stdout:2/490: dread - d0/d6/db/d28/f73 zero size 2024-04-24T08:24:50.750 INFO:tasks.workunit.client.1.smithi149.stdout:5/659: truncate db/d12/d22/f31 149576 0 2024-04-24T08:24:50.760 INFO:tasks.workunit.client.1.smithi149.stdout:7/546: truncate d9/dc/d2e/f41 595740 0 2024-04-24T08:24:50.762 INFO:tasks.workunit.client.0.smithi090.stdout:7/271: dwrite db/d17/d29/d32/f36 [0,4194304] 0 2024-04-24T08:24:50.763 INFO:tasks.workunit.client.0.smithi090.stdout:2/207: dwrite d0/d7/f32 [0,4194304] 0 2024-04-24T08:24:50.764 INFO:tasks.workunit.client.0.smithi090.stdout:2/208: stat d0/d2e/l3e 0 2024-04-24T08:24:50.766 INFO:tasks.workunit.client.1.smithi149.stdout:8/471: dread d1/de/f38 [0,4194304] 0 2024-04-24T08:24:50.770 INFO:tasks.workunit.client.0.smithi090.stdout:2/209: chown d0/d7/fd 6 1 2024-04-24T08:24:50.780 INFO:tasks.workunit.client.1.smithi149.stdout:4/510: dread d2/d24/f9b [0,4194304] 0 2024-04-24T08:24:50.788 INFO:tasks.workunit.client.1.smithi149.stdout:6/430: getdents d0/d1a/d28/d31/d4c/d4e/d62/d7a 0 2024-04-24T08:24:50.812 INFO:tasks.workunit.client.1.smithi149.stdout:4/511: dread d2/d10/d1a/d34/f37 [0,4194304] 0 2024-04-24T08:24:50.816 INFO:tasks.workunit.client.0.smithi090.stdout:6/253: truncate d8/d1d/f1e 2317212 0 2024-04-24T08:24:50.819 INFO:tasks.workunit.client.0.smithi090.stdout:4/152: dread d3/df/f1e [0,4194304] 0 2024-04-24T08:24:50.820 INFO:tasks.workunit.client.0.smithi090.stdout:5/281: getdents d8/da/d14 0 2024-04-24T08:24:50.821 INFO:tasks.workunit.client.1.smithi149.stdout:3/602: mkdir d0/d5/da/d19/d1b/d22/dd4 0 2024-04-24T08:24:50.823 INFO:tasks.workunit.client.0.smithi090.stdout:5/282: chown d8/da/d14/l16 331 1 2024-04-24T08:24:50.825 INFO:tasks.workunit.client.1.smithi149.stdout:3/603: truncate d0/d5/d53/d39/f8a 889031 0 2024-04-24T08:24:50.826 INFO:tasks.workunit.client.1.smithi149.stdout:9/586: dwrite d1/d2/d15/f2f [0,4194304] 0 2024-04-24T08:24:50.827 INFO:tasks.workunit.client.1.smithi149.stdout:2/491: chown d0/d6/db/d39/d3d/d44/d75/f79 5 1 2024-04-24T08:24:50.829 INFO:tasks.workunit.client.1.smithi149.stdout:0/495: link d2/d9/df/d48/c55 d2/d9/df/d92/c9b 0 2024-04-24T08:24:50.832 INFO:tasks.workunit.client.1.smithi149.stdout:5/660: mkdir db/d12/d22/d45/d5d/de6 0 2024-04-24T08:24:50.836 INFO:tasks.workunit.client.0.smithi090.stdout:6/254: dwrite d8/dc/f3a [0,4194304] 0 2024-04-24T08:24:50.843 INFO:tasks.workunit.client.1.smithi149.stdout:6/431: creat d0/d1a/d28/d31/d4c/d4e/d62/f91 x:0 0 0 2024-04-24T08:24:50.850 INFO:tasks.workunit.client.1.smithi149.stdout:0/496: dwrite d2/d9/df/d40/d4a/d4e/f6d [0,4194304] 0 2024-04-24T08:24:50.851 INFO:tasks.workunit.client.0.smithi090.stdout:5/283: dwrite d8/da/d3a/d36/d50/f57 [0,4194304] 0 2024-04-24T08:24:50.851 INFO:tasks.workunit.client.0.smithi090.stdout:4/153: creat d3/d18/d1d/f27 x:0 0 0 2024-04-24T08:24:50.853 INFO:tasks.workunit.client.1.smithi149.stdout:0/497: chown d2/d17/f5e 816 1 2024-04-24T08:24:50.853 INFO:tasks.workunit.client.1.smithi149.stdout:4/512: mkdir d2/d24/da7 0 2024-04-24T08:24:50.854 INFO:tasks.workunit.client.0.smithi090.stdout:4/154: chown d3/d18/d1d/f27 815323132 1 2024-04-24T08:24:50.855 INFO:tasks.workunit.client.0.smithi090.stdout:5/284: dread d8/da/d3a/d2d/f4d [0,4194304] 0 2024-04-24T08:24:50.861 INFO:tasks.workunit.client.0.smithi090.stdout:3/194: mkdir d2/d3a 0 2024-04-24T08:24:50.865 INFO:tasks.workunit.client.0.smithi090.stdout:5/285: dread d8/da/f1b [0,4194304] 0 2024-04-24T08:24:50.866 INFO:tasks.workunit.client.0.smithi090.stdout:3/195: truncate d2/d15/f1d 699066 0 2024-04-24T08:24:50.867 INFO:tasks.workunit.client.1.smithi149.stdout:3/604: mknod d0/d5/d1a/d70/da3/cd5 0 2024-04-24T08:24:50.869 INFO:tasks.workunit.client.1.smithi149.stdout:3/605: chown d0/d5/d53/d34/f35 4333308 1 2024-04-24T08:24:50.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/255: dwrite d8/f1b [0,4194304] 0 2024-04-24T08:24:50.872 INFO:tasks.workunit.client.1.smithi149.stdout:2/492: chown d0/d6/db/d27/c7b 1019770965 1 2024-04-24T08:24:50.876 INFO:tasks.workunit.client.0.smithi090.stdout:8/175: getdents d2/da/d13/d15 0 2024-04-24T08:24:50.876 INFO:tasks.workunit.client.1.smithi149.stdout:8/472: creat d1/de/d18/d91/d9f/fa3 x:0 0 0 2024-04-24T08:24:50.883 INFO:tasks.workunit.client.0.smithi090.stdout:7/272: symlink db/d17/l6a 0 2024-04-24T08:24:50.885 INFO:tasks.workunit.client.0.smithi090.stdout:8/176: dread d2/f2c [0,4194304] 0 2024-04-24T08:24:50.888 INFO:tasks.workunit.client.0.smithi090.stdout:6/256: creat d8/d1d/d2f/d30/f4b x:0 0 0 2024-04-24T08:24:50.889 INFO:tasks.workunit.client.0.smithi090.stdout:4/155: getdents d3/d24 0 2024-04-24T08:24:50.890 INFO:tasks.workunit.client.1.smithi149.stdout:7/547: link d9/d43/d4f/c60 d9/dc/d2e/cb6 0 2024-04-24T08:24:50.891 INFO:tasks.workunit.client.1.smithi149.stdout:2/493: dwrite d0/d6/db/f72 [0,4194304] 0 2024-04-24T08:24:50.891 INFO:tasks.workunit.client.0.smithi090.stdout:4/156: chown d3/f15 123166315 1 2024-04-24T08:24:50.892 INFO:tasks.workunit.client.1.smithi149.stdout:7/548: read - d9/dc/d7b/f7d zero size 2024-04-24T08:24:50.893 INFO:tasks.workunit.client.1.smithi149.stdout:8/473: fsync d1/de/d18/d32/d63/d76/f86 0 2024-04-24T08:24:50.897 INFO:tasks.workunit.client.0.smithi090.stdout:2/210: getdents d0/d7/d28 0 2024-04-24T08:24:50.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/254: sync 2024-04-24T08:24:50.903 INFO:tasks.workunit.client.0.smithi090.stdout:3/196: symlink d2/d25/d2c/l3b 0 2024-04-24T08:24:50.905 INFO:tasks.workunit.client.1.smithi149.stdout:3/606: mkdir d0/d5/d53/dd6 0 2024-04-24T08:24:50.909 INFO:tasks.workunit.client.1.smithi149.stdout:2/494: chown d0/d6/db/cf 487 1 2024-04-24T08:24:50.921 INFO:tasks.workunit.client.1.smithi149.stdout:9/587: getdents d1/d2/d33/d76 0 2024-04-24T08:24:50.928 INFO:tasks.workunit.client.0.smithi090.stdout:8/177: creat d2/da/d13/d34/f3c x:0 0 0 2024-04-24T08:24:50.929 INFO:tasks.workunit.client.1.smithi149.stdout:8/474: dwrite d1/de/d18/d32/d65/d8b/d7d/f96 [0,4194304] 0 2024-04-24T08:24:50.931 INFO:tasks.workunit.client.0.smithi090.stdout:8/178: write d2/da/d13/d34/f3c [645716,36994] 0 2024-04-24T08:24:50.931 INFO:tasks.workunit.client.0.smithi090.stdout:6/257: readlink d8/d1d/d1f/l25 0 2024-04-24T08:24:50.934 INFO:tasks.workunit.client.0.smithi090.stdout:1/224: mkdir d3/d5/da/d53 0 2024-04-24T08:24:50.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/255: fdatasync f8 0 2024-04-24T08:24:50.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/203: sync 2024-04-24T08:24:50.937 INFO:tasks.workunit.client.1.smithi149.stdout:9/588: dwrite d1/d2/f97 [0,4194304] 0 2024-04-24T08:24:50.949 INFO:tasks.workunit.client.1.smithi149.stdout:9/589: creat d1/da/d13/d74/dbb/fcd x:0 0 0 2024-04-24T08:24:50.953 INFO:tasks.workunit.client.0.smithi090.stdout:3/197: truncate d2/df/f19 3068576 0 2024-04-24T08:24:50.956 INFO:tasks.workunit.client.0.smithi090.stdout:0/256: read d1a/f30 [7106064,101846] 0 2024-04-24T08:24:50.956 INFO:tasks.workunit.client.0.smithi090.stdout:6/258: creat d8/d1d/d2f/d30/d38/f4c x:0 0 0 2024-04-24T08:24:50.956 INFO:tasks.workunit.client.0.smithi090.stdout:3/198: fdatasync d2/df/d21/f32 0 2024-04-24T08:24:50.957 INFO:tasks.workunit.client.1.smithi149.stdout:9/590: dread d1/da/dd/d55/fb1 [0,4194304] 0 2024-04-24T08:24:50.961 INFO:tasks.workunit.client.1.smithi149.stdout:9/591: write d1/da/dd/d79/fa1 [446350,92524] 0 2024-04-24T08:24:50.962 INFO:tasks.workunit.client.0.smithi090.stdout:0/257: mkdir d1a/d4b/d53 0 2024-04-24T08:24:50.965 INFO:tasks.workunit.client.0.smithi090.stdout:6/259: mkdir d8/d1d/d1f/d3e/d4d 0 2024-04-24T08:24:50.966 INFO:tasks.workunit.client.1.smithi149.stdout:9/592: creat d1/d4c/d91/fce x:0 0 0 2024-04-24T08:24:50.968 INFO:tasks.workunit.client.0.smithi090.stdout:9/204: truncate d0/fb 1412176 0 2024-04-24T08:24:50.971 INFO:tasks.workunit.client.0.smithi090.stdout:6/260: mknod d8/d1d/d1f/d3e/c4e 0 2024-04-24T08:24:50.974 INFO:tasks.workunit.client.1.smithi149.stdout:9/593: truncate d1/de/d12/f1d 318877 0 2024-04-24T08:24:50.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/258: creat d1a/d2a/d3c/f54 x:0 0 0 2024-04-24T08:24:50.976 INFO:tasks.workunit.client.0.smithi090.stdout:9/205: creat d0/dc/d24/d2d/f3f x:0 0 0 2024-04-24T08:24:50.976 INFO:tasks.workunit.client.0.smithi090.stdout:0/259: write d1a/d2a/d35/f3e [945331,124145] 0 2024-04-24T08:24:50.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/199: getdents d2/df/d1a/d37 0 2024-04-24T08:24:50.983 INFO:tasks.workunit.client.1.smithi149.stdout:1/668: symlink d0/d5/ld7 0 2024-04-24T08:24:50.988 INFO:tasks.workunit.client.1.smithi149.stdout:1/669: link d0/d5/df/f28 d0/d5/db/d18/d4d/d50/fd8 0 2024-04-24T08:24:50.991 INFO:tasks.workunit.client.1.smithi149.stdout:1/670: mknod d0/d5/db/d18/cd9 0 2024-04-24T08:24:50.993 INFO:tasks.workunit.client.1.smithi149.stdout:1/671: mkdir d0/d39/d53/dda 0 2024-04-24T08:24:50.994 INFO:tasks.workunit.client.1.smithi149.stdout:4/513: write d2/d18/f1b [698254,104459] 0 2024-04-24T08:24:50.996 INFO:tasks.workunit.client.1.smithi149.stdout:0/498: write d2/d9/df/d48/f4c [93075,103536] 0 2024-04-24T08:24:50.996 INFO:tasks.workunit.client.1.smithi149.stdout:1/672: write d0/d1b/d4b/f58 [1393278,109667] 0 2024-04-24T08:24:51.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/499: getdents d2/d17 0 2024-04-24T08:24:51.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/500: readlink d2/d17/l3a 0 2024-04-24T08:24:51.006 INFO:tasks.workunit.client.0.smithi090.stdout:5/286: rename d8/c34 to d8/da/d18/d21/c62 0 2024-04-24T08:24:51.007 INFO:tasks.workunit.client.0.smithi090.stdout:4/157: write d3/f9 [2324734,124442] 0 2024-04-24T08:24:51.007 INFO:tasks.workunit.client.0.smithi090.stdout:7/273: truncate db/d17/d1b/f26 3231141 0 2024-04-24T08:24:51.008 INFO:tasks.workunit.client.0.smithi090.stdout:5/287: stat d8/da/d18/d21/l4f 0 2024-04-24T08:24:51.008 INFO:tasks.workunit.client.0.smithi090.stdout:9/206: dwrite d0/d19/f1a [0,4194304] 0 2024-04-24T08:24:51.010 INFO:tasks.workunit.client.1.smithi149.stdout:2/495: write d0/d6/db/f1c [538170,91844] 0 2024-04-24T08:24:51.010 INFO:tasks.workunit.client.1.smithi149.stdout:0/501: rmdir d2/d9/df/d40/d4a/d4e/d6f 39 2024-04-24T08:24:51.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/475: write d1/d1f/d22/f30 [8506998,93253] 0 2024-04-24T08:24:51.013 INFO:tasks.workunit.client.0.smithi090.stdout:0/260: dwrite d1a/d2a/d35/f3e [0,4194304] 0 2024-04-24T08:24:51.015 INFO:tasks.workunit.client.1.smithi149.stdout:1/673: dwrite d0/d39/fc0 [0,4194304] 0 2024-04-24T08:24:51.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/288: readlink d8/da/d3a/d36/l45 0 2024-04-24T08:24:51.020 INFO:tasks.workunit.client.1.smithi149.stdout:8/476: fdatasync d1/d1f/d25/f78 0 2024-04-24T08:24:51.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/179: write d2/f10 [2739700,48277] 0 2024-04-24T08:24:51.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/502: write d2/d3c/f61 [666953,83599] 0 2024-04-24T08:24:51.023 INFO:tasks.workunit.client.0.smithi090.stdout:1/225: write f0 [670439,68093] 0 2024-04-24T08:24:51.024 INFO:tasks.workunit.client.0.smithi090.stdout:5/289: write d8/d56/f5f [6354,89778] 0 2024-04-24T08:24:51.025 INFO:tasks.workunit.client.1.smithi149.stdout:2/496: creat d0/d6/db/d39/d78/d8a/f99 x:0 0 0 2024-04-24T08:24:51.027 INFO:tasks.workunit.client.0.smithi090.stdout:2/211: dwrite d0/d14/f38 [0,4194304] 0 2024-04-24T08:24:51.028 INFO:tasks.workunit.client.1.smithi149.stdout:8/477: rmdir d1/de/d18/d32/d65/d8b 39 2024-04-24T08:24:51.032 INFO:tasks.workunit.client.1.smithi149.stdout:0/503: creat d2/d9/df/d56/f9c x:0 0 0 2024-04-24T08:24:51.034 INFO:tasks.workunit.client.0.smithi090.stdout:4/158: link d3/df/c14 d3/d18/d1d/d22/c28 0 2024-04-24T08:24:51.035 INFO:tasks.workunit.client.0.smithi090.stdout:8/180: creat d2/da/d13/d34/f3d x:0 0 0 2024-04-24T08:24:51.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/274: dwrite db/f1a [0,4194304] 0 2024-04-24T08:24:51.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/497: mknod d0/d6/db/d39/d78/c9a 0 2024-04-24T08:24:51.040 INFO:tasks.workunit.client.0.smithi090.stdout:1/226: unlink d3/d1d/d34/f52 0 2024-04-24T08:24:51.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/275: write db/d17/d29/d38/d3e/f3f [958994,73293] 0 2024-04-24T08:24:51.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/478: mknod d1/de/ca4 0 2024-04-24T08:24:51.046 INFO:tasks.workunit.client.0.smithi090.stdout:0/261: dwrite d1a/d2a/f27 [0,4194304] 0 2024-04-24T08:24:51.048 INFO:tasks.workunit.client.1.smithi149.stdout:2/498: mkdir d0/d6/db/d28/d7f/d9b 0 2024-04-24T08:24:51.049 INFO:tasks.workunit.client.0.smithi090.stdout:0/262: write f18 [2099550,54286] 0 2024-04-24T08:24:51.052 INFO:tasks.workunit.client.1.smithi149.stdout:7/549: rename d9/dc/d11/f58 to d9/dc/fb7 0 2024-04-24T08:24:51.055 INFO:tasks.workunit.client.1.smithi149.stdout:7/550: readlink d9/dc/d2e/l3d 0 2024-04-24T08:24:51.056 INFO:tasks.workunit.client.0.smithi090.stdout:5/290: dwrite d8/da/d18/f22 [4194304,4194304] 0 2024-04-24T08:24:51.056 INFO:tasks.workunit.client.0.smithi090.stdout:4/159: write d3/f15 [200369,96884] 0 2024-04-24T08:24:51.056 INFO:tasks.workunit.client.0.smithi090.stdout:4/160: chown d3 1 1 2024-04-24T08:24:51.058 INFO:tasks.workunit.client.0.smithi090.stdout:5/291: write f7 [7406849,67323] 0 2024-04-24T08:24:51.058 INFO:tasks.workunit.client.1.smithi149.stdout:1/674: getdents d0/d39/da0 0 2024-04-24T08:24:51.063 INFO:tasks.workunit.client.0.smithi090.stdout:8/181: mkdir d2/da/d13/d15/d24/d3e 0 2024-04-24T08:24:51.065 INFO:tasks.workunit.client.1.smithi149.stdout:2/499: creat d0/d6/db/d39/d78/d7c/f9c x:0 0 0 2024-04-24T08:24:51.070 INFO:tasks.workunit.client.1.smithi149.stdout:3/607: rename d0/d5/l51 to d0/d5/da/d19/d1b/d22/d38/ld7 0 2024-04-24T08:24:51.073 INFO:tasks.workunit.client.1.smithi149.stdout:8/479: dwrite d1/d1f/d31/f84 [0,4194304] 0 2024-04-24T08:24:51.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/207: dread d0/f18 [0,4194304] 0 2024-04-24T08:24:51.076 INFO:tasks.workunit.client.0.smithi090.stdout:4/161: dwrite d3/f16 [0,4194304] 0 2024-04-24T08:24:51.077 INFO:tasks.workunit.client.1.smithi149.stdout:7/551: chown d9/dc/d2e/f6f 316277 1 2024-04-24T08:24:51.079 INFO:tasks.workunit.client.0.smithi090.stdout:4/162: write d3/f13 [2806875,68238] 0 2024-04-24T08:24:51.082 INFO:tasks.workunit.client.1.smithi149.stdout:2/500: fsync d0/d6/db/d39/d3d/d58/f5e 0 2024-04-24T08:24:51.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/212: creat d0/d14/d17/d2a/d31/f41 x:0 0 0 2024-04-24T08:24:51.086 INFO:tasks.workunit.client.0.smithi090.stdout:0/263: stat d1a/d2a/d3c/l46 0 2024-04-24T08:24:51.086 INFO:tasks.workunit.client.0.smithi090.stdout:3/200: truncate d2/df/d1a/f28 277277 0 2024-04-24T08:24:51.087 INFO:tasks.workunit.client.0.smithi090.stdout:2/213: dread d0/d7/d24/f27 [0,4194304] 0 2024-04-24T08:24:51.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/214: write d0/ff [4038142,30787] 0 2024-04-24T08:24:51.096 INFO:tasks.workunit.client.0.smithi090.stdout:0/264: dread d1a/d2a/d35/f3e [0,4194304] 0 2024-04-24T08:24:51.096 INFO:tasks.workunit.client.1.smithi149.stdout:1/675: mkdir d0/d1b/d5d/dd6/ddb 0 2024-04-24T08:24:51.097 INFO:tasks.workunit.client.0.smithi090.stdout:0/265: write d1a/d2a/f27 [5236870,57840] 0 2024-04-24T08:24:51.098 INFO:tasks.workunit.client.1.smithi149.stdout:4/514: write d2/d24/d2e/d38/f71 [5053612,32614] 0 2024-04-24T08:24:51.102 INFO:tasks.workunit.client.1.smithi149.stdout:4/515: stat d2/d18/d1e/d3b/f78 0 2024-04-24T08:24:51.103 INFO:tasks.workunit.client.1.smithi149.stdout:7/552: mkdir d9/dc/d2e/d3c/db8 0 2024-04-24T08:24:51.105 INFO:tasks.workunit.client.0.smithi090.stdout:5/292: creat d8/da/d3a/d2d/d37/f63 x:0 0 0 2024-04-24T08:24:51.105 INFO:tasks.workunit.client.1.smithi149.stdout:7/553: stat d9/dc/d11/d26/l47 0 2024-04-24T08:24:51.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/676: dread d0/d1b/d4b/f58 [0,4194304] 0 2024-04-24T08:24:51.108 INFO:tasks.workunit.client.0.smithi090.stdout:1/227: mkdir d3/d1d/d54 0 2024-04-24T08:24:51.109 INFO:tasks.workunit.client.0.smithi090.stdout:1/228: fdatasync d3/d5/f14 0 2024-04-24T08:24:51.111 INFO:tasks.workunit.client.1.smithi149.stdout:3/608: unlink d0/d26/d3c/f65 0 2024-04-24T08:24:51.112 INFO:tasks.workunit.client.1.smithi149.stdout:1/677: write d0/d5/d4f/d60/f78 [962669,128505] 0 2024-04-24T08:24:51.115 INFO:tasks.workunit.client.0.smithi090.stdout:8/182: rename d2/da/f1c to d2/da/d13/d15/f3f 0 2024-04-24T08:24:51.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/480: mknod d1/d1f/d5e/d9c/ca5 0 2024-04-24T08:24:51.117 INFO:tasks.workunit.client.0.smithi090.stdout:7/276: symlink db/d17/d29/l6b 0 2024-04-24T08:24:51.120 INFO:tasks.workunit.client.0.smithi090.stdout:9/208: creat d0/d19/d33/f40 x:0 0 0 2024-04-24T08:24:51.122 INFO:tasks.workunit.client.1.smithi149.stdout:0/504: rename d2/d17/d41/f4d to d2/d9/f9d 0 2024-04-24T08:24:51.126 INFO:tasks.workunit.client.0.smithi090.stdout:4/163: truncate d3/df/f1e 1009767 0 2024-04-24T08:24:51.129 INFO:tasks.workunit.client.1.smithi149.stdout:1/678: creat d0/d1b/d5d/fdc x:0 0 0 2024-04-24T08:24:51.130 INFO:tasks.workunit.client.0.smithi090.stdout:3/201: creat d2/df/f3c x:0 0 0 2024-04-24T08:24:51.135 INFO:tasks.workunit.client.0.smithi090.stdout:2/215: symlink d0/d14/d36/l42 0 2024-04-24T08:24:51.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/266: rename d1a/d2a/d35/f49 to d1a/d2a/d35/f55 0 2024-04-24T08:24:51.141 INFO:tasks.workunit.client.0.smithi090.stdout:0/267: stat d1a/d2a 0 2024-04-24T08:24:51.144 INFO:tasks.workunit.client.0.smithi090.stdout:5/293: symlink d8/da/d3a/l64 0 2024-04-24T08:24:51.146 INFO:tasks.workunit.client.1.smithi149.stdout:2/501: rename d0/d6/db/d39/d78/d7c/f9c to d0/d6/db/d39/d3d/d44/d57/f9d 0 2024-04-24T08:24:51.148 INFO:tasks.workunit.client.0.smithi090.stdout:1/229: unlink d3/l23 0 2024-04-24T08:24:51.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/554: creat d9/dc/d11/d83/da6/d67/d69/dad/fb9 x:0 0 0 2024-04-24T08:24:51.153 INFO:tasks.workunit.client.0.smithi090.stdout:8/183: rename d2/da/d13/d15/f32 to d2/da/d13/d34/f40 0 2024-04-24T08:24:51.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/209: mknod d0/dc/d24/c41 0 2024-04-24T08:24:51.156 INFO:tasks.workunit.client.0.smithi090.stdout:9/210: dread - d0/d19/d20/d32/f3c zero size 2024-04-24T08:24:51.157 INFO:tasks.workunit.client.0.smithi090.stdout:4/164: unlink d3/f15 0 2024-04-24T08:24:51.157 INFO:tasks.workunit.client.0.smithi090.stdout:9/211: dread - d0/dc/f29 zero size 2024-04-24T08:24:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:2/502: rmdir d0/d6/db/d27/d64/d66 39 2024-04-24T08:24:51.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/212: dread - d0/f1b zero size 2024-04-24T08:24:51.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/202: symlink d2/d25/d2c/l3d 0 2024-04-24T08:24:51.162 INFO:tasks.workunit.client.0.smithi090.stdout:3/203: chown d2/d15/f16 9 1 2024-04-24T08:24:51.164 INFO:tasks.workunit.client.0.smithi090.stdout:2/216: chown d0/l16 62215 1 2024-04-24T08:24:51.165 INFO:tasks.workunit.client.1.smithi149.stdout:0/505: getdents d2/d9/df/d92 0 2024-04-24T08:24:51.168 INFO:tasks.workunit.client.1.smithi149.stdout:7/555: mkdir d9/dc/d11/d83/da6/d67/dba 0 2024-04-24T08:24:51.171 INFO:tasks.workunit.client.1.smithi149.stdout:8/481: rename d1/de/d18/d32/d63/d76/f86 to d1/d1f/fa6 0 2024-04-24T08:24:51.173 INFO:tasks.workunit.client.0.smithi090.stdout:5/294: creat d8/da/d18/f65 x:0 0 0 2024-04-24T08:24:51.175 INFO:tasks.workunit.client.1.smithi149.stdout:3/609: dwrite d0/d5/da/f28 [0,4194304] 0 2024-04-24T08:24:51.177 INFO:tasks.workunit.client.0.smithi090.stdout:8/184: mknod d2/da/d13/c41 0 2024-04-24T08:24:51.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/185: write d2/da/d13/d15/d24/f2f [3038208,9097] 0 2024-04-24T08:24:51.185 INFO:tasks.workunit.client.1.smithi149.stdout:8/482: symlink d1/de/d18/d32/d63/d76/la7 0 2024-04-24T08:24:51.187 INFO:tasks.workunit.client.0.smithi090.stdout:9/213: rename d0/d19/d20/l35 to d0/l42 0 2024-04-24T08:24:51.188 INFO:tasks.workunit.client.0.smithi090.stdout:3/204: creat d2/df/d21/f3e x:0 0 0 2024-04-24T08:24:51.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/610: creat d0/d5/da/d19/d76/fd8 x:0 0 0 2024-04-24T08:24:51.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/217: rmdir d0/d14 39 2024-04-24T08:24:51.191 INFO:tasks.workunit.client.1.smithi149.stdout:0/506: mkdir d2/d9/df/d40/d4a/d4e/d6f/d9e 0 2024-04-24T08:24:51.193 INFO:tasks.workunit.client.0.smithi090.stdout:5/295: dwrite d8/da/d18/d21/f48 [0,4194304] 0 2024-04-24T08:24:51.193 INFO:tasks.workunit.client.1.smithi149.stdout:2/503: write d0/d6/db/f15 [2241618,2921] 0 2024-04-24T08:24:51.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/611: mkdir d0/d5/da/d5a/d63/d6a/dd9 0 2024-04-24T08:24:51.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/165: chown d3/df/f1e 11 1 2024-04-24T08:24:51.200 INFO:tasks.workunit.client.1.smithi149.stdout:1/679: rename d0/d5/db/d18/d4d/d50/daf to d0/ddd 0 2024-04-24T08:24:51.200 INFO:tasks.workunit.client.1.smithi149.stdout:3/612: chown d0/d5/d53/d4c/fa8 12751 1 2024-04-24T08:24:51.202 INFO:tasks.workunit.client.0.smithi090.stdout:9/214: creat d0/d2a/f43 x:0 0 0 2024-04-24T08:24:51.202 INFO:tasks.workunit.client.0.smithi090.stdout:4/166: write d3/d18/d1d/f27 [232343,107356] 0 2024-04-24T08:24:51.203 INFO:tasks.workunit.client.1.smithi149.stdout:0/507: truncate d2/d9/df/d2f/f43 1223255 0 2024-04-24T08:24:51.205 INFO:tasks.workunit.client.1.smithi149.stdout:0/508: chown d2/d9/df/d48/f4f 138851 1 2024-04-24T08:24:51.206 INFO:tasks.workunit.client.1.smithi149.stdout:1/680: unlink d0/d5/db/f87 0 2024-04-24T08:24:51.207 INFO:tasks.workunit.client.1.smithi149.stdout:0/509: dread - d2/d9/df/d56/f80 zero size 2024-04-24T08:24:51.208 INFO:tasks.workunit.client.1.smithi149.stdout:2/504: dread d0/d6/db/d39/f8e [0,4194304] 0 2024-04-24T08:24:51.210 INFO:tasks.workunit.client.0.smithi090.stdout:5/296: creat d8/da/d14/d2f/f66 x:0 0 0 2024-04-24T08:24:51.211 INFO:tasks.workunit.client.1.smithi149.stdout:0/510: fdatasync d2/d9/df/d40/d4a/d4e/d6f/f8a 0 2024-04-24T08:24:51.212 INFO:tasks.workunit.client.1.smithi149.stdout:8/483: dread d1/d1f/f2c [0,4194304] 0 2024-04-24T08:24:51.212 INFO:tasks.workunit.client.0.smithi090.stdout:5/297: write d8/da/f1b [968168,110542] 0 2024-04-24T08:24:51.214 INFO:tasks.workunit.client.1.smithi149.stdout:8/484: dread - d1/de/d18/d32/d63/f85 zero size 2024-04-24T08:24:51.215 INFO:tasks.workunit.client.1.smithi149.stdout:1/681: chown d0/d39/da0/db8/dc5/cc9 211707820 1 2024-04-24T08:24:51.217 INFO:tasks.workunit.client.0.smithi090.stdout:1/230: creat d3/d1d/f55 x:0 0 0 2024-04-24T08:24:51.218 INFO:tasks.workunit.client.1.smithi149.stdout:7/556: write d9/dc/d11/d38/f4e [2066080,125848] 0 2024-04-24T08:24:51.219 INFO:tasks.workunit.client.0.smithi090.stdout:8/186: write d2/f2c [260457,113226] 0 2024-04-24T08:24:51.221 INFO:tasks.workunit.client.0.smithi090.stdout:0/268: getdents d1a 0 2024-04-24T08:24:51.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/298: creat d8/da/d3a/d36/d50/f67 x:0 0 0 2024-04-24T08:24:51.228 INFO:tasks.workunit.client.1.smithi149.stdout:3/613: write d0/d5/da/d19/d76/f9b [546903,37908] 0 2024-04-24T08:24:51.230 INFO:tasks.workunit.client.1.smithi149.stdout:2/505: rename d0/f22 to d0/d6/db/f9e 0 2024-04-24T08:24:51.230 INFO:tasks.workunit.client.0.smithi090.stdout:1/231: dread d3/f2e [0,4194304] 0 2024-04-24T08:24:51.234 INFO:tasks.workunit.client.0.smithi090.stdout:0/269: rename f13 to d1a/f56 0 2024-04-24T08:24:51.234 INFO:tasks.workunit.client.0.smithi090.stdout:3/205: dwrite d2/f24 [0,4194304] 0 2024-04-24T08:24:51.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/187: dread d2/da/d13/d15/f3f [0,4194304] 0 2024-04-24T08:24:51.234 INFO:tasks.workunit.client.1.smithi149.stdout:2/506: write d0/d6/db/f72 [2032435,86006] 0 2024-04-24T08:24:51.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/188: write d2/f12 [21198,54765] 0 2024-04-24T08:24:51.237 INFO:tasks.workunit.client.0.smithi090.stdout:5/299: creat d8/da/d3a/d36/d50/f68 x:0 0 0 2024-04-24T08:24:51.240 INFO:tasks.workunit.client.1.smithi149.stdout:7/557: dwrite d9/d43/d4f/d66/f6e [4194304,4194304] 0 2024-04-24T08:24:51.240 INFO:tasks.workunit.client.0.smithi090.stdout:2/218: getdents d0/d2e 0 2024-04-24T08:24:51.244 INFO:tasks.workunit.client.1.smithi149.stdout:7/558: write d9/dc/d11/d83/da6/d67/f9e [440345,71006] 0 2024-04-24T08:24:51.246 INFO:tasks.workunit.client.1.smithi149.stdout:5/661: sync 2024-04-24T08:24:51.247 INFO:tasks.workunit.client.1.smithi149.stdout:5/662: chown db/d12 595398 1 2024-04-24T08:24:51.249 INFO:tasks.workunit.client.1.smithi149.stdout:0/511: write d2/d9/d1a/d36/f38 [525901,11833] 0 2024-04-24T08:24:51.249 INFO:tasks.workunit.client.1.smithi149.stdout:5/663: chown db/d12/c38 1647 1 2024-04-24T08:24:51.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/206: symlink d2/d25/d33/l3f 0 2024-04-24T08:24:51.251 INFO:tasks.workunit.client.0.smithi090.stdout:0/270: dwrite f8 [0,4194304] 0 2024-04-24T08:24:51.252 INFO:tasks.workunit.client.0.smithi090.stdout:9/215: dread d0/fb [0,4194304] 0 2024-04-24T08:24:51.256 INFO:tasks.workunit.client.1.smithi149.stdout:0/512: write d2/d9/df/d48/f66 [4384945,65948] 0 2024-04-24T08:24:51.258 INFO:tasks.workunit.client.1.smithi149.stdout:8/485: dread d1/de/d18/d43/f88 [0,4194304] 0 2024-04-24T08:24:51.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/219: fdatasync d0/d14/f15 0 2024-04-24T08:24:51.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/300: rename d8/da/d3a/d36/d50/f67 to d8/da/d14/f69 0 2024-04-24T08:24:51.263 INFO:tasks.workunit.client.0.smithi090.stdout:5/301: readlink d8/d59/l5d 0 2024-04-24T08:24:51.263 INFO:tasks.workunit.client.1.smithi149.stdout:2/507: mkdir d0/d6/db/d27/d81/d9f 0 2024-04-24T08:24:51.264 INFO:tasks.workunit.client.1.smithi149.stdout:5/664: unlink db/d23/fd3 0 2024-04-24T08:24:51.264 INFO:tasks.workunit.client.1.smithi149.stdout:0/513: read d2/d9/d1a/d36/f38 [632744,92874] 0 2024-04-24T08:24:51.264 INFO:tasks.workunit.client.0.smithi090.stdout:5/302: stat d8/f10 0 2024-04-24T08:24:51.269 INFO:tasks.workunit.client.0.smithi090.stdout:3/207: dwrite d2/d15/f2e [0,4194304] 0 2024-04-24T08:24:51.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/232: symlink d3/d1d/l56 0 2024-04-24T08:24:51.275 INFO:tasks.workunit.client.1.smithi149.stdout:1/682: rename d0/d5/db/d18/cd9 to d0/d5/cde 0 2024-04-24T08:24:51.275 INFO:tasks.workunit.client.0.smithi090.stdout:1/233: truncate d3/d1d/d34/f3b 678199 0 2024-04-24T08:24:51.278 INFO:tasks.workunit.client.1.smithi149.stdout:8/486: rmdir d1/d1f/d31 39 2024-04-24T08:24:51.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/216: truncate d0/d19/f1d 632091 0 2024-04-24T08:24:51.282 INFO:tasks.workunit.client.1.smithi149.stdout:7/559: dwrite d9/dc/de/f31 [0,4194304] 0 2024-04-24T08:24:51.282 INFO:tasks.workunit.client.1.smithi149.stdout:8/487: dread - d1/de/d18/f7e zero size 2024-04-24T08:24:51.283 INFO:tasks.workunit.client.0.smithi090.stdout:8/189: truncate d2/da/d13/d34/f40 97913 0 2024-04-24T08:24:51.286 INFO:tasks.workunit.client.0.smithi090.stdout:2/220: rename d0/d7/d28/f3a to d0/d2c/f43 0 2024-04-24T08:24:51.288 INFO:tasks.workunit.client.0.smithi090.stdout:5/303: creat d8/da/d3a/d2d/d37/f6a x:0 0 0 2024-04-24T08:24:51.290 INFO:tasks.workunit.client.0.smithi090.stdout:9/217: truncate d0/f2 2202724 0 2024-04-24T08:24:51.291 INFO:tasks.workunit.client.1.smithi149.stdout:5/665: dwrite db/d12/fb0 [0,4194304] 0 2024-04-24T08:24:51.306 INFO:tasks.workunit.client.0.smithi090.stdout:1/234: rename d3/f26 to d3/d5/da/dc/f57 0 2024-04-24T08:24:51.306 INFO:tasks.workunit.client.0.smithi090.stdout:2/221: symlink d0/d2e/l44 0 2024-04-24T08:24:51.309 INFO:tasks.workunit.client.1.smithi149.stdout:5/666: dwrite db/d12/d1e/fc1 [0,4194304] 0 2024-04-24T08:24:51.309 INFO:tasks.workunit.client.1.smithi149.stdout:3/614: rename d0/d5/d53/d34/d3a/f33 to d0/fda 0 2024-04-24T08:24:51.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/667: dread db/d23/f80 [0,4194304] 0 2024-04-24T08:24:51.316 INFO:tasks.workunit.client.1.smithi149.stdout:3/615: truncate d0/d5/da/d19/d1b/d72/fcd 229137 0 2024-04-24T08:24:51.323 INFO:tasks.workunit.client.0.smithi090.stdout:1/235: rename d3/d5/da/d53 to d3/d5/d58 0 2024-04-24T08:24:51.323 INFO:tasks.workunit.client.1.smithi149.stdout:8/488: mknod d1/d1f/d25/d7a/ca8 0 2024-04-24T08:24:51.324 INFO:tasks.workunit.client.1.smithi149.stdout:3/616: write d0/d5/d1a/d70/da3/fb0 [244216,29594] 0 2024-04-24T08:24:51.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/683: rename d0/d39 to d0/ddf 0 2024-04-24T08:24:51.326 INFO:tasks.workunit.client.0.smithi090.stdout:1/236: write d3/d5/d3e/f18 [180876,65143] 0 2024-04-24T08:24:51.327 INFO:tasks.workunit.client.0.smithi090.stdout:3/208: link d2/l18 d2/d3a/l40 0 2024-04-24T08:24:51.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/209: chown d2/df/f22 95995 1 2024-04-24T08:24:51.328 INFO:tasks.workunit.client.1.smithi149.stdout:2/508: getdents d0/d6/db/d27 0 2024-04-24T08:24:51.329 INFO:tasks.workunit.client.0.smithi090.stdout:2/222: mknod d0/d14/d17/d2a/c45 0 2024-04-24T08:24:51.331 INFO:tasks.workunit.client.1.smithi149.stdout:7/560: rmdir d9/d43/d95/db5 0 2024-04-24T08:24:51.331 INFO:tasks.workunit.client.1.smithi149.stdout:3/617: read d0/d5/da/d59/fbf [579970,114879] 0 2024-04-24T08:24:51.334 INFO:tasks.workunit.client.0.smithi090.stdout:0/271: mkdir d1a/d4b/d53/d57 0 2024-04-24T08:24:51.334 INFO:tasks.workunit.client.1.smithi149.stdout:5/668: creat db/d12/d60/fe7 x:0 0 0 2024-04-24T08:24:51.336 INFO:tasks.workunit.client.1.smithi149.stdout:2/509: dread d0/d6/db/d28/f48 [0,4194304] 0 2024-04-24T08:24:51.337 INFO:tasks.workunit.client.0.smithi090.stdout:9/218: dread d0/d19/f28 [0,4194304] 0 2024-04-24T08:24:51.338 INFO:tasks.workunit.client.0.smithi090.stdout:1/237: dread d3/d5/f14 [0,4194304] 0 2024-04-24T08:24:51.338 INFO:tasks.workunit.client.0.smithi090.stdout:8/190: rmdir d2/da/d13/d15/d24/d3e 0 2024-04-24T08:24:51.338 INFO:tasks.workunit.client.1.smithi149.stdout:0/514: dwrite d2/d9/df/d2f/f43 [0,4194304] 0 2024-04-24T08:24:51.339 INFO:tasks.workunit.client.0.smithi090.stdout:1/238: truncate d3/f41 346337 0 2024-04-24T08:24:51.340 INFO:tasks.workunit.client.1.smithi149.stdout:2/510: write d0/d6/db/d28/f3a [610151,116546] 0 2024-04-24T08:24:51.340 INFO:tasks.workunit.client.0.smithi090.stdout:8/191: write d2/da/d13/d34/f3d [19151,95148] 0 2024-04-24T08:24:51.341 INFO:tasks.workunit.client.1.smithi149.stdout:0/515: chown d2/d9/df/d40/d4a/l71 749415 1 2024-04-24T08:24:51.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/192: truncate d2/f2c 606001 0 2024-04-24T08:24:51.343 INFO:tasks.workunit.client.1.smithi149.stdout:6/432: sync 2024-04-24T08:24:51.343 INFO:tasks.workunit.client.1.smithi149.stdout:9/594: sync 2024-04-24T08:24:51.345 INFO:tasks.workunit.client.1.smithi149.stdout:9/595: readlink d1/de/d12/l6b 0 2024-04-24T08:24:51.347 INFO:tasks.workunit.client.1.smithi149.stdout:9/596: fsync d1/da/dd/f24 0 2024-04-24T08:24:51.348 INFO:tasks.workunit.client.0.smithi090.stdout:3/210: rename d2/lc to d2/d25/d2c/l41 0 2024-04-24T08:24:51.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/597: write d1/da/dd/fac [1040693,18536] 0 2024-04-24T08:24:51.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/261: sync 2024-04-24T08:24:51.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/272: read d1a/f24 [5029372,78328] 0 2024-04-24T08:24:51.361 INFO:tasks.workunit.client.0.smithi090.stdout:2/223: dread d0/d7/f33 [0,4194304] 0 2024-04-24T08:24:51.362 INFO:tasks.workunit.client.0.smithi090.stdout:1/239: creat d3/d5/da/f59 x:0 0 0 2024-04-24T08:24:51.362 INFO:tasks.workunit.client.0.smithi090.stdout:2/224: chown d0/d2e/l44 23 1 2024-04-24T08:24:51.363 INFO:tasks.workunit.client.0.smithi090.stdout:8/193: mknod d2/da/d13/d34/c42 0 2024-04-24T08:24:51.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/225: stat d0/d14/l20 0 2024-04-24T08:24:51.364 INFO:tasks.workunit.client.0.smithi090.stdout:1/240: write d3/d5/da/dc/f27 [663696,64915] 0 2024-04-24T08:24:51.368 INFO:tasks.workunit.client.0.smithi090.stdout:5/304: getdents d8/da/d3a/d2d 0 2024-04-24T08:24:51.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/262: mkdir d8/d1d/d1f/d3e/d4f 0 2024-04-24T08:24:51.376 INFO:tasks.workunit.client.0.smithi090.stdout:1/241: mkdir d3/d1d/d34/d5a 0 2024-04-24T08:24:51.381 INFO:tasks.workunit.client.0.smithi090.stdout:3/211: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:24:51.382 INFO:tasks.workunit.client.0.smithi090.stdout:3/212: readlink d2/ld 0 2024-04-24T08:24:51.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/684: creat d0/ddf/da0/fe0 x:0 0 0 2024-04-24T08:24:51.392 INFO:tasks.workunit.client.1.smithi149.stdout:8/489: rename d1/de/d18/d43/l8a to d1/de/d18/d32/d63/la9 0 2024-04-24T08:24:51.402 INFO:tasks.workunit.client.1.smithi149.stdout:5/669: write db/d23/f80 [1451445,71256] 0 2024-04-24T08:24:51.404 INFO:tasks.workunit.client.1.smithi149.stdout:2/511: creat d0/d6/fa0 x:0 0 0 2024-04-24T08:24:51.405 INFO:tasks.workunit.client.1.smithi149.stdout:0/516: fdatasync d2/d9/f2d 0 2024-04-24T08:24:51.411 INFO:tasks.workunit.client.1.smithi149.stdout:3/618: write d0/d5/d53/d34/f3b [4103079,101586] 0 2024-04-24T08:24:51.415 INFO:tasks.workunit.client.1.smithi149.stdout:5/670: mkdir db/d12/d1e/d65/de8 0 2024-04-24T08:24:51.415 INFO:tasks.workunit.client.1.smithi149.stdout:3/619: read d0/d26/d3c/db1/fd0 [7213084,115152] 0 2024-04-24T08:24:51.416 INFO:tasks.workunit.client.1.smithi149.stdout:4/516: sync 2024-04-24T08:24:51.416 INFO:tasks.workunit.client.1.smithi149.stdout:6/433: creat d0/d1a/d28/d31/d4c/d4e/d62/d7a/f92 x:0 0 0 2024-04-24T08:24:51.416 INFO:tasks.workunit.client.1.smithi149.stdout:0/517: rmdir d2/d9/d1a/d36 39 2024-04-24T08:24:51.418 INFO:tasks.workunit.client.1.smithi149.stdout:9/598: getdents d1/d2/d33/d76 0 2024-04-24T08:24:51.424 INFO:tasks.workunit.client.1.smithi149.stdout:7/561: getdents d9/d43/d4f 0 2024-04-24T08:24:51.425 INFO:tasks.workunit.client.1.smithi149.stdout:5/671: truncate db/d12/d2f/d74/fa2 287134 0 2024-04-24T08:24:51.425 INFO:tasks.workunit.client.1.smithi149.stdout:9/599: symlink d1/d2/d33/d76/lcf 0 2024-04-24T08:24:51.429 INFO:tasks.workunit.client.1.smithi149.stdout:8/490: dwrite d1/d1f/d25/f80 [0,4194304] 0 2024-04-24T08:24:51.429 INFO:tasks.workunit.client.1.smithi149.stdout:1/685: truncate d0/d5/db/d18/d2f/fa4 3755744 0 2024-04-24T08:24:51.431 INFO:tasks.workunit.client.1.smithi149.stdout:3/620: write d0/fda [1084289,38480] 0 2024-04-24T08:24:51.431 INFO:tasks.workunit.client.1.smithi149.stdout:5/672: dread - db/d12/d60/d8f/da0/fda zero size 2024-04-24T08:24:51.431 INFO:tasks.workunit.client.1.smithi149.stdout:0/518: rename d2/d17/l22 to d2/d9/d1a/d36/d85/l9f 0 2024-04-24T08:24:51.435 INFO:tasks.workunit.client.1.smithi149.stdout:7/562: write d9/dc/d11/d26/fae [1762896,71828] 0 2024-04-24T08:24:51.440 INFO:tasks.workunit.client.1.smithi149.stdout:7/563: readlink d9/d43/d4f/d66/la9 0 2024-04-24T08:24:51.441 INFO:tasks.workunit.client.1.smithi149.stdout:9/600: rename d1/d2/d34/f78 to d1/d2/dc8/fd0 0 2024-04-24T08:24:51.442 INFO:tasks.workunit.client.0.smithi090.stdout:8/194: link d2/da/d13/d34/f3c d2/da/d13/d15/d24/f43 0 2024-04-24T08:24:51.443 INFO:tasks.workunit.client.1.smithi149.stdout:8/491: mknod d1/de/d18/caa 0 2024-04-24T08:24:51.444 INFO:tasks.workunit.client.0.smithi090.stdout:9/219: link d0/l42 d0/d19/d20/l44 0 2024-04-24T08:24:51.444 INFO:tasks.workunit.client.0.smithi090.stdout:5/305: write f6 [2109802,60220] 0 2024-04-24T08:24:51.444 INFO:tasks.workunit.client.0.smithi090.stdout:1/242: creat d3/d1d/d34/d5a/f5b x:0 0 0 2024-04-24T08:24:51.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/621: creat d0/d5/d53/d4c/fdb x:0 0 0 2024-04-24T08:24:51.449 INFO:tasks.workunit.client.1.smithi149.stdout:3/622: fdatasync d0/fda 0 2024-04-24T08:24:51.449 INFO:tasks.workunit.client.0.smithi090.stdout:2/226: getdents d0/d14/d17/d2a/d31 0 2024-04-24T08:24:51.451 INFO:tasks.workunit.client.0.smithi090.stdout:1/243: write d3/d1d/f37 [529278,101933] 0 2024-04-24T08:24:51.452 INFO:tasks.workunit.client.1.smithi149.stdout:6/434: dread d0/d1a/d28/d31/d38/f3a [0,4194304] 0 2024-04-24T08:24:51.453 INFO:tasks.workunit.client.0.smithi090.stdout:9/220: read d0/d19/f1e [5602873,24017] 0 2024-04-24T08:24:51.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/512: dwrite d0/d6/db/f10 [0,4194304] 0 2024-04-24T08:24:51.455 INFO:tasks.workunit.client.1.smithi149.stdout:5/673: write db/d12/f72 [130311,53327] 0 2024-04-24T08:24:51.456 INFO:tasks.workunit.client.0.smithi090.stdout:8/195: chown d2/da/d13/l33 49500689 1 2024-04-24T08:24:51.456 INFO:tasks.workunit.client.0.smithi090.stdout:3/213: dwrite d2/d15/f31 [0,4194304] 0 2024-04-24T08:24:51.456 INFO:tasks.workunit.client.0.smithi090.stdout:9/221: fdatasync d0/d2a/f43 0 2024-04-24T08:24:51.460 INFO:tasks.workunit.client.0.smithi090.stdout:5/306: mknod d8/da/d14/d2f/d44/c6b 0 2024-04-24T08:24:51.468 INFO:tasks.workunit.client.1.smithi149.stdout:9/601: creat d1/da/d13/fd1 x:0 0 0 2024-04-24T08:24:51.471 INFO:tasks.workunit.client.1.smithi149.stdout:4/517: dread d2/d18/d1e/d3c/f80 [0,4194304] 0 2024-04-24T08:24:51.473 INFO:tasks.workunit.client.0.smithi090.stdout:9/222: chown d0/l42 0 1 2024-04-24T08:24:51.475 INFO:tasks.workunit.client.0.smithi090.stdout:0/273: dread d1a/f30 [0,4194304] 0 2024-04-24T08:24:51.476 INFO:tasks.workunit.client.0.smithi090.stdout:3/214: unlink d2/df/f22 0 2024-04-24T08:24:51.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/435: dread d0/f1 [4194304,4194304] 0 2024-04-24T08:24:51.478 INFO:tasks.workunit.client.1.smithi149.stdout:2/513: unlink d0/d6/db/d39/d3d/d44/d57/f5a 0 2024-04-24T08:24:51.482 INFO:tasks.workunit.client.1.smithi149.stdout:0/519: rename d2/d9/df/d48/f4c to d2/d9/df/d40/d4a/d4e/fa0 0 2024-04-24T08:24:51.482 INFO:tasks.workunit.client.0.smithi090.stdout:2/227: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:51.483 INFO:tasks.workunit.client.1.smithi149.stdout:6/436: write d0/d1a/f2b [5180795,62316] 0 2024-04-24T08:24:51.484 INFO:tasks.workunit.client.0.smithi090.stdout:8/196: rename d2/da/d13/d15/l1a to d2/da/d13/d15/d24/l44 0 2024-04-24T08:24:51.487 INFO:tasks.workunit.client.1.smithi149.stdout:8/492: mknod d1/d1f/d5e/d92/cab 0 2024-04-24T08:24:51.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/307: creat d8/da/d3a/d36/d50/f6c x:0 0 0 2024-04-24T08:24:51.490 INFO:tasks.workunit.client.0.smithi090.stdout:5/308: readlink d8/da/d3a/d2d/l30 0 2024-04-24T08:24:51.490 INFO:tasks.workunit.client.1.smithi149.stdout:3/623: dwrite d0/d5/d53/d34/f3b [0,4194304] 0 2024-04-24T08:24:51.490 INFO:tasks.workunit.client.0.smithi090.stdout:3/215: dread d2/df/d1a/f2b [0,4194304] 0 2024-04-24T08:24:51.491 INFO:tasks.workunit.client.1.smithi149.stdout:7/564: dwrite d9/dc/d11/d38/f56 [0,4194304] 0 2024-04-24T08:24:51.494 INFO:tasks.workunit.client.1.smithi149.stdout:5/674: write db/d12/d2f/d74/f97 [373759,18425] 0 2024-04-24T08:24:51.497 INFO:tasks.workunit.client.1.smithi149.stdout:8/493: write d1/de/d18/d32/d65/f81 [4185213,61097] 0 2024-04-24T08:24:51.499 INFO:tasks.workunit.client.0.smithi090.stdout:9/223: rename d0/f4 to d0/d19/f45 0 2024-04-24T08:24:51.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/244: link d3/d25/d30/l49 d3/d5/d58/l5c 0 2024-04-24T08:24:51.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/245: write d3/d5/f28 [463967,23742] 0 2024-04-24T08:24:51.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/246: readlink d3/d1d/l56 0 2024-04-24T08:24:51.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/247: fsync d3/d5/d3e/f18 0 2024-04-24T08:24:51.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/248: readlink d3/d1d/d34/l48 0 2024-04-24T08:24:51.512 INFO:tasks.workunit.client.0.smithi090.stdout:2/228: dwrite d0/f22 [4194304,4194304] 0 2024-04-24T08:24:51.514 INFO:tasks.workunit.client.0.smithi090.stdout:2/229: write d0/d7/f32 [1079983,108192] 0 2024-04-24T08:24:51.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/309: mkdir d8/d56/d6d 0 2024-04-24T08:24:51.516 INFO:tasks.workunit.client.0.smithi090.stdout:2/230: read d0/d14/f15 [3815755,7531] 0 2024-04-24T08:24:51.517 INFO:tasks.workunit.client.1.smithi149.stdout:9/602: symlink d1/d2/dc8/dc9/ld2 0 2024-04-24T08:24:51.517 INFO:tasks.workunit.client.0.smithi090.stdout:3/216: symlink d2/d25/d2c/l42 0 2024-04-24T08:24:51.518 INFO:tasks.workunit.client.0.smithi090.stdout:2/231: read d0/d7/f32 [2221628,110995] 0 2024-04-24T08:24:51.520 INFO:tasks.workunit.client.1.smithi149.stdout:3/624: creat d0/d5/da/d19/d1b/d72/fdc x:0 0 0 2024-04-24T08:24:51.528 INFO:tasks.workunit.client.1.smithi149.stdout:1/686: rename d0/d5/db/d63/d98 to d0/d1b/d5d/de1 0 2024-04-24T08:24:51.530 INFO:tasks.workunit.client.1.smithi149.stdout:3/625: write d0/d5/d1a/d70/fba [1003053,22642] 0 2024-04-24T08:24:51.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/494: dwrite d1/de/d18/f45 [0,4194304] 0 2024-04-24T08:24:51.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/197: rename d2/da/d13/d15/l17 to d2/l45 0 2024-04-24T08:24:51.546 INFO:tasks.workunit.client.0.smithi090.stdout:7/277: sync 2024-04-24T08:24:51.546 INFO:tasks.workunit.client.0.smithi090.stdout:1/249: write d3/d5/da/dc/f47 [977089,29401] 0 2024-04-24T08:24:51.547 INFO:tasks.workunit.client.1.smithi149.stdout:0/520: dwrite d2/d3c/d57/f78 [0,4194304] 0 2024-04-24T08:24:51.547 INFO:tasks.workunit.client.1.smithi149.stdout:5/675: fsync db/d12/d1e/d85/fa5 0 2024-04-24T08:24:51.547 INFO:tasks.workunit.client.1.smithi149.stdout:7/565: rename d9/dc/d11/d38/c5e to d9/dc/d2e/cbb 0 2024-04-24T08:24:51.548 INFO:tasks.workunit.client.0.smithi090.stdout:2/232: mkdir d0/d14/d17/d2a/d31/d46 0 2024-04-24T08:24:51.556 INFO:tasks.workunit.client.0.smithi090.stdout:0/274: truncate ff 794774 0 2024-04-24T08:24:51.559 INFO:tasks.workunit.client.0.smithi090.stdout:9/224: truncate d0/f2 1616624 0 2024-04-24T08:24:51.560 INFO:tasks.workunit.client.0.smithi090.stdout:9/225: chown d0/c30 163476 1 2024-04-24T08:24:51.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/198: creat d2/da/d13/d15/d24/f46 x:0 0 0 2024-04-24T08:24:51.563 INFO:tasks.workunit.client.1.smithi149.stdout:3/626: symlink d0/d5/da/d19/d1b/d72/ldd 0 2024-04-24T08:24:51.564 INFO:tasks.workunit.client.1.smithi149.stdout:2/514: dread d0/f55 [0,4194304] 0 2024-04-24T08:24:51.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/603: dwrite d1/d4c/d91/fce [0,4194304] 0 2024-04-24T08:24:51.571 INFO:tasks.workunit.client.1.smithi149.stdout:5/676: readlink db/d12/d60/d8f/lba 0 2024-04-24T08:24:51.574 INFO:tasks.workunit.client.1.smithi149.stdout:4/518: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:51.574 INFO:tasks.workunit.client.0.smithi090.stdout:1/250: creat d3/d1d/d34/d5a/f5d x:0 0 0 2024-04-24T08:24:51.575 INFO:tasks.workunit.client.0.smithi090.stdout:4/167: sync 2024-04-24T08:24:51.575 INFO:tasks.workunit.client.1.smithi149.stdout:6/437: dread d0/f4 [0,4194304] 0 2024-04-24T08:24:51.579 INFO:tasks.workunit.client.1.smithi149.stdout:8/495: dwrite d1/d1f/f8d [0,4194304] 0 2024-04-24T08:24:51.580 INFO:tasks.workunit.client.0.smithi090.stdout:4/168: write d3/f13 [1592926,74880] 0 2024-04-24T08:24:51.580 INFO:tasks.workunit.client.0.smithi090.stdout:2/233: symlink d0/d7/l47 0 2024-04-24T08:24:51.584 INFO:tasks.workunit.client.1.smithi149.stdout:9/604: write d1/d2/d15/d1c/f20 [2035227,44922] 0 2024-04-24T08:24:51.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/519: write d2/d18/d1e/f33 [990633,5953] 0 2024-04-24T08:24:51.585 INFO:tasks.workunit.client.0.smithi090.stdout:4/169: read d3/f16 [3211982,8363] 0 2024-04-24T08:24:51.585 INFO:tasks.workunit.client.1.smithi149.stdout:1/687: dwrite d0/d5/df/fca [0,4194304] 0 2024-04-24T08:24:51.586 INFO:tasks.workunit.client.0.smithi090.stdout:8/199: dwrite d2/f10 [0,4194304] 0 2024-04-24T08:24:51.587 INFO:tasks.workunit.client.1.smithi149.stdout:0/521: symlink d2/d9/df/d40/d4a/d4e/la1 0 2024-04-24T08:24:51.587 INFO:tasks.workunit.client.1.smithi149.stdout:7/566: truncate d9/dc/d11/f6c 598657 0 2024-04-24T08:24:51.591 INFO:tasks.workunit.client.0.smithi090.stdout:7/278: fsync db/fc 0 2024-04-24T08:24:51.593 INFO:tasks.workunit.client.0.smithi090.stdout:5/310: link d8/da/d3a/l64 d8/da/d14/d2f/l6e 0 2024-04-24T08:24:51.593 INFO:tasks.workunit.client.1.smithi149.stdout:2/515: chown d0/d6/db/f42 105111686 1 2024-04-24T08:24:51.594 INFO:tasks.workunit.client.1.smithi149.stdout:4/520: truncate d2/d18/d1e/d3b/fa0 985737 0 2024-04-24T08:24:51.595 INFO:tasks.workunit.client.1.smithi149.stdout:0/522: write d2/d9/df/d40/f67 [1067514,123887] 0 2024-04-24T08:24:51.598 INFO:tasks.workunit.client.0.smithi090.stdout:3/217: truncate d2/df/d1a/f2b 1142199 0 2024-04-24T08:24:51.600 INFO:tasks.workunit.client.1.smithi149.stdout:2/516: write d0/d6/db/f1f [8288929,105748] 0 2024-04-24T08:24:51.600 INFO:tasks.workunit.client.0.smithi090.stdout:1/251: creat d3/d1d/d34/d5a/f5e x:0 0 0 2024-04-24T08:24:51.601 INFO:tasks.workunit.client.0.smithi090.stdout:1/252: write d3/d25/f4a [493475,25776] 0 2024-04-24T08:24:51.616 INFO:tasks.workunit.client.1.smithi149.stdout:3/627: write d0/d5/d53/d34/f35 [3083798,113396] 0 2024-04-24T08:24:51.617 INFO:tasks.workunit.client.0.smithi090.stdout:2/234: rename d0/d14/l3c to d0/d14/d17/l48 0 2024-04-24T08:24:51.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/226: write d0/f5 [174055,30859] 0 2024-04-24T08:24:51.619 INFO:tasks.workunit.client.1.smithi149.stdout:6/438: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d7a/f92 [0,4194304] 0 2024-04-24T08:24:51.620 INFO:tasks.workunit.client.0.smithi090.stdout:9/227: fdatasync d0/d19/f28 0 2024-04-24T08:24:51.624 INFO:tasks.workunit.client.1.smithi149.stdout:6/439: fsync d0/d1a/d28/d31/f82 0 2024-04-24T08:24:51.626 INFO:tasks.workunit.client.0.smithi090.stdout:7/279: mknod db/d17/d29/d38/d3e/d46/c6c 0 2024-04-24T08:24:51.628 INFO:tasks.workunit.client.1.smithi149.stdout:0/523: creat d2/d9/df/d40/fa2 x:0 0 0 2024-04-24T08:24:51.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/605: link d1/d2/d15/d1c/d42/f4b d1/da/d4a/fd3 0 2024-04-24T08:24:51.638 INFO:tasks.workunit.client.0.smithi090.stdout:0/275: mkdir d1a/d2a/d58 0 2024-04-24T08:24:51.639 INFO:tasks.workunit.client.0.smithi090.stdout:8/200: rename d2/da/d13/d34/c42 to d2/da/d13/c47 0 2024-04-24T08:24:51.640 INFO:tasks.workunit.client.0.smithi090.stdout:9/228: creat d0/d19/f46 x:0 0 0 2024-04-24T08:24:51.642 INFO:tasks.workunit.client.1.smithi149.stdout:1/688: mknod d0/d1b/d5d/dd6/ddb/ce2 0 2024-04-24T08:24:51.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/440: dread d0/fb [4194304,4194304] 0 2024-04-24T08:24:51.644 INFO:tasks.workunit.client.1.smithi149.stdout:7/567: symlink d9/dc/d11/d38/dac/lbc 0 2024-04-24T08:24:51.647 INFO:tasks.workunit.client.1.smithi149.stdout:6/441: readlink d0/d1a/d28/d31/d33/l49 0 2024-04-24T08:24:51.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/524: rename d2/d9/df/d92/c9b to d2/d9/df/d40/d4a/ca3 0 2024-04-24T08:24:51.649 INFO:tasks.workunit.client.1.smithi149.stdout:0/525: stat d2/d9/df/d48/f4f 0 2024-04-24T08:24:51.649 INFO:tasks.workunit.client.0.smithi090.stdout:5/311: rename d8/da/f1b to d8/da/d3a/d36/f6f 0 2024-04-24T08:24:51.651 INFO:tasks.workunit.client.0.smithi090.stdout:8/201: mkdir d2/d48 0 2024-04-24T08:24:51.652 INFO:tasks.workunit.client.1.smithi149.stdout:9/606: creat d1/d4c/d91/fd4 x:0 0 0 2024-04-24T08:24:51.652 INFO:tasks.workunit.client.1.smithi149.stdout:1/689: stat d0/d5/c10 0 2024-04-24T08:24:51.653 INFO:tasks.workunit.client.0.smithi090.stdout:9/229: rmdir d0/dc/d24 39 2024-04-24T08:24:51.655 INFO:tasks.workunit.client.1.smithi149.stdout:7/568: symlink d9/dc/de/lbd 0 2024-04-24T08:24:51.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/280: link db/d17/d29/f3b db/d17/d39/d59/f6d 0 2024-04-24T08:24:51.658 INFO:tasks.workunit.client.0.smithi090.stdout:7/281: readlink db/d17/l6a 0 2024-04-24T08:24:51.658 INFO:tasks.workunit.client.1.smithi149.stdout:6/442: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93 0 2024-04-24T08:24:51.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/276: creat d1a/d4b/d53/d57/f59 x:0 0 0 2024-04-24T08:24:51.659 INFO:tasks.workunit.client.1.smithi149.stdout:6/443: chown d0/d1a 33 1 2024-04-24T08:24:51.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/526: creat d2/d9/d1a/d36/d85/fa4 x:0 0 0 2024-04-24T08:24:51.662 INFO:tasks.workunit.client.0.smithi090.stdout:2/235: dread d0/d7/f11 [0,4194304] 0 2024-04-24T08:24:51.663 INFO:tasks.workunit.client.1.smithi149.stdout:3/628: dwrite d0/d5/d53/f3f [0,4194304] 0 2024-04-24T08:24:51.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/690: creat d0/ddf/da0/fe3 x:0 0 0 2024-04-24T08:24:51.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/312: creat d8/da/d14/d2f/d44/f70 x:0 0 0 2024-04-24T08:24:51.667 INFO:tasks.workunit.client.0.smithi090.stdout:0/277: rmdir d1a/d4b/d53 39 2024-04-24T08:24:51.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/313: truncate d8/da/d18/f1e 1618622 0 2024-04-24T08:24:51.667 INFO:tasks.workunit.client.1.smithi149.stdout:0/527: read d2/d9/df/d40/f67 [1174866,54141] 0 2024-04-24T08:24:51.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/278: write d1a/f3d [2051996,13025] 0 2024-04-24T08:24:51.669 INFO:tasks.workunit.client.1.smithi149.stdout:9/607: mkdir d1/d2/d15/d1c/d42/dae/dd5 0 2024-04-24T08:24:51.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/279: fsync f8 0 2024-04-24T08:24:51.674 INFO:tasks.workunit.client.1.smithi149.stdout:7/569: dwrite d9/d43/f7e [4194304,4194304] 0 2024-04-24T08:24:51.674 INFO:tasks.workunit.client.1.smithi149.stdout:1/691: creat d0/da/fe4 x:0 0 0 2024-04-24T08:24:51.674 INFO:tasks.workunit.client.1.smithi149.stdout:6/444: fsync d0/d1a/f2e 0 2024-04-24T08:24:51.679 INFO:tasks.workunit.client.0.smithi090.stdout:5/314: creat d8/da/d3a/d36/d50/f71 x:0 0 0 2024-04-24T08:24:51.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/253: dread d3/f4 [4194304,4194304] 0 2024-04-24T08:24:51.681 INFO:tasks.workunit.client.0.smithi090.stdout:5/315: read - d8/da/d3a/f43 zero size 2024-04-24T08:24:51.683 INFO:tasks.workunit.client.0.smithi090.stdout:5/316: dread - d8/da/d18/d21/f41 zero size 2024-04-24T08:24:51.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/317: readlink d8/l32 0 2024-04-24T08:24:51.685 INFO:tasks.workunit.client.1.smithi149.stdout:1/692: mkdir d0/d1b/d5d/de5 0 2024-04-24T08:24:51.686 INFO:tasks.workunit.client.1.smithi149.stdout:7/570: creat d9/dc/de/d44/fbe x:0 0 0 2024-04-24T08:24:51.686 INFO:tasks.workunit.client.1.smithi149.stdout:6/445: rmdir d0/d1a/d28/d4a 39 2024-04-24T08:24:51.687 INFO:tasks.workunit.client.1.smithi149.stdout:1/693: fsync d0/ddd/f56 0 2024-04-24T08:24:51.688 INFO:tasks.workunit.client.0.smithi090.stdout:5/318: stat d8/da/d18/c38 0 2024-04-24T08:24:51.691 INFO:tasks.workunit.client.1.smithi149.stdout:6/446: chown d0/d1a/f29 8273017 1 2024-04-24T08:24:51.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/571: symlink d9/dc/d11/d26/lbf 0 2024-04-24T08:24:51.696 INFO:tasks.workunit.client.1.smithi149.stdout:9/608: rename d1/d2/f41 to d1/d2/fd6 0 2024-04-24T08:24:51.697 INFO:tasks.workunit.client.1.smithi149.stdout:3/629: getdents d0/d5/d1a/d70/da3 0 2024-04-24T08:24:51.698 INFO:tasks.workunit.client.0.smithi090.stdout:2/236: dwrite d0/d7/f33 [0,4194304] 0 2024-04-24T08:24:51.701 INFO:tasks.workunit.client.1.smithi149.stdout:7/572: creat d9/dc/d2e/d51/fc0 x:0 0 0 2024-04-24T08:24:51.703 INFO:tasks.workunit.client.1.smithi149.stdout:9/609: creat d1/d99/fd7 x:0 0 0 2024-04-24T08:24:51.705 INFO:tasks.workunit.client.1.smithi149.stdout:3/630: rmdir d0/d88 39 2024-04-24T08:24:51.706 INFO:tasks.workunit.client.1.smithi149.stdout:6/447: creat d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/f94 x:0 0 0 2024-04-24T08:24:51.706 INFO:tasks.workunit.client.1.smithi149.stdout:9/610: readlink d1/d2/d33/d5d/l9f 0 2024-04-24T08:24:51.707 INFO:tasks.workunit.client.1.smithi149.stdout:7/573: creat d9/dc/d7b/fc1 x:0 0 0 2024-04-24T08:24:51.714 INFO:tasks.workunit.client.1.smithi149.stdout:8/496: dwrite d1/de/f38 [0,4194304] 0 2024-04-24T08:24:51.714 INFO:tasks.workunit.client.1.smithi149.stdout:7/574: truncate d9/dc/de/d44/fab 534105 0 2024-04-24T08:24:51.717 INFO:tasks.workunit.client.0.smithi090.stdout:2/237: link d0/d14/f2b d0/d2e/f49 0 2024-04-24T08:24:51.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/631: write d0/d26/d3c/db1/fd0 [10119585,113454] 0 2024-04-24T08:24:51.721 INFO:tasks.workunit.client.1.smithi149.stdout:9/611: creat d1/d2/d15/d1c/d42/dae/fd8 x:0 0 0 2024-04-24T08:24:51.722 INFO:tasks.workunit.client.1.smithi149.stdout:6/448: creat d0/d1a/d28/f95 x:0 0 0 2024-04-24T08:24:51.723 INFO:tasks.workunit.client.1.smithi149.stdout:8/497: fdatasync d1/f61 0 2024-04-24T08:24:51.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/632: chown d0/d5/d53/d4c 3217110 1 2024-04-24T08:24:51.727 INFO:tasks.workunit.client.0.smithi090.stdout:2/238: symlink d0/d2e/l4a 0 2024-04-24T08:24:51.732 INFO:tasks.workunit.client.1.smithi149.stdout:4/521: dwrite d2/d10/d1a/f36 [4194304,4194304] 0 2024-04-24T08:24:51.733 INFO:tasks.workunit.client.0.smithi090.stdout:0/280: chown d1a/f30 63059 1 2024-04-24T08:24:51.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/239: mknod d0/d14/d17/d2a/c4b 0 2024-04-24T08:24:51.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/498: mknod d1/d1f/d22/cac 0 2024-04-24T08:24:51.737 INFO:tasks.workunit.client.1.smithi149.stdout:3/633: fdatasync d0/d5/da/f14 0 2024-04-24T08:24:51.740 INFO:tasks.workunit.client.1.smithi149.stdout:3/634: read - d0/d5/da/f14 zero size 2024-04-24T08:24:51.740 INFO:tasks.workunit.client.0.smithi090.stdout:2/240: rmdir d0/d7/d24 39 2024-04-24T08:24:51.742 INFO:tasks.workunit.client.0.smithi090.stdout:2/241: chown d0/d14/d17/f1b 92802 1 2024-04-24T08:24:51.743 INFO:tasks.workunit.client.1.smithi149.stdout:3/635: dread - d0/d5/da/d19/d76/fd8 zero size 2024-04-24T08:24:51.745 INFO:tasks.workunit.client.0.smithi090.stdout:2/242: write d0/d14/d17/f18 [2111367,118819] 0 2024-04-24T08:24:51.745 INFO:tasks.workunit.client.0.smithi090.stdout:0/281: unlink c11 0 2024-04-24T08:24:51.746 INFO:tasks.workunit.client.1.smithi149.stdout:5/677: truncate db/d12/d22/d45/d5d/f63 769968 0 2024-04-24T08:24:51.750 INFO:tasks.workunit.client.0.smithi090.stdout:0/282: chown d1a/c33 2 1 2024-04-24T08:24:51.753 INFO:tasks.workunit.client.0.smithi090.stdout:0/283: readlink d1a/d2a/d3c/l46 0 2024-04-24T08:24:51.754 INFO:tasks.workunit.client.1.smithi149.stdout:3/636: fsync d0/d5/da/d59/fc9 0 2024-04-24T08:24:51.756 INFO:tasks.workunit.client.1.smithi149.stdout:3/637: chown d0/d5/d53/d4c/fdb 121148 1 2024-04-24T08:24:51.758 INFO:tasks.workunit.client.0.smithi090.stdout:0/284: chown d1a/d2a/c39 49 1 2024-04-24T08:24:51.761 INFO:tasks.workunit.client.0.smithi090.stdout:3/218: dwrite d2/df/d1a/f28 [0,4194304] 0 2024-04-24T08:24:51.761 INFO:tasks.workunit.client.1.smithi149.stdout:6/449: getdents d0/d1a/d28/d31 0 2024-04-24T08:24:51.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/243: dread d0/d14/d17/f18 [0,4194304] 0 2024-04-24T08:24:51.763 INFO:tasks.workunit.client.1.smithi149.stdout:4/522: getdents d2/d18/d1e/d3c 0 2024-04-24T08:24:51.764 INFO:tasks.workunit.client.0.smithi090.stdout:2/244: chown d0/d2e/f49 9722 1 2024-04-24T08:24:51.766 INFO:tasks.workunit.client.1.smithi149.stdout:4/523: write d2/d18/d1e/d3b/f78 [246115,101565] 0 2024-04-24T08:24:51.768 INFO:tasks.workunit.client.0.smithi090.stdout:0/285: mknod d1a/c5a 0 2024-04-24T08:24:51.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/638: link d0/d5/fc7 d0/d5/da/d19/d1b/d22/dd4/fde 0 2024-04-24T08:24:51.769 INFO:tasks.workunit.client.1.smithi149.stdout:4/524: write d2/d24/d2e/d38/f5c [868985,119444] 0 2024-04-24T08:24:51.771 INFO:tasks.workunit.client.0.smithi090.stdout:3/219: creat d2/d25/d33/f43 x:0 0 0 2024-04-24T08:24:51.772 INFO:tasks.workunit.client.1.smithi149.stdout:4/525: chown d2/d18/d1e/d3c/d48/d69/c85 811335718 1 2024-04-24T08:24:51.772 INFO:tasks.workunit.client.0.smithi090.stdout:8/202: write d2/f36 [928991,21587] 0 2024-04-24T08:24:51.774 INFO:tasks.workunit.client.0.smithi090.stdout:0/286: creat d1a/d2a/d3c/f5b x:0 0 0 2024-04-24T08:24:51.775 INFO:tasks.workunit.client.1.smithi149.stdout:6/450: creat d0/d1a/d28/d31/d4c/d4e/f96 x:0 0 0 2024-04-24T08:24:51.775 INFO:tasks.workunit.client.0.smithi090.stdout:8/203: write d2/f3 [1033476,128937] 0 2024-04-24T08:24:51.776 INFO:tasks.workunit.client.0.smithi090.stdout:7/282: truncate db/d17/d39/f49 3858923 0 2024-04-24T08:24:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:51 smithi149 ceph-mon[31282]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 151 MiB/s wr, 273 op/s 2024-04-24T08:24:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:51.778 INFO:tasks.workunit.client.1.smithi149.stdout:4/526: dread - d2/d18/d1e/d3c/d48/d69/f77 zero size 2024-04-24T08:24:51.778 INFO:tasks.workunit.client.0.smithi090.stdout:2/245: mknod d0/c4c 0 2024-04-24T08:24:51.781 INFO:tasks.workunit.client.1.smithi149.stdout:4/527: chown d2/c2b 21400655 1 2024-04-24T08:24:51.781 INFO:tasks.workunit.client.1.smithi149.stdout:0/528: truncate d2/d9/df/d56/f5f 698227 0 2024-04-24T08:24:51.783 INFO:tasks.workunit.client.0.smithi090.stdout:1/254: truncate d3/d5/da/dc/f3a 1373913 0 2024-04-24T08:24:51.784 INFO:tasks.workunit.client.1.smithi149.stdout:6/451: mkdir d0/d1a/d28/d31/d4c/d80/d6f/d97 0 2024-04-24T08:24:51.785 INFO:tasks.workunit.client.0.smithi090.stdout:7/283: mknod db/d54/c6e 0 2024-04-24T08:24:51.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/204: creat d2/da/d13/d15/f49 x:0 0 0 2024-04-24T08:24:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:51 smithi090 ceph-mon[25016]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 151 MiB/s wr, 273 op/s 2024-04-24T08:24:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:24:51.793 INFO:tasks.workunit.client.0.smithi090.stdout:0/287: creat d1a/d2a/d58/f5c x:0 0 0 2024-04-24T08:24:51.794 INFO:tasks.workunit.client.1.smithi149.stdout:1/694: dwrite d0/d5/db/f8c [0,4194304] 0 2024-04-24T08:24:51.797 INFO:tasks.workunit.client.0.smithi090.stdout:6/263: sync 2024-04-24T08:24:51.798 INFO:tasks.workunit.client.0.smithi090.stdout:7/284: unlink db/d17/l1c 0 2024-04-24T08:24:51.798 INFO:tasks.workunit.client.1.smithi149.stdout:6/452: symlink d0/d1a/d28/l98 0 2024-04-24T08:24:51.801 INFO:tasks.workunit.client.0.smithi090.stdout:8/205: unlink d2/da/d13/d15/d24/f43 0 2024-04-24T08:24:51.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/528: dwrite d2/d24/f9b [4194304,4194304] 0 2024-04-24T08:24:51.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/529: rename d2/d9/df/d40/d4a/d4e/la1 to d2/d17/la5 0 2024-04-24T08:24:51.824 INFO:tasks.workunit.client.1.smithi149.stdout:9/612: dwrite d1/da/f3e [0,4194304] 0 2024-04-24T08:24:51.830 INFO:tasks.workunit.client.0.smithi090.stdout:7/285: rename db/d17/d39/f66 to db/d17/d29/d38/d3e/d46/f6f 0 2024-04-24T08:24:51.833 INFO:tasks.workunit.client.1.smithi149.stdout:3/639: truncate d0/d5/da/f97 2455633 0 2024-04-24T08:24:51.834 INFO:tasks.workunit.client.0.smithi090.stdout:3/220: getdents d2/df/d1a 0 2024-04-24T08:24:51.835 INFO:tasks.workunit.client.0.smithi090.stdout:3/221: stat d2/d25/c35 0 2024-04-24T08:24:51.836 INFO:tasks.workunit.client.1.smithi149.stdout:1/695: rename d0/d1b/d4b/c5b to d0/d1b/d4b/da1/ce6 0 2024-04-24T08:24:51.841 INFO:tasks.workunit.client.0.smithi090.stdout:0/288: rmdir d1a/d4b/d53/d57 39 2024-04-24T08:24:51.845 INFO:tasks.workunit.client.0.smithi090.stdout:7/286: chown db/d17/d29/d32/f61 469 1 2024-04-24T08:24:51.845 INFO:tasks.workunit.client.1.smithi149.stdout:8/499: dwrite d1/d1f/d22/f5f [0,4194304] 0 2024-04-24T08:24:51.847 INFO:tasks.workunit.client.0.smithi090.stdout:7/287: write db/d17/d39/f41 [382228,92659] 0 2024-04-24T08:24:51.849 INFO:tasks.workunit.client.0.smithi090.stdout:7/288: chown db/l13 11942 1 2024-04-24T08:24:51.850 INFO:tasks.workunit.client.0.smithi090.stdout:7/289: stat db/d17/d29/d32/f40 0 2024-04-24T08:24:51.850 INFO:tasks.workunit.client.1.smithi149.stdout:4/529: mkdir d2/da8 0 2024-04-24T08:24:51.850 INFO:tasks.workunit.client.0.smithi090.stdout:7/290: fdatasync f7 0 2024-04-24T08:24:51.851 INFO:tasks.workunit.client.1.smithi149.stdout:4/530: fsync d2/d10/d1a/da2/fa3 0 2024-04-24T08:24:51.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/289: chown d1a/d2a/c28 793559621 1 2024-04-24T08:24:51.860 INFO:tasks.workunit.client.1.smithi149.stdout:0/530: mknod d2/d9/df/ca6 0 2024-04-24T08:24:51.864 INFO:tasks.workunit.client.0.smithi090.stdout:7/291: rename db/d17/d29/f3b to db/d17/d1b/f70 0 2024-04-24T08:24:51.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/255: dwrite d3/d5/da/dc/f3a [0,4194304] 0 2024-04-24T08:24:51.869 INFO:tasks.workunit.client.1.smithi149.stdout:3/640: mknod d0/d5/da/cdf 0 2024-04-24T08:24:51.871 INFO:tasks.workunit.client.1.smithi149.stdout:3/641: readlink d0/d5/d53/d34/l6c 0 2024-04-24T08:24:51.873 INFO:tasks.workunit.client.1.smithi149.stdout:4/531: dwrite d2/d10/d1a/da2/fa3 [0,4194304] 0 2024-04-24T08:24:51.873 INFO:tasks.workunit.client.1.smithi149.stdout:8/500: mkdir d1/de/d18/d32/d65/dad 0 2024-04-24T08:24:51.875 INFO:tasks.workunit.client.1.smithi149.stdout:3/642: readlink d0/d5/da/d19/d80/lb9 0 2024-04-24T08:24:51.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/256: dwrite d3/d5/f14 [4194304,4194304] 0 2024-04-24T08:24:51.882 INFO:tasks.workunit.client.1.smithi149.stdout:1/696: creat d0/d1b/fe7 x:0 0 0 2024-04-24T08:24:51.886 INFO:tasks.workunit.client.0.smithi090.stdout:2/246: truncate d0/d14/f1d 1626138 0 2024-04-24T08:24:51.887 INFO:tasks.workunit.client.0.smithi090.stdout:0/290: creat d1a/d4b/d53/f5d x:0 0 0 2024-04-24T08:24:51.893 INFO:tasks.workunit.client.1.smithi149.stdout:3/643: readlink d0/d5/da/d19/d1b/d22/d38/l4a 0 2024-04-24T08:24:51.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/532: dwrite d2/d18/d1e/d3b/f7c [4194304,4194304] 0 2024-04-24T08:24:51.897 INFO:tasks.workunit.client.0.smithi090.stdout:8/206: truncate d2/da/d13/d15/d24/f2f 720238 0 2024-04-24T08:24:51.899 INFO:tasks.workunit.client.0.smithi090.stdout:3/222: write d2/f17 [1162180,40896] 0 2024-04-24T08:24:51.899 INFO:tasks.workunit.client.0.smithi090.stdout:1/257: dread d3/f4 [4194304,4194304] 0 2024-04-24T08:24:51.901 INFO:tasks.workunit.client.0.smithi090.stdout:8/207: truncate d2/f36 985082 0 2024-04-24T08:24:51.905 INFO:tasks.workunit.client.1.smithi149.stdout:1/697: readlink d0/l2 0 2024-04-24T08:24:51.906 INFO:tasks.workunit.client.1.smithi149.stdout:0/531: write d2/d17/f5e [1031128,21488] 0 2024-04-24T08:24:51.906 INFO:tasks.workunit.client.0.smithi090.stdout:2/247: mkdir d0/d14/d4d 0 2024-04-24T08:24:51.906 INFO:tasks.workunit.client.0.smithi090.stdout:8/208: stat d2/da/l31 0 2024-04-24T08:24:51.907 INFO:tasks.workunit.client.1.smithi149.stdout:3/644: symlink d0/d5/d1a/le0 0 2024-04-24T08:24:51.909 INFO:tasks.workunit.client.0.smithi090.stdout:8/209: chown d2/da/d13/d15/d24/l3b 268 1 2024-04-24T08:24:51.913 INFO:tasks.workunit.client.0.smithi090.stdout:8/210: fdatasync d2/da/d13/f18 0 2024-04-24T08:24:51.913 INFO:tasks.workunit.client.1.smithi149.stdout:3/645: read - d0/d5/da/d19/d1b/d22/d38/f7e zero size 2024-04-24T08:24:51.914 INFO:tasks.workunit.client.0.smithi090.stdout:0/291: mknod d1a/c5e 0 2024-04-24T08:24:51.918 INFO:tasks.workunit.client.0.smithi090.stdout:7/292: dread f9 [0,4194304] 0 2024-04-24T08:24:51.918 INFO:tasks.workunit.client.1.smithi149.stdout:4/533: rename d2/d24/f2a to d2/d24/d2e/d3f/fa9 0 2024-04-24T08:24:51.918 INFO:tasks.workunit.client.1.smithi149.stdout:8/501: getdents d1/d1f 0 2024-04-24T08:24:51.919 INFO:tasks.workunit.client.1.smithi149.stdout:6/453: dwrite d0/d1a/f4d [0,4194304] 0 2024-04-24T08:24:51.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/613: dwrite d1/d2/d15/f30 [0,4194304] 0 2024-04-24T08:24:51.923 INFO:tasks.workunit.client.1.smithi149.stdout:4/534: fdatasync d2/d24/d2e/d38/f5c 0 2024-04-24T08:24:51.926 INFO:tasks.workunit.client.0.smithi090.stdout:3/223: dwrite d2/df/d21/f27 [0,4194304] 0 2024-04-24T08:24:51.927 INFO:tasks.workunit.client.0.smithi090.stdout:1/258: dwrite d3/f41 [0,4194304] 0 2024-04-24T08:24:51.927 INFO:tasks.workunit.client.0.smithi090.stdout:7/293: symlink db/d17/d39/d59/l71 0 2024-04-24T08:24:51.933 INFO:tasks.workunit.client.1.smithi149.stdout:3/646: chown d0/l23 91 1 2024-04-24T08:24:51.934 INFO:tasks.workunit.client.0.smithi090.stdout:1/259: fdatasync d3/f4 0 2024-04-24T08:24:51.934 INFO:tasks.workunit.client.0.smithi090.stdout:3/224: creat d2/d25/d33/f44 x:0 0 0 2024-04-24T08:24:51.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/292: dwrite f18 [0,4194304] 0 2024-04-24T08:24:51.936 INFO:tasks.workunit.client.0.smithi090.stdout:1/260: readlink d3/d5/da/l22 0 2024-04-24T08:24:51.937 INFO:tasks.workunit.client.0.smithi090.stdout:3/225: dread - d2/df/d21/f32 zero size 2024-04-24T08:24:51.938 INFO:tasks.workunit.client.0.smithi090.stdout:0/293: write d1a/d2a/d58/f5c [287004,15127] 0 2024-04-24T08:24:51.940 INFO:tasks.workunit.client.0.smithi090.stdout:3/226: fsync d2/df/d21/f30 0 2024-04-24T08:24:51.941 INFO:tasks.workunit.client.0.smithi090.stdout:0/294: write d1a/d2a/d3c/f54 [830807,68952] 0 2024-04-24T08:24:51.945 INFO:tasks.workunit.client.1.smithi149.stdout:3/647: write d0/d5/d53/d34/f35 [973164,64705] 0 2024-04-24T08:24:51.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/294: creat db/d17/d29/d38/f72 x:0 0 0 2024-04-24T08:24:51.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/295: chown db/d17/d29/d38 126972 1 2024-04-24T08:24:51.949 INFO:tasks.workunit.client.1.smithi149.stdout:1/698: rename d0/ddf/d53/dda to d0/d5/db/d18/d4d/de8 0 2024-04-24T08:24:51.952 INFO:tasks.workunit.client.0.smithi090.stdout:1/261: unlink d3/d1d/d34/l44 0 2024-04-24T08:24:51.952 INFO:tasks.workunit.client.1.smithi149.stdout:1/699: read - d0/d5/db/d18/fb9 zero size 2024-04-24T08:24:51.955 INFO:tasks.workunit.client.1.smithi149.stdout:1/700: fdatasync d0/d5/db/d18/fcc 0 2024-04-24T08:24:51.964 INFO:tasks.workunit.client.1.smithi149.stdout:6/454: mknod d0/d1a/d28/d31/d38/d54/c99 0 2024-04-24T08:24:51.966 INFO:tasks.workunit.client.1.smithi149.stdout:6/455: readlink d0/l22 0 2024-04-24T08:24:51.968 INFO:tasks.workunit.client.1.smithi149.stdout:6/456: fsync d0/f26 0 2024-04-24T08:24:51.982 INFO:tasks.workunit.client.0.smithi090.stdout:8/211: dwrite d2/da/d13/d15/d24/f2f [0,4194304] 0 2024-04-24T08:24:51.983 INFO:tasks.workunit.client.0.smithi090.stdout:3/227: link d2/d25/d33/c38 d2/d15/d36/c45 0 2024-04-24T08:24:51.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/248: write d0/d2e/f49 [513854,127860] 0 2024-04-24T08:24:51.983 INFO:tasks.workunit.client.1.smithi149.stdout:0/532: write d2/d9/d1a/d36/f38 [190275,54490] 0 2024-04-24T08:24:51.985 INFO:tasks.workunit.client.1.smithi149.stdout:1/701: creat d0/ddd/fe9 x:0 0 0 2024-04-24T08:24:51.985 INFO:tasks.workunit.client.0.smithi090.stdout:8/212: chown d2/l8 48 1 2024-04-24T08:24:51.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/295: rename d1a/f3d to d1a/f5f 0 2024-04-24T08:24:51.992 INFO:tasks.workunit.client.0.smithi090.stdout:1/262: dwrite d3/d5/f1a [0,4194304] 0 2024-04-24T08:24:51.996 INFO:tasks.workunit.client.0.smithi090.stdout:1/263: dread - d3/d5/da/f4d zero size 2024-04-24T08:24:51.997 INFO:tasks.workunit.client.1.smithi149.stdout:9/614: link d1/d2/d34/f92 d1/d2/d15/fd9 0 2024-04-24T08:24:52.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/296: write fa [2109455,67614] 0 2024-04-24T08:24:52.005 INFO:tasks.workunit.client.0.smithi090.stdout:7/297: dread - db/d17/d29/d38/d3e/f58 zero size 2024-04-24T08:24:52.006 INFO:tasks.workunit.client.1.smithi149.stdout:8/502: creat d1/de/d18/fae x:0 0 0 2024-04-24T08:24:52.008 INFO:tasks.workunit.client.1.smithi149.stdout:3/648: creat d0/d5/da/d19/d1b/d22/dd4/fe1 x:0 0 0 2024-04-24T08:24:52.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/264: symlink d3/d25/l5f 0 2024-04-24T08:24:52.012 INFO:tasks.workunit.client.1.smithi149.stdout:3/649: chown d0/d5/d53/c4d 683400255 1 2024-04-24T08:24:52.013 INFO:tasks.workunit.client.0.smithi090.stdout:1/265: chown d3/d5/f33 92312 1 2024-04-24T08:24:52.014 INFO:tasks.workunit.client.1.smithi149.stdout:4/535: write d2/d10/d1a/f3e [242032,73022] 0 2024-04-24T08:24:52.015 INFO:tasks.workunit.client.0.smithi090.stdout:8/213: unlink d2/da/d13/c47 0 2024-04-24T08:24:52.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/298: rmdir db/d17/d29 39 2024-04-24T08:24:52.019 INFO:tasks.workunit.client.1.smithi149.stdout:9/615: dwrite d1/d2/d15/f3c [0,4194304] 0 2024-04-24T08:24:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:6/457: mknod d0/d1a/d28/c9a 0 2024-04-24T08:24:52.024 INFO:tasks.workunit.client.0.smithi090.stdout:1/266: dread d3/d5/da/dc/f47 [0,4194304] 0 2024-04-24T08:24:52.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/228: truncate d2/fa 4182261 0 2024-04-24T08:24:52.025 INFO:tasks.workunit.client.0.smithi090.stdout:1/267: stat d3/d5/da/c3c 0 2024-04-24T08:24:52.026 INFO:tasks.workunit.client.0.smithi090.stdout:1/268: stat d3/d5/da/dc/f27 0 2024-04-24T08:24:52.032 INFO:tasks.workunit.client.1.smithi149.stdout:4/536: dwrite d2/f6c [0,4194304] 0 2024-04-24T08:24:52.034 INFO:tasks.workunit.client.0.smithi090.stdout:1/269: write d3/d1d/d34/d5a/f5d [1006713,15955] 0 2024-04-24T08:24:52.035 INFO:tasks.workunit.client.0.smithi090.stdout:8/214: rename d2/da/d13/d15/c1f to d2/da/d13/d15/d24/c4a 0 2024-04-24T08:24:52.036 INFO:tasks.workunit.client.0.smithi090.stdout:2/249: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:52.036 INFO:tasks.workunit.client.1.smithi149.stdout:3/650: rename d0/d5/da/d19/d1b/d22/cb8 to d0/d5/d53/d34/d3a/d93/ce2 0 2024-04-24T08:24:52.038 INFO:tasks.workunit.client.0.smithi090.stdout:0/296: rename d1a/l2e to d1a/l60 0 2024-04-24T08:24:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:9/616: dwrite d1/da/d13/fd1 [0,4194304] 0 2024-04-24T08:24:52.044 INFO:tasks.workunit.client.0.smithi090.stdout:3/229: mknod d2/d25/d33/c46 0 2024-04-24T08:24:52.044 INFO:tasks.workunit.client.1.smithi149.stdout:6/458: creat d0/d1a/d28/d31/d33/f9b x:0 0 0 2024-04-24T08:24:52.045 INFO:tasks.workunit.client.1.smithi149.stdout:3/651: write d0/d5/da/d19/d76/f9b [719177,10028] 0 2024-04-24T08:24:52.050 INFO:tasks.workunit.client.0.smithi090.stdout:7/299: creat db/d17/d29/d32/f73 x:0 0 0 2024-04-24T08:24:52.052 INFO:tasks.workunit.client.0.smithi090.stdout:7/300: fsync db/d17/f28 0 2024-04-24T08:24:52.053 INFO:tasks.workunit.client.1.smithi149.stdout:0/533: creat d2/d9/d69/fa7 x:0 0 0 2024-04-24T08:24:52.057 INFO:tasks.workunit.client.0.smithi090.stdout:1/270: dwrite d3/d5/da/dc/f2b [0,4194304] 0 2024-04-24T08:24:52.061 INFO:tasks.workunit.client.1.smithi149.stdout:3/652: dread - d0/d5/da/d19/d1b/d22/dd4/fe1 zero size 2024-04-24T08:24:52.065 INFO:tasks.workunit.client.1.smithi149.stdout:9/617: unlink d1/da/c3f 0 2024-04-24T08:24:52.067 INFO:tasks.workunit.client.1.smithi149.stdout:2/517: sync 2024-04-24T08:24:52.069 INFO:tasks.workunit.client.0.smithi090.stdout:4/170: sync 2024-04-24T08:24:52.069 INFO:tasks.workunit.client.0.smithi090.stdout:9/230: sync 2024-04-24T08:24:52.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/319: sync 2024-04-24T08:24:52.069 INFO:tasks.workunit.client.0.smithi090.stdout:6/264: sync 2024-04-24T08:24:52.069 INFO:tasks.workunit.client.0.smithi090.stdout:3/230: creat d2/df/d1a/d37/f47 x:0 0 0 2024-04-24T08:24:52.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/297: dwrite d1a/f1b [0,4194304] 0 2024-04-24T08:24:52.074 INFO:tasks.workunit.client.1.smithi149.stdout:1/702: dwrite d0/ddf/d53/d61/f70 [0,4194304] 0 2024-04-24T08:24:52.082 INFO:tasks.workunit.client.1.smithi149.stdout:7/575: sync 2024-04-24T08:24:52.082 INFO:tasks.workunit.client.1.smithi149.stdout:5/678: sync 2024-04-24T08:24:52.084 INFO:tasks.workunit.client.1.smithi149.stdout:7/576: readlink d9/d43/l5f 0 2024-04-24T08:24:52.085 INFO:tasks.workunit.client.1.smithi149.stdout:8/503: getdents d1 0 2024-04-24T08:24:52.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/215: unlink d2/da/d13/d15/d24/l44 0 2024-04-24T08:24:52.088 INFO:tasks.workunit.client.0.smithi090.stdout:8/216: read d2/da/d13/d15/f3f [1500970,72702] 0 2024-04-24T08:24:52.095 INFO:tasks.workunit.client.0.smithi090.stdout:2/250: creat d0/d14/d4d/f4e x:0 0 0 2024-04-24T08:24:52.098 INFO:tasks.workunit.client.0.smithi090.stdout:7/301: write db/d17/f2d [1379868,127622] 0 2024-04-24T08:24:52.102 INFO:tasks.workunit.client.1.smithi149.stdout:9/618: symlink d1/d2/dc8/dc9/lda 0 2024-04-24T08:24:52.103 INFO:tasks.workunit.client.0.smithi090.stdout:1/271: creat d3/d5/d58/f60 x:0 0 0 2024-04-24T08:24:52.105 INFO:tasks.workunit.client.1.smithi149.stdout:2/518: dread d0/f8 [0,4194304] 0 2024-04-24T08:24:52.107 INFO:tasks.workunit.client.0.smithi090.stdout:4/171: symlink d3/l29 0 2024-04-24T08:24:52.107 INFO:tasks.workunit.client.1.smithi149.stdout:5/679: creat db/d12/d2f/d74/dc6/fe9 x:0 0 0 2024-04-24T08:24:52.108 INFO:tasks.workunit.client.0.smithi090.stdout:4/172: fdatasync d3/f9 0 2024-04-24T08:24:52.112 INFO:tasks.workunit.client.1.smithi149.stdout:7/577: mkdir d9/dc/d7b/dc2 0 2024-04-24T08:24:52.112 INFO:tasks.workunit.client.0.smithi090.stdout:5/320: creat d8/d25/f72 x:0 0 0 2024-04-24T08:24:52.113 INFO:tasks.workunit.client.1.smithi149.stdout:4/537: link d2/d18/d1e/d3c/d48/c6b d2/d18/d1e/caa 0 2024-04-24T08:24:52.117 INFO:tasks.workunit.client.1.smithi149.stdout:0/534: dwrite d2/d9/df/d40/d4a/f64 [0,4194304] 0 2024-04-24T08:24:52.118 INFO:tasks.workunit.client.0.smithi090.stdout:8/217: rename d2/f36 to d2/da/d13/f4b 0 2024-04-24T08:24:52.118 INFO:tasks.workunit.client.1.smithi149.stdout:6/459: dwrite d0/d1a/f27 [0,4194304] 0 2024-04-24T08:24:52.119 INFO:tasks.workunit.client.0.smithi090.stdout:8/218: fsync d2/da/f35 0 2024-04-24T08:24:52.122 INFO:tasks.workunit.client.0.smithi090.stdout:9/231: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:52.123 INFO:tasks.workunit.client.0.smithi090.stdout:7/302: symlink db/d17/d39/d59/l74 0 2024-04-24T08:24:52.124 INFO:tasks.workunit.client.0.smithi090.stdout:7/303: write db/d17/d29/f30 [688722,74366] 0 2024-04-24T08:24:52.126 INFO:tasks.workunit.client.0.smithi090.stdout:7/304: chown db/d17/d39/d59/l71 944938 1 2024-04-24T08:24:52.128 INFO:tasks.workunit.client.0.smithi090.stdout:1/272: mknod d3/d5/c61 0 2024-04-24T08:24:52.131 INFO:tasks.workunit.client.1.smithi149.stdout:5/680: stat db/d12/d1e/f3c 0 2024-04-24T08:24:52.134 INFO:tasks.workunit.client.1.smithi149.stdout:9/619: dread d1/da/dd/f69 [0,4194304] 0 2024-04-24T08:24:52.134 INFO:tasks.workunit.client.1.smithi149.stdout:8/504: mkdir d1/de/d18/daf 0 2024-04-24T08:24:52.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/251: dread d0/f22 [0,4194304] 0 2024-04-24T08:24:52.137 INFO:tasks.workunit.client.1.smithi149.stdout:8/505: stat d1/de/d18/f3e 0 2024-04-24T08:24:52.138 INFO:tasks.workunit.client.0.smithi090.stdout:4/173: chown d3/d18/d1d/d22/c28 14258 1 2024-04-24T08:24:52.138 INFO:tasks.workunit.client.0.smithi090.stdout:2/252: fsync d0/d14/f1f 0 2024-04-24T08:24:52.138 INFO:tasks.workunit.client.1.smithi149.stdout:1/703: dwrite d0/d5/db/d63/f86 [0,4194304] 0 2024-04-24T08:24:52.139 INFO:tasks.workunit.client.0.smithi090.stdout:2/253: write d0/d14/f2b [1126445,69089] 0 2024-04-24T08:24:52.143 INFO:tasks.workunit.client.1.smithi149.stdout:8/506: chown d1/f61 450782 1 2024-04-24T08:24:52.145 INFO:tasks.workunit.client.1.smithi149.stdout:0/535: chown d2/d9/d1a/c32 0 1 2024-04-24T08:24:52.146 INFO:tasks.workunit.client.1.smithi149.stdout:3/653: write d0/d5/da/d59/fc9 [700345,87061] 0 2024-04-24T08:24:52.149 INFO:tasks.workunit.client.1.smithi149.stdout:4/538: dwrite d2/d24/d2e/d38/f9c [0,4194304] 0 2024-04-24T08:24:52.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/254: dread d0/ff [0,4194304] 0 2024-04-24T08:24:52.154 INFO:tasks.workunit.client.0.smithi090.stdout:5/321: unlink d8/da/d3a/d36/l45 0 2024-04-24T08:24:52.154 INFO:tasks.workunit.client.1.smithi149.stdout:6/460: write d0/d1a/d28/d4a/f72 [1043906,88646] 0 2024-04-24T08:24:52.155 INFO:tasks.workunit.client.1.smithi149.stdout:2/519: write d0/d6/db/d28/f48 [470094,2684] 0 2024-04-24T08:24:52.160 INFO:tasks.workunit.client.1.smithi149.stdout:9/620: dread d1/d2/d34/fbe [0,4194304] 0 2024-04-24T08:24:52.162 INFO:tasks.workunit.client.1.smithi149.stdout:7/578: dread d9/fd [0,4194304] 0 2024-04-24T08:24:52.163 INFO:tasks.workunit.client.1.smithi149.stdout:2/520: write d0/d6/db/d27/d64/f74 [3320744,103533] 0 2024-04-24T08:24:52.163 INFO:tasks.workunit.client.0.smithi090.stdout:8/219: symlink d2/da/d13/l4c 0 2024-04-24T08:24:52.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/232: write d0/dc/d24/d2d/f3f [223977,8136] 0 2024-04-24T08:24:52.167 INFO:tasks.workunit.client.1.smithi149.stdout:7/579: readlink d9/dc/d11/d83/da6/d67/la3 0 2024-04-24T08:24:52.167 INFO:tasks.workunit.client.1.smithi149.stdout:1/704: rmdir d0/ddf/d53/d61 39 2024-04-24T08:24:52.167 INFO:tasks.workunit.client.0.smithi090.stdout:9/233: write d0/f34 [1769561,114757] 0 2024-04-24T08:24:52.168 INFO:tasks.workunit.client.0.smithi090.stdout:9/234: fsync d0/d19/d33/f36 0 2024-04-24T08:24:52.169 INFO:tasks.workunit.client.1.smithi149.stdout:1/705: write d0/d1b/d5d/fdc [137456,26489] 0 2024-04-24T08:24:52.170 INFO:tasks.workunit.client.1.smithi149.stdout:4/539: dread d2/f14 [0,4194304] 0 2024-04-24T08:24:52.171 INFO:tasks.workunit.client.0.smithi090.stdout:7/305: rename db/d17/d1b to db/d17/d29/d32/d5a/d75 0 2024-04-24T08:24:52.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/654: dread d0/d5/da/f28 [0,4194304] 0 2024-04-24T08:24:52.176 INFO:tasks.workunit.client.1.smithi149.stdout:8/507: rmdir d1/de/d18 39 2024-04-24T08:24:52.179 INFO:tasks.workunit.client.1.smithi149.stdout:2/521: dread d0/d6/db/f72 [0,4194304] 0 2024-04-24T08:24:52.180 INFO:tasks.workunit.client.1.smithi149.stdout:3/655: dread d0/d5/da/d19/d1b/d22/dd4/fde [0,4194304] 0 2024-04-24T08:24:52.184 INFO:tasks.workunit.client.0.smithi090.stdout:4/174: creat d3/d18/d1d/f2a x:0 0 0 2024-04-24T08:24:52.185 INFO:tasks.workunit.client.0.smithi090.stdout:4/175: chown d3/f13 242098 1 2024-04-24T08:24:52.187 INFO:tasks.workunit.client.1.smithi149.stdout:5/681: symlink db/d12/d22/lea 0 2024-04-24T08:24:52.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/176: read d3/df/f1e [692288,35116] 0 2024-04-24T08:24:52.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/298: creat d1a/d4b/f61 x:0 0 0 2024-04-24T08:24:52.195 INFO:tasks.workunit.client.1.smithi149.stdout:9/621: creat d1/de/fdb x:0 0 0 2024-04-24T08:24:52.196 INFO:tasks.workunit.client.1.smithi149.stdout:9/622: stat d1/da/d4a/dad/lcb 0 2024-04-24T08:24:52.198 INFO:tasks.workunit.client.0.smithi090.stdout:5/322: creat d8/da/d14/d2f/d44/f73 x:0 0 0 2024-04-24T08:24:52.202 INFO:tasks.workunit.client.1.smithi149.stdout:8/508: write d1/de/d18/d32/d63/d76/f9a [29882,3778] 0 2024-04-24T08:24:52.203 INFO:tasks.workunit.client.0.smithi090.stdout:9/235: rename d0/f34 to d0/dc/d24/d2d/f47 0 2024-04-24T08:24:52.204 INFO:tasks.workunit.client.1.smithi149.stdout:8/509: chown d1/de/l5b 142 1 2024-04-24T08:24:52.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/236: readlink d0/d19/d20/l44 0 2024-04-24T08:24:52.206 INFO:tasks.workunit.client.1.smithi149.stdout:0/536: creat d2/d9/df/d40/d4a/d4e/d6f/d9e/fa8 x:0 0 0 2024-04-24T08:24:52.212 INFO:tasks.workunit.client.1.smithi149.stdout:4/540: write d2/d10/f27 [1258587,79438] 0 2024-04-24T08:24:52.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/255: dwrite d0/d7/f32 [0,4194304] 0 2024-04-24T08:24:52.216 INFO:tasks.workunit.client.0.smithi090.stdout:2/256: chown d0/d14/d4d 1508686 1 2024-04-24T08:24:52.220 INFO:tasks.workunit.client.0.smithi090.stdout:2/257: write d0/d14/d17/d2a/d31/f41 [628047,101131] 0 2024-04-24T08:24:52.220 INFO:tasks.workunit.client.1.smithi149.stdout:5/682: creat db/d1f/feb x:0 0 0 2024-04-24T08:24:52.222 INFO:tasks.workunit.client.0.smithi090.stdout:3/231: getdents d2/d25 0 2024-04-24T08:24:52.224 INFO:tasks.workunit.client.1.smithi149.stdout:0/537: dwrite d2/d9/df/d40/fa2 [0,4194304] 0 2024-04-24T08:24:52.225 INFO:tasks.workunit.client.1.smithi149.stdout:5/683: truncate db/d12/f82 4532602 0 2024-04-24T08:24:52.225 INFO:tasks.workunit.client.1.smithi149.stdout:0/538: stat d2/d9 0 2024-04-24T08:24:52.225 INFO:tasks.workunit.client.0.smithi090.stdout:6/265: link d8/d1d/d29/l45 d8/d1d/d2f/l50 0 2024-04-24T08:24:52.228 INFO:tasks.workunit.client.0.smithi090.stdout:0/299: mkdir d1a/d2a/d35/d62 0 2024-04-24T08:24:52.230 INFO:tasks.workunit.client.1.smithi149.stdout:9/623: dread d1/da/dd/f32 [0,4194304] 0 2024-04-24T08:24:52.235 INFO:tasks.workunit.client.1.smithi149.stdout:2/522: symlink d0/d6/db/d28/d7f/d9b/la1 0 2024-04-24T08:24:52.236 INFO:tasks.workunit.client.1.smithi149.stdout:3/656: truncate d0/d5/d53/f47 263497 0 2024-04-24T08:24:52.237 INFO:tasks.workunit.client.1.smithi149.stdout:6/461: creat d0/d1a/d28/f9c x:0 0 0 2024-04-24T08:24:52.238 INFO:tasks.workunit.client.1.smithi149.stdout:2/523: stat d0/d6/db/d39/d78 0 2024-04-24T08:24:52.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/273: link d3/d25/d30/l36 d3/d1d/d54/l62 0 2024-04-24T08:24:52.240 INFO:tasks.workunit.client.0.smithi090.stdout:4/177: creat d3/d24/f2b x:0 0 0 2024-04-24T08:24:52.241 INFO:tasks.workunit.client.1.smithi149.stdout:6/462: chown d0/d1a/f4d 216409334 1 2024-04-24T08:24:52.242 INFO:tasks.workunit.client.0.smithi090.stdout:2/258: fsync d0/d2c/f43 0 2024-04-24T08:24:52.243 INFO:tasks.workunit.client.1.smithi149.stdout:5/684: unlink db/d35/cdb 0 2024-04-24T08:24:52.243 INFO:tasks.workunit.client.1.smithi149.stdout:2/524: fdatasync d0/d6/db/d39/d78/d8a/f99 0 2024-04-24T08:24:52.244 INFO:tasks.workunit.client.0.smithi090.stdout:2/259: fsync d0/d14/f34 0 2024-04-24T08:24:52.244 INFO:tasks.workunit.client.0.smithi090.stdout:3/232: creat d2/d25/f48 x:0 0 0 2024-04-24T08:24:52.245 INFO:tasks.workunit.client.1.smithi149.stdout:0/539: creat d2/d9/df/d40/d4a/fa9 x:0 0 0 2024-04-24T08:24:52.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/260: chown d0/d14/d17/d2a/c4b 7 1 2024-04-24T08:24:52.250 INFO:tasks.workunit.client.1.smithi149.stdout:1/706: link d0/d5/db/d63/f86 d0/d5/df/fea 0 2024-04-24T08:24:52.250 INFO:tasks.workunit.client.1.smithi149.stdout:0/540: write d2/d9/df/d2f/d6c/f9a [419048,72500] 0 2024-04-24T08:24:52.256 INFO:tasks.workunit.client.1.smithi149.stdout:8/510: dwrite d1/d1f/f68 [0,4194304] 0 2024-04-24T08:24:52.256 INFO:tasks.workunit.client.0.smithi090.stdout:1/274: dwrite d3/d5/da/dc/f2b [0,4194304] 0 2024-04-24T08:24:52.257 INFO:tasks.workunit.client.0.smithi090.stdout:8/220: creat d2/da/d13/d34/f4d x:0 0 0 2024-04-24T08:24:52.258 INFO:tasks.workunit.client.0.smithi090.stdout:5/323: symlink d8/da/d18/d21/d4c/l74 0 2024-04-24T08:24:52.259 INFO:tasks.workunit.client.0.smithi090.stdout:7/306: link db/d17/d29/l6b db/d17/d29/d38/d3e/d46/l76 0 2024-04-24T08:24:52.262 INFO:tasks.workunit.client.0.smithi090.stdout:9/237: symlink d0/dc/l48 0 2024-04-24T08:24:52.263 INFO:tasks.workunit.client.0.smithi090.stdout:9/238: read - d0/d19/f46 zero size 2024-04-24T08:24:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:5/685: dread db/d12/f5b [0,4194304] 0 2024-04-24T08:24:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:6/463: write d0/d1a/f69 [2130453,40675] 0 2024-04-24T08:24:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:4/541: truncate d2/d18/d1e/d3b/f7c 4861738 0 2024-04-24T08:24:52.270 INFO:tasks.workunit.client.1.smithi149.stdout:5/686: chown db/d12/d22/fcb 34994 1 2024-04-24T08:24:52.271 INFO:tasks.workunit.client.1.smithi149.stdout:4/542: write d2/d18/f64 [621264,108649] 0 2024-04-24T08:24:52.271 INFO:tasks.workunit.client.0.smithi090.stdout:4/178: rename d3/l12 to d3/d18/d1d/d22/l2c 0 2024-04-24T08:24:52.275 INFO:tasks.workunit.client.1.smithi149.stdout:0/541: rename d2/c15 to d2/d9/df/d2f/caa 0 2024-04-24T08:24:52.277 INFO:tasks.workunit.client.0.smithi090.stdout:1/275: dwrite d3/d5/f3f [0,4194304] 0 2024-04-24T08:24:52.278 INFO:tasks.workunit.client.0.smithi090.stdout:1/276: stat d3/d5/f3f 0 2024-04-24T08:24:52.279 INFO:tasks.workunit.client.1.smithi149.stdout:8/511: fdatasync d1/de/d18/d32/d65/d8b/d7d/f96 0 2024-04-24T08:24:52.286 INFO:tasks.workunit.client.0.smithi090.stdout:4/179: dwrite d3/f13 [0,4194304] 0 2024-04-24T08:24:52.291 INFO:tasks.workunit.client.1.smithi149.stdout:6/464: unlink d0/d1a/d28/d31/d33/l3d 0 2024-04-24T08:24:52.296 INFO:tasks.workunit.client.1.smithi149.stdout:9/624: truncate d1/de/d12/f3d 2992828 0 2024-04-24T08:24:52.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/233: creat d2/df/d1a/d37/f49 x:0 0 0 2024-04-24T08:24:52.300 INFO:tasks.workunit.client.0.smithi090.stdout:6/266: symlink d8/d1d/l51 0 2024-04-24T08:24:52.300 INFO:tasks.workunit.client.1.smithi149.stdout:2/525: dwrite d0/f35 [0,4194304] 0 2024-04-24T08:24:52.301 INFO:tasks.workunit.client.0.smithi090.stdout:0/300: symlink d1a/d2a/l63 0 2024-04-24T08:24:52.305 INFO:tasks.workunit.client.0.smithi090.stdout:2/261: stat d0/d7/d24/c30 0 2024-04-24T08:24:52.312 INFO:tasks.workunit.client.1.smithi149.stdout:5/687: write db/d12/d1e/d65/f7a [840641,9302] 0 2024-04-24T08:24:52.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/688: chown db/d12/d1e/d85/d92/da6/fce 4499 1 2024-04-24T08:24:52.314 INFO:tasks.workunit.client.0.smithi090.stdout:7/307: rmdir db/d17/d39 39 2024-04-24T08:24:52.316 INFO:tasks.workunit.client.0.smithi090.stdout:9/239: mkdir d0/dc/d24/d2d/d49 0 2024-04-24T08:24:52.317 INFO:tasks.workunit.client.1.smithi149.stdout:6/465: stat d0/d1a/l57 0 2024-04-24T08:24:52.321 INFO:tasks.workunit.client.1.smithi149.stdout:1/707: creat d0/ddf/feb x:0 0 0 2024-04-24T08:24:52.323 INFO:tasks.workunit.client.1.smithi149.stdout:4/543: symlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/lab 0 2024-04-24T08:24:52.326 INFO:tasks.workunit.client.0.smithi090.stdout:0/301: rmdir d1a/d4b 39 2024-04-24T08:24:52.327 INFO:tasks.workunit.client.0.smithi090.stdout:2/262: stat d0/lc 0 2024-04-24T08:24:52.328 INFO:tasks.workunit.client.1.smithi149.stdout:9/625: fdatasync d1/d2/d34/f40 0 2024-04-24T08:24:52.330 INFO:tasks.workunit.client.1.smithi149.stdout:8/512: write d1/f4f [658061,75029] 0 2024-04-24T08:24:52.332 INFO:tasks.workunit.client.0.smithi090.stdout:7/308: rmdir db/d17/d29/d38/d3e 39 2024-04-24T08:24:52.335 INFO:tasks.workunit.client.1.smithi149.stdout:1/708: mkdir d0/ddf/da0/db8/dec 0 2024-04-24T08:24:52.337 INFO:tasks.workunit.client.0.smithi090.stdout:9/240: mknod d0/dc/d24/d2d/c4a 0 2024-04-24T08:24:52.337 INFO:tasks.workunit.client.1.smithi149.stdout:1/709: chown d0/d5/db/d18/d2f/c67 238509 1 2024-04-24T08:24:52.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/221: dread d2/f3 [0,4194304] 0 2024-04-24T08:24:52.340 INFO:tasks.workunit.client.1.smithi149.stdout:0/542: dwrite d2/d9/df/f59 [4194304,4194304] 0 2024-04-24T08:24:52.341 INFO:tasks.workunit.client.1.smithi149.stdout:0/543: chown d2/d9/d1a 279319 1 2024-04-24T08:24:52.342 INFO:tasks.workunit.client.0.smithi090.stdout:0/302: chown d1a/l2d 27086890 1 2024-04-24T08:24:52.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/222: dread - d2/da/d13/d15/d24/f46 zero size 2024-04-24T08:24:52.345 INFO:tasks.workunit.client.0.smithi090.stdout:8/223: chown d2/da/d13/d15/f3f 10385 1 2024-04-24T08:24:52.345 INFO:tasks.workunit.client.0.smithi090.stdout:4/180: link d3/df/c14 d3/d18/c2d 0 2024-04-24T08:24:52.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/309: creat db/d17/d29/d32/d5a/d75/f77 x:0 0 0 2024-04-24T08:24:52.347 INFO:tasks.workunit.client.0.smithi090.stdout:6/267: dwrite d8/fa [8388608,4194304] 0 2024-04-24T08:24:52.358 INFO:tasks.workunit.client.0.smithi090.stdout:8/224: write d2/f2c [1184282,15572] 0 2024-04-24T08:24:52.361 INFO:tasks.workunit.client.0.smithi090.stdout:3/234: getdents d2/df/d21 0 2024-04-24T08:24:52.362 INFO:tasks.workunit.client.0.smithi090.stdout:7/310: write db/d17/d29/d32/f57 [423599,33507] 0 2024-04-24T08:24:52.368 INFO:tasks.workunit.client.0.smithi090.stdout:0/303: write d1a/d4b/d53/d57/f59 [952245,109217] 0 2024-04-24T08:24:52.369 INFO:tasks.workunit.client.0.smithi090.stdout:5/324: dread d8/d59/f61 [0,4194304] 0 2024-04-24T08:24:52.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/263: link d0/l16 d0/d14/d17/d2a/d31/d46/l4f 0 2024-04-24T08:24:52.370 INFO:tasks.workunit.client.0.smithi090.stdout:9/241: write d0/fb [546577,38905] 0 2024-04-24T08:24:52.371 INFO:tasks.workunit.client.0.smithi090.stdout:8/225: fsync d2/f12 0 2024-04-24T08:24:52.372 INFO:tasks.workunit.client.0.smithi090.stdout:4/181: rename d3/d18/d1d/d22/l2c to d3/l2e 0 2024-04-24T08:24:52.374 INFO:tasks.workunit.client.0.smithi090.stdout:8/226: write d2/da/d13/f3a [861778,107444] 0 2024-04-24T08:24:52.375 INFO:tasks.workunit.client.0.smithi090.stdout:3/235: symlink d2/df/l4a 0 2024-04-24T08:24:52.377 INFO:tasks.workunit.client.0.smithi090.stdout:7/311: mkdir db/d54/d78 0 2024-04-24T08:24:52.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/227: write d2/da/d13/d15/f49 [56545,100458] 0 2024-04-24T08:24:52.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/264: mkdir d0/d14/d4d/d50 0 2024-04-24T08:24:52.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/544: mkdir d2/d10/d1a/dac 0 2024-04-24T08:24:52.384 INFO:tasks.workunit.client.0.smithi090.stdout:4/182: rename d3/df/f1b to d3/d25/f2f 0 2024-04-24T08:24:52.386 INFO:tasks.workunit.client.0.smithi090.stdout:3/236: rmdir d2/d25 39 2024-04-24T08:24:52.388 INFO:tasks.workunit.client.1.smithi149.stdout:2/526: rename d0/d6/db/l36 to d0/d6/db/d39/d3d/d44/d57/la2 0 2024-04-24T08:24:52.389 INFO:tasks.workunit.client.0.smithi090.stdout:9/242: dwrite d0/d19/f46 [0,4194304] 0 2024-04-24T08:24:52.391 INFO:tasks.workunit.client.1.smithi149.stdout:8/513: symlink d1/de/d18/d32/d65/d8b/lb0 0 2024-04-24T08:24:52.392 INFO:tasks.workunit.client.0.smithi090.stdout:8/228: chown d2/l45 3515 1 2024-04-24T08:24:52.392 INFO:tasks.workunit.client.0.smithi090.stdout:5/325: symlink d8/d59/l75 0 2024-04-24T08:24:52.396 INFO:tasks.workunit.client.1.smithi149.stdout:7/580: sync 2024-04-24T08:24:52.398 INFO:tasks.workunit.client.0.smithi090.stdout:6/268: truncate d8/fa 5476355 0 2024-04-24T08:24:52.400 INFO:tasks.workunit.client.1.smithi149.stdout:0/544: mknod d2/d9/d69/cab 0 2024-04-24T08:24:52.403 INFO:tasks.workunit.client.1.smithi149.stdout:0/545: stat d2/d9/d1a/d36 0 2024-04-24T08:24:52.403 INFO:tasks.workunit.client.0.smithi090.stdout:2/265: dwrite d0/d14/d17/f1b [0,4194304] 0 2024-04-24T08:24:52.406 INFO:tasks.workunit.client.0.smithi090.stdout:7/312: rename db/d17/d29/d38/f5e to db/d17/d29/d32/d5a/d75/f79 0 2024-04-24T08:24:52.407 INFO:tasks.workunit.client.1.smithi149.stdout:4/545: symlink d2/d18/d1e/d3c/lad 0 2024-04-24T08:24:52.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/546: chown d2/d9/d1a/d36/f74 175370 1 2024-04-24T08:24:52.408 INFO:tasks.workunit.client.0.smithi090.stdout:9/243: dwrite d0/d19/f1c [0,4194304] 0 2024-04-24T08:24:52.410 INFO:tasks.workunit.client.0.smithi090.stdout:3/237: mkdir d2/df/d1a/d37/d4b 0 2024-04-24T08:24:52.411 INFO:tasks.workunit.client.1.smithi149.stdout:2/527: rmdir d0/d6/db/d39/d3d 39 2024-04-24T08:24:52.414 INFO:tasks.workunit.client.0.smithi090.stdout:0/304: creat d1a/f64 x:0 0 0 2024-04-24T08:24:52.419 INFO:tasks.workunit.client.0.smithi090.stdout:0/305: dread ff [0,4194304] 0 2024-04-24T08:24:52.422 INFO:tasks.workunit.client.1.smithi149.stdout:6/466: truncate d0/d1a/d28/d31/d33/f90 1193411 0 2024-04-24T08:24:52.423 INFO:tasks.workunit.client.0.smithi090.stdout:6/269: dwrite d8/d1d/d2f/d30/d38/f4c [0,4194304] 0 2024-04-24T08:24:52.423 INFO:tasks.workunit.client.0.smithi090.stdout:7/313: dread db/f1a [0,4194304] 0 2024-04-24T08:24:52.424 INFO:tasks.workunit.client.1.smithi149.stdout:6/467: readlink d0/l40 0 2024-04-24T08:24:52.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/266: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:24:52.425 INFO:tasks.workunit.client.0.smithi090.stdout:6/270: chown d8/dc 0 1 2024-04-24T08:24:52.427 INFO:tasks.workunit.client.1.smithi149.stdout:4/546: dwrite d2/f6c [0,4194304] 0 2024-04-24T08:24:52.433 INFO:tasks.workunit.client.0.smithi090.stdout:8/229: creat d2/da/d13/d15/f4e x:0 0 0 2024-04-24T08:24:52.433 INFO:tasks.workunit.client.0.smithi090.stdout:6/271: chown d8/d1d/d29/f3b 12218 1 2024-04-24T08:24:52.433 INFO:tasks.workunit.client.1.smithi149.stdout:7/581: symlink d9/dc/d11/d38/lc3 0 2024-04-24T08:24:52.435 INFO:tasks.workunit.client.0.smithi090.stdout:6/272: fdatasync d8/d1d/d2f/d30/f4b 0 2024-04-24T08:24:52.439 INFO:tasks.workunit.client.1.smithi149.stdout:5/689: getdents db/d12/d1e 0 2024-04-24T08:24:52.440 INFO:tasks.workunit.client.1.smithi149.stdout:0/547: creat d2/d9/df/d40/d4a/d4e/d6f/d9e/fac x:0 0 0 2024-04-24T08:24:52.442 INFO:tasks.workunit.client.1.smithi149.stdout:5/690: dread - db/d12/d1e/d65/fe1 zero size 2024-04-24T08:24:52.449 INFO:tasks.workunit.client.1.smithi149.stdout:1/710: link d0/ddf/d53/d61/f70 d0/d5/db/d18/fed 0 2024-04-24T08:24:52.450 INFO:tasks.workunit.client.0.smithi090.stdout:3/238: creat d2/df/d1a/f4c x:0 0 0 2024-04-24T08:24:52.452 INFO:tasks.workunit.client.1.smithi149.stdout:1/711: truncate d0/ddd/fe9 150220 0 2024-04-24T08:24:52.454 INFO:tasks.workunit.client.1.smithi149.stdout:9/626: rename d1/da/dd/c75 to d1/cdc 0 2024-04-24T08:24:52.457 INFO:tasks.workunit.client.1.smithi149.stdout:9/627: fsync d1/da/dd/d79/fa1 0 2024-04-24T08:24:52.457 INFO:tasks.workunit.client.1.smithi149.stdout:0/548: rmdir d2/d9 39 2024-04-24T08:24:52.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/528: symlink d0/d6/db/d27/d81/d9f/la3 0 2024-04-24T08:24:52.464 INFO:tasks.workunit.client.0.smithi090.stdout:5/326: dread d8/da/fc [0,4194304] 0 2024-04-24T08:24:52.466 INFO:tasks.workunit.client.1.smithi149.stdout:5/691: symlink db/d12/d1e/d85/d92/lec 0 2024-04-24T08:24:52.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/327: dread - d8/da/d3a/d36/d50/f6c zero size 2024-04-24T08:24:52.469 INFO:tasks.workunit.client.1.smithi149.stdout:0/549: dread d2/d3c/f61 [0,4194304] 0 2024-04-24T08:24:52.470 INFO:tasks.workunit.client.0.smithi090.stdout:7/314: truncate db/d17/d29/d32/f61 3546779 0 2024-04-24T08:24:52.473 INFO:tasks.workunit.client.1.smithi149.stdout:6/468: rename d0/d1a/d28/d4a/f7f to d0/d1a/d28/d31/d4c/d8a/f9d 0 2024-04-24T08:24:52.476 INFO:tasks.workunit.client.1.smithi149.stdout:1/712: chown d0/d1b/d5d/de1/l9f 17902630 1 2024-04-24T08:24:52.476 INFO:tasks.workunit.client.1.smithi149.stdout:6/469: stat d0/d1a/d28/d31/c5b 0 2024-04-24T08:24:52.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/273: dread d8/f1b [0,4194304] 0 2024-04-24T08:24:52.477 INFO:tasks.workunit.client.0.smithi090.stdout:2/267: rename d0/d7/d24/c26 to d0/d14/d36/c51 0 2024-04-24T08:24:52.478 INFO:tasks.workunit.client.1.smithi149.stdout:8/514: truncate d1/d1f/d31/f84 512622 0 2024-04-24T08:24:52.479 INFO:tasks.workunit.client.1.smithi149.stdout:1/713: dread - d0/da/fe4 zero size 2024-04-24T08:24:52.479 INFO:tasks.workunit.client.1.smithi149.stdout:9/628: truncate d1/d2/d15/d1c/d42/f65 161129 0 2024-04-24T08:24:52.481 INFO:tasks.workunit.client.0.smithi090.stdout:6/274: fsync d8/d1d/d29/f2a 0 2024-04-24T08:24:52.481 INFO:tasks.workunit.client.0.smithi090.stdout:0/306: dwrite d1a/f24 [0,4194304] 0 2024-04-24T08:24:52.486 INFO:tasks.workunit.client.1.smithi149.stdout:9/629: write d1/d2/d15/fbc [254447,123420] 0 2024-04-24T08:24:52.486 INFO:tasks.workunit.client.1.smithi149.stdout:2/529: chown d0/d6/db/d39/d3d/f51 61775898 1 2024-04-24T08:24:52.487 INFO:tasks.workunit.client.0.smithi090.stdout:3/239: symlink d2/d15/l4d 0 2024-04-24T08:24:52.489 INFO:tasks.workunit.client.1.smithi149.stdout:0/550: readlink d2/d9/d1a/d36/l37 0 2024-04-24T08:24:52.490 INFO:tasks.workunit.client.1.smithi149.stdout:2/530: stat d0/d6/db/d39/l86 0 2024-04-24T08:24:52.492 INFO:tasks.workunit.client.1.smithi149.stdout:9/630: read - d1/d4c/d91/fb9 zero size 2024-04-24T08:24:52.492 INFO:tasks.workunit.client.1.smithi149.stdout:5/692: dwrite db/d12/d2f/d74/dc6/fe9 [0,4194304] 0 2024-04-24T08:24:52.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/328: mkdir d8/d76 0 2024-04-24T08:24:52.493 INFO:tasks.workunit.client.0.smithi090.stdout:5/329: stat d8/da/d3a/d2d/l30 0 2024-04-24T08:24:52.494 INFO:tasks.workunit.client.1.smithi149.stdout:1/714: truncate d0/d5/db/d18/d2f/fd1 516416 0 2024-04-24T08:24:52.501 INFO:tasks.workunit.client.1.smithi149.stdout:7/582: rename d9/dc/d2e/c5d to d9/dc/d11/d84/da7/cc4 0 2024-04-24T08:24:52.506 INFO:tasks.workunit.client.1.smithi149.stdout:6/470: truncate d0/f10 2302877 0 2024-04-24T08:24:52.513 INFO:tasks.workunit.client.0.smithi090.stdout:0/307: unlink f12 0 2024-04-24T08:24:52.514 INFO:tasks.workunit.client.0.smithi090.stdout:9/244: write d0/d19/f1d [258679,18279] 0 2024-04-24T08:24:52.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/183: getdents d3/d18/d1d 0 2024-04-24T08:24:52.515 INFO:tasks.workunit.client.1.smithi149.stdout:7/583: dread d9/dc/d11/d38/f56 [0,4194304] 0 2024-04-24T08:24:52.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/308: chown d1a/d4b/d53/d57/f59 6269672 1 2024-04-24T08:24:52.516 INFO:tasks.workunit.client.1.smithi149.stdout:9/631: creat d1/da/d4a/fdd x:0 0 0 2024-04-24T08:24:52.518 INFO:tasks.workunit.client.0.smithi090.stdout:4/184: read d3/d18/d1d/f1f [118274,66196] 0 2024-04-24T08:24:52.519 INFO:tasks.workunit.client.1.smithi149.stdout:0/551: creat d2/d9/df/d40/d4a/d4e/d6f/d9e/fad x:0 0 0 2024-04-24T08:24:52.520 INFO:tasks.workunit.client.0.smithi090.stdout:0/309: write d1a/d4b/d53/d57/f59 [1700873,13921] 0 2024-04-24T08:24:52.522 INFO:tasks.workunit.client.0.smithi090.stdout:7/315: rmdir db/d17/d39 39 2024-04-24T08:24:52.524 INFO:tasks.workunit.client.1.smithi149.stdout:7/584: dread d9/dc/fb7 [0,4194304] 0 2024-04-24T08:24:52.526 INFO:tasks.workunit.client.0.smithi090.stdout:6/275: dwrite d8/f16 [0,4194304] 0 2024-04-24T08:24:52.529 INFO:tasks.workunit.client.1.smithi149.stdout:6/471: fdatasync d0/d1a/f2e 0 2024-04-24T08:24:52.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/245: fsync d0/d19/f1e 0 2024-04-24T08:24:52.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/240: dread d2/df/f10 [0,4194304] 0 2024-04-24T08:24:52.530 INFO:tasks.workunit.client.0.smithi090.stdout:9/246: dread - d0/dc/f3e zero size 2024-04-24T08:24:52.537 INFO:tasks.workunit.client.1.smithi149.stdout:8/515: mknod d1/de/d18/d32/d65/cb1 0 2024-04-24T08:24:52.543 INFO:tasks.workunit.client.0.smithi090.stdout:0/310: truncate d1a/f2c 4591219 0 2024-04-24T08:24:52.543 INFO:tasks.workunit.client.0.smithi090.stdout:7/316: unlink db/d17/l4f 0 2024-04-24T08:24:52.545 INFO:tasks.workunit.client.1.smithi149.stdout:0/552: rename d2/c90 to d2/d9/d1a/d36/cae 0 2024-04-24T08:24:52.546 INFO:tasks.workunit.client.0.smithi090.stdout:0/311: write d1a/d4b/f4d [730739,87860] 0 2024-04-24T08:24:52.547 INFO:tasks.workunit.client.1.smithi149.stdout:5/693: dread db/d12/d2f/f62 [0,4194304] 0 2024-04-24T08:24:52.547 INFO:tasks.workunit.client.1.smithi149.stdout:1/715: dwrite d0/d1b/d5d/f9a [4194304,4194304] 0 2024-04-24T08:24:52.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/230: getdents d2/da/d13/d34 0 2024-04-24T08:24:52.548 INFO:tasks.workunit.client.0.smithi090.stdout:6/276: fsync d8/d1d/d29/f2c 0 2024-04-24T08:24:52.549 INFO:tasks.workunit.client.0.smithi090.stdout:9/247: dwrite d0/dc/f29 [0,4194304] 0 2024-04-24T08:24:52.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/317: dread - db/d17/d29/d32/d5a/d75/f77 zero size 2024-04-24T08:24:52.550 INFO:tasks.workunit.client.1.smithi149.stdout:1/716: chown d0/d1b/d4b/da1/cb3 0 1 2024-04-24T08:24:52.552 INFO:tasks.workunit.client.0.smithi090.stdout:6/277: fdatasync d8/d1d/d29/f2a 0 2024-04-24T08:24:52.553 INFO:tasks.workunit.client.0.smithi090.stdout:5/330: write d8/da/d3a/d36/d50/f5e [1486310,5562] 0 2024-04-24T08:24:52.554 INFO:tasks.workunit.client.0.smithi090.stdout:9/248: dread - d0/dc/f3e zero size 2024-04-24T08:24:52.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/278: fsync d8/f15 0 2024-04-24T08:24:52.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/249: fsync d0/d2a/f43 0 2024-04-24T08:24:52.557 INFO:tasks.workunit.client.0.smithi090.stdout:7/318: read db/d17/d29/f30 [468582,78933] 0 2024-04-24T08:24:52.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/250: write d0/d19/f1d [1652084,49857] 0 2024-04-24T08:24:52.560 INFO:tasks.workunit.client.1.smithi149.stdout:6/472: creat d0/d1a/d28/d31/d33/f9e x:0 0 0 2024-04-24T08:24:52.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/251: chown d0/d19/d33/f36 0 1 2024-04-24T08:24:52.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/319: truncate db/d17/d29/f30 973615 0 2024-04-24T08:24:52.563 INFO:tasks.workunit.client.1.smithi149.stdout:2/531: creat d0/d6/db/d27/d64/fa4 x:0 0 0 2024-04-24T08:24:52.564 INFO:tasks.workunit.client.0.smithi090.stdout:9/252: chown d0/d19/d20/l44 25136 1 2024-04-24T08:24:52.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/516: dread - d1/de/d18/f6f zero size 2024-04-24T08:24:52.565 INFO:tasks.workunit.client.1.smithi149.stdout:9/632: creat d1/d2/d15/d1c/d42/dae/dd5/fde x:0 0 0 2024-04-24T08:24:52.568 INFO:tasks.workunit.client.1.smithi149.stdout:0/553: chown d2/d9/df/d56/f5f 549232 1 2024-04-24T08:24:52.568 INFO:tasks.workunit.client.0.smithi090.stdout:7/320: stat db/d17/d29/d32/f57 0 2024-04-24T08:24:52.568 INFO:tasks.workunit.client.0.smithi090.stdout:4/185: creat d3/df/f30 x:0 0 0 2024-04-24T08:24:52.569 INFO:tasks.workunit.client.0.smithi090.stdout:3/241: truncate d2/df/f19 3877449 0 2024-04-24T08:24:52.570 INFO:tasks.workunit.client.0.smithi090.stdout:9/253: write d0/d19/f28 [2423773,57748] 0 2024-04-24T08:24:52.570 INFO:tasks.workunit.client.1.smithi149.stdout:1/717: unlink d0/d1b/d5d/c7e 0 2024-04-24T08:24:52.573 INFO:tasks.workunit.client.0.smithi090.stdout:5/331: mknod d8/da/d3a/d36/d50/c77 0 2024-04-24T08:24:52.574 INFO:tasks.workunit.client.0.smithi090.stdout:9/254: write d0/d2a/f43 [788459,42487] 0 2024-04-24T08:24:52.575 INFO:tasks.workunit.client.1.smithi149.stdout:6/473: read d0/f17 [1348672,18753] 0 2024-04-24T08:24:52.577 INFO:tasks.workunit.client.1.smithi149.stdout:9/633: creat d1/d99/fdf x:0 0 0 2024-04-24T08:24:52.577 INFO:tasks.workunit.client.1.smithi149.stdout:6/474: readlink d0/l55 0 2024-04-24T08:24:52.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/255: write d0/dc/f2c [107766,120703] 0 2024-04-24T08:24:52.579 INFO:tasks.workunit.client.0.smithi090.stdout:7/321: unlink db/d17/d29/d38/f72 0 2024-04-24T08:24:52.581 INFO:tasks.workunit.client.0.smithi090.stdout:9/256: readlink d0/dc/l48 0 2024-04-24T08:24:52.582 INFO:tasks.workunit.client.0.smithi090.stdout:0/312: rename d1a/d2a/l63 to d1a/d4b/d53/l65 0 2024-04-24T08:24:52.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/554: mknod d2/d3c/caf 0 2024-04-24T08:24:52.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/718: creat d0/ddd/fee x:0 0 0 2024-04-24T08:24:52.590 INFO:tasks.workunit.client.1.smithi149.stdout:9/634: rename d1/d2/d15/l6e to d1/d2/d33/d60/le0 0 2024-04-24T08:24:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:1/719: read d0/d5/d4f/d60/f78 [481498,54428] 0 2024-04-24T08:24:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:2/532: dwrite d0/d6/f16 [0,4194304] 0 2024-04-24T08:24:52.593 INFO:tasks.workunit.client.0.smithi090.stdout:8/231: dread d2/da/d13/f28 [0,4194304] 0 2024-04-24T08:24:52.594 INFO:tasks.workunit.client.1.smithi149.stdout:1/720: truncate d0/ddf/da0/fe0 31130 0 2024-04-24T08:24:52.595 INFO:tasks.workunit.client.0.smithi090.stdout:7/322: dwrite db/d17/d29/d32/f57 [0,4194304] 0 2024-04-24T08:24:52.595 INFO:tasks.workunit.client.1.smithi149.stdout:3/657: sync 2024-04-24T08:24:52.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/323: dread - db/d17/d29/d32/d5a/d75/f68 zero size 2024-04-24T08:24:52.599 INFO:tasks.workunit.client.0.smithi090.stdout:6/279: rename d8/d1d/d29/f2a to d8/d1d/d2f/d30/f52 0 2024-04-24T08:24:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:7/585: truncate d9/d43/d4f/d66/f6e 3823653 0 2024-04-24T08:24:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:6/475: truncate d0/d1a/d28/f2c 393155 0 2024-04-24T08:24:52.600 INFO:tasks.workunit.client.0.smithi090.stdout:7/324: stat db/f15 0 2024-04-24T08:24:52.601 INFO:tasks.workunit.client.1.smithi149.stdout:0/555: mkdir d2/d9/df/d2f/d6c/d72/db0 0 2024-04-24T08:24:52.616 INFO:tasks.workunit.client.0.smithi090.stdout:0/313: symlink d1a/d2a/d35/d62/l66 0 2024-04-24T08:24:52.618 INFO:tasks.workunit.client.1.smithi149.stdout:5/694: getdents db/d12/d60/db2 0 2024-04-24T08:24:52.619 INFO:tasks.workunit.client.0.smithi090.stdout:3/242: truncate d2/f24 2425481 0 2024-04-24T08:24:52.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/476: dwrite d0/d1a/d28/d4a/f77 [0,4194304] 0 2024-04-24T08:24:52.621 INFO:tasks.workunit.client.0.smithi090.stdout:4/186: dread d3/d25/f2f [0,4194304] 0 2024-04-24T08:24:52.621 INFO:tasks.workunit.client.0.smithi090.stdout:5/332: rename d8/da/d14/d2f/l58 to d8/da/d18/d21/d4c/l78 0 2024-04-24T08:24:52.621 INFO:tasks.workunit.client.1.smithi149.stdout:5/695: chown db/d12/d60/db2/d83/dc8/cc7 927670 1 2024-04-24T08:24:52.622 INFO:tasks.workunit.client.0.smithi090.stdout:5/333: dread - d8/da/d14/f69 zero size 2024-04-24T08:24:52.623 INFO:tasks.workunit.client.1.smithi149.stdout:3/658: unlink d0/d5/d1a/d70/da3/fb0 0 2024-04-24T08:24:52.625 INFO:tasks.workunit.client.0.smithi090.stdout:5/334: dread d8/d25/f3d [0,4194304] 0 2024-04-24T08:24:52.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/696: write db/d12/d1e/d85/d92/da6/fce [949610,36981] 0 2024-04-24T08:24:52.630 INFO:tasks.workunit.client.0.smithi090.stdout:3/243: symlink d2/d15/l4e 0 2024-04-24T08:24:52.632 INFO:tasks.workunit.client.1.smithi149.stdout:0/556: rename d2/d9/df/f1b to d2/d9/df/d2f/d6c/d72/fb1 0 2024-04-24T08:24:52.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/517: link d1/d1f/d25/l4d d1/de/lb2 0 2024-04-24T08:24:52.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/280: rename d8/d1d/d2f/d30/d38/f4c to d8/dc/f53 0 2024-04-24T08:24:52.634 INFO:tasks.workunit.client.1.smithi149.stdout:2/533: creat d0/d6/db/d39/d3d/d58/d98/fa5 x:0 0 0 2024-04-24T08:24:52.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/635: mkdir d1/da/dd/d79/db6/de1 0 2024-04-24T08:24:52.636 INFO:tasks.workunit.client.0.smithi090.stdout:7/325: dread db/d17/d29/f30 [0,4194304] 0 2024-04-24T08:24:52.639 INFO:tasks.workunit.client.0.smithi090.stdout:9/257: getdents d0/d19/d20 0 2024-04-24T08:24:52.643 INFO:tasks.workunit.client.1.smithi149.stdout:0/557: fsync d2/d9/df/d2f/d6c/d72/f99 0 2024-04-24T08:24:52.647 INFO:tasks.workunit.client.1.smithi149.stdout:7/586: getdents d9/d43/d4f/da5 0 2024-04-24T08:24:52.647 INFO:tasks.workunit.client.0.smithi090.stdout:0/314: symlink d1a/d2a/l67 0 2024-04-24T08:24:52.649 INFO:tasks.workunit.client.0.smithi090.stdout:5/335: dwrite d8/da/d18/d21/f41 [0,4194304] 0 2024-04-24T08:24:52.650 INFO:tasks.workunit.client.1.smithi149.stdout:3/659: dwrite d0/d5/da/d59/fcf [0,4194304] 0 2024-04-24T08:24:52.651 INFO:tasks.workunit.client.1.smithi149.stdout:6/477: creat d0/d1a/d28/d31/d33/f9f x:0 0 0 2024-04-24T08:24:52.652 INFO:tasks.workunit.client.1.smithi149.stdout:7/587: chown d9/dc/de/d44/lb3 17 1 2024-04-24T08:24:52.655 INFO:tasks.workunit.client.1.smithi149.stdout:7/588: readlink d9/dc/d11/d26/l36 0 2024-04-24T08:24:52.656 INFO:tasks.workunit.client.0.smithi090.stdout:1/277: sync 2024-04-24T08:24:52.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/268: sync 2024-04-24T08:24:52.657 INFO:tasks.workunit.client.1.smithi149.stdout:5/697: dwrite db/d12/d1e/fc1 [0,4194304] 0 2024-04-24T08:24:52.657 INFO:tasks.workunit.client.1.smithi149.stdout:6/478: fsync d0/d1a/d28/d31/d4c/d4e/d62/d66/f7b 0 2024-04-24T08:24:52.660 INFO:tasks.workunit.client.0.smithi090.stdout:3/244: rmdir d2/df/d21 39 2024-04-24T08:24:52.661 INFO:tasks.workunit.client.1.smithi149.stdout:7/589: chown d9/dc/d11/d26/l36 2499549 1 2024-04-24T08:24:52.668 INFO:tasks.workunit.client.1.smithi149.stdout:1/721: creat d0/d1b/fef x:0 0 0 2024-04-24T08:24:52.674 INFO:tasks.workunit.client.0.smithi090.stdout:7/326: truncate db/f11 70201 0 2024-04-24T08:24:52.676 INFO:tasks.workunit.client.0.smithi090.stdout:8/232: rmdir d2/d48 0 2024-04-24T08:24:52.679 INFO:tasks.workunit.client.0.smithi090.stdout:6/281: dread f6 [0,4194304] 0 2024-04-24T08:24:52.679 INFO:tasks.workunit.client.1.smithi149.stdout:6/479: creat d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 x:0 0 0 2024-04-24T08:24:52.680 INFO:tasks.workunit.client.1.smithi149.stdout:3/660: dread d0/d5/da/f6e [0,4194304] 0 2024-04-24T08:24:52.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/590: dread d9/dc/d11/d83/da6/d67/f9e [0,4194304] 0 2024-04-24T08:24:52.680 INFO:tasks.workunit.client.0.smithi090.stdout:9/258: rename d0/d19/f45 to d0/dc/d24/f4b 0 2024-04-24T08:24:52.681 INFO:tasks.workunit.client.0.smithi090.stdout:4/187: dwrite d3/df/f1e [0,4194304] 0 2024-04-24T08:24:52.683 INFO:tasks.workunit.client.0.smithi090.stdout:4/188: write d3/f13 [65388,32722] 0 2024-04-24T08:24:52.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/698: rename db/d12/d2f/f46 to db/d1f/d7f/fed 0 2024-04-24T08:24:52.685 INFO:tasks.workunit.client.1.smithi149.stdout:5/699: chown db/da4/fa8 222052 1 2024-04-24T08:24:52.690 INFO:tasks.workunit.client.0.smithi090.stdout:0/315: mkdir d1a/d2a/d35/d68 0 2024-04-24T08:24:52.694 INFO:tasks.workunit.client.1.smithi149.stdout:2/534: dread d0/d6/db/d27/d64/f74 [0,4194304] 0 2024-04-24T08:24:52.697 INFO:tasks.workunit.client.0.smithi090.stdout:9/259: dwrite d0/d19/f25 [0,4194304] 0 2024-04-24T08:24:52.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/278: creat d3/d5/d3e/f63 x:0 0 0 2024-04-24T08:24:52.703 INFO:tasks.workunit.client.0.smithi090.stdout:2/269: creat d0/d14/d17/d2a/d31/f52 x:0 0 0 2024-04-24T08:24:52.708 INFO:tasks.workunit.client.0.smithi090.stdout:3/245: write d2/d25/d33/f44 [5765,71487] 0 2024-04-24T08:24:52.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/480: rmdir d0/d1a/d28/d31/d33 39 2024-04-24T08:24:52.711 INFO:tasks.workunit.client.0.smithi090.stdout:5/336: write d8/f10 [935079,61908] 0 2024-04-24T08:24:52.712 INFO:tasks.workunit.client.1.smithi149.stdout:6/481: chown d0/f11 642915 1 2024-04-24T08:24:52.714 INFO:tasks.workunit.client.1.smithi149.stdout:6/482: stat d0/d1a/d28/d31 0 2024-04-24T08:24:52.715 INFO:tasks.workunit.client.1.smithi149.stdout:3/661: mknod d0/d5/d53/d6d/ce3 0 2024-04-24T08:24:52.717 INFO:tasks.workunit.client.1.smithi149.stdout:6/483: chown d0/d1a/d28/d31/d4c/d4e 833649 1 2024-04-24T08:24:52.719 INFO:tasks.workunit.client.1.smithi149.stdout:1/722: write d0/ddf/da0/f5e [80409,110597] 0 2024-04-24T08:24:52.721 INFO:tasks.workunit.client.1.smithi149.stdout:7/591: creat d9/d43/d64/fc5 x:0 0 0 2024-04-24T08:24:52.722 INFO:tasks.workunit.client.1.smithi149.stdout:4/547: sync 2024-04-24T08:24:52.722 INFO:tasks.workunit.client.1.smithi149.stdout:0/558: truncate d2/d9/df/d40/d4a/f64 2408778 0 2024-04-24T08:24:52.724 INFO:tasks.workunit.client.1.smithi149.stdout:5/700: rename db/d12/d1e/d85/la9 to db/d12/d1e/lee 0 2024-04-24T08:24:52.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/282: creat d8/d1d/d2f/d30/f54 x:0 0 0 2024-04-24T08:24:52.726 INFO:tasks.workunit.client.1.smithi149.stdout:6/484: write d0/d1a/d28/d31/d4c/d4e/d62/d7a/f92 [2762179,125397] 0 2024-04-24T08:24:52.727 INFO:tasks.workunit.client.1.smithi149.stdout:4/548: fsync d2/d10/d1a/da2/fa3 0 2024-04-24T08:24:52.730 INFO:tasks.workunit.client.0.smithi090.stdout:4/189: chown d3/d18/c2d 113543275 1 2024-04-24T08:24:52.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/260: unlink d0/c16 0 2024-04-24T08:24:52.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/261: dread - d0/f1b zero size 2024-04-24T08:24:52.735 INFO:tasks.workunit.client.0.smithi090.stdout:1/279: creat d3/d5/da/f64 x:0 0 0 2024-04-24T08:24:52.739 INFO:tasks.workunit.client.1.smithi149.stdout:7/592: rename d9/dc/de/d44/f78 to d9/dc/d11/d83/da6/d67/d69/dad/fc6 0 2024-04-24T08:24:52.740 INFO:tasks.workunit.client.1.smithi149.stdout:9/636: getdents d1/d2/d15/d1c/d42 0 2024-04-24T08:24:52.741 INFO:tasks.workunit.client.0.smithi090.stdout:5/337: dread d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:24:52.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/233: dwrite d2/da/d13/d15/f3f [0,4194304] 0 2024-04-24T08:24:52.742 INFO:tasks.workunit.client.1.smithi149.stdout:6/485: read - d0/d1a/d28/d31/d33/f9e zero size 2024-04-24T08:24:52.745 INFO:tasks.workunit.client.1.smithi149.stdout:1/723: write d0/ddf/d76/fae [101647,109415] 0 2024-04-24T08:24:52.747 INFO:tasks.workunit.client.1.smithi149.stdout:1/724: chown d0/d1b/fe7 43141243 1 2024-04-24T08:24:52.749 INFO:tasks.workunit.client.1.smithi149.stdout:3/662: symlink d0/d5/d53/dd6/le4 0 2024-04-24T08:24:52.752 INFO:tasks.workunit.client.0.smithi090.stdout:9/262: write d0/f18 [4360279,118157] 0 2024-04-24T08:24:52.752 INFO:tasks.workunit.client.1.smithi149.stdout:5/701: unlink db/d12/d60/lbd 0 2024-04-24T08:24:52.754 INFO:tasks.workunit.client.1.smithi149.stdout:5/702: chown db/d12/d60/db2/d83/dc5 2111977 1 2024-04-24T08:24:52.756 INFO:tasks.workunit.client.0.smithi090.stdout:7/327: creat db/d17/f7a x:0 0 0 2024-04-24T08:24:52.757 INFO:tasks.workunit.client.1.smithi149.stdout:4/549: write d2/d18/d1e/d3c/d48/f87 [321124,83090] 0 2024-04-24T08:24:52.759 INFO:tasks.workunit.client.0.smithi090.stdout:6/283: getdents d8/d1d/d1f/d3e/d4f 0 2024-04-24T08:24:52.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/280: dwrite d3/d5/da/dc/f40 [0,4194304] 0 2024-04-24T08:24:52.760 INFO:tasks.workunit.client.0.smithi090.stdout:0/316: truncate d1a/f1b 528506 0 2024-04-24T08:24:52.763 INFO:tasks.workunit.client.0.smithi090.stdout:8/234: write d2/da/d13/f4b [1288530,101224] 0 2024-04-24T08:24:52.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/284: dread - d8/f3f zero size 2024-04-24T08:24:52.766 INFO:tasks.workunit.client.0.smithi090.stdout:2/270: creat d0/d14/f53 x:0 0 0 2024-04-24T08:24:52.767 INFO:tasks.workunit.client.1.smithi149.stdout:6/486: dwrite d0/d1a/d28/d31/d4c/d80/f68 [0,4194304] 0 2024-04-24T08:24:52.769 INFO:tasks.workunit.client.0.smithi090.stdout:3/246: dwrite d2/df/d21/f27 [0,4194304] 0 2024-04-24T08:24:52.770 INFO:tasks.workunit.client.0.smithi090.stdout:6/285: write d8/d1d/d2f/d30/f4b [568544,125869] 0 2024-04-24T08:24:52.773 INFO:tasks.workunit.client.1.smithi149.stdout:6/487: chown d0/d1a/d28/d31/d4c/d4e/c53 20 1 2024-04-24T08:24:52.777 INFO:tasks.workunit.client.1.smithi149.stdout:2/535: creat d0/d6/db/d39/fa6 x:0 0 0 2024-04-24T08:24:52.778 INFO:tasks.workunit.client.1.smithi149.stdout:1/725: symlink d0/d5/d4f/d60/lf0 0 2024-04-24T08:24:52.782 INFO:tasks.workunit.client.0.smithi090.stdout:5/338: link d8/da/d3a/f43 d8/da/d3a/d36/d50/f79 0 2024-04-24T08:24:52.784 INFO:tasks.workunit.client.0.smithi090.stdout:5/339: chown d8/da/d3a/d36/d50/f68 2 1 2024-04-24T08:24:52.784 INFO:tasks.workunit.client.1.smithi149.stdout:5/703: dwrite db/d12/d60/db2/f6d [0,4194304] 0 2024-04-24T08:24:52.785 INFO:tasks.workunit.client.0.smithi090.stdout:5/340: fsync d8/f31 0 2024-04-24T08:24:52.787 INFO:tasks.workunit.client.1.smithi149.stdout:4/550: dwrite d2/d24/d2e/d38/f9c [0,4194304] 0 2024-04-24T08:24:52.787 INFO:tasks.workunit.client.1.smithi149.stdout:7/593: mknod d9/dc/d11/d83/da6/d67/cc7 0 2024-04-24T08:24:52.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/190: rename d3/df/l17 to d3/df/l31 0 2024-04-24T08:24:52.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/263: dread d0/f18 [0,4194304] 0 2024-04-24T08:24:52.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/281: mkdir d3/d5/da/dc/d65 0 2024-04-24T08:24:52.808 INFO:tasks.workunit.client.1.smithi149.stdout:6/488: mknod d0/d1a/d7e/ca1 0 2024-04-24T08:24:52.810 INFO:tasks.workunit.client.0.smithi090.stdout:1/282: write d3/d1d/d34/d5a/f5d [1639087,69957] 0 2024-04-24T08:24:52.811 INFO:tasks.workunit.client.1.smithi149.stdout:3/663: write d0/d5/da/d5e/f30 [1180737,47382] 0 2024-04-24T08:24:52.812 INFO:tasks.workunit.client.1.smithi149.stdout:6/489: chown d0/d1a/d28/d31/d38/d54/d87 30045894 1 2024-04-24T08:24:52.812 INFO:tasks.workunit.client.1.smithi149.stdout:4/551: dread d2/d10/d1a/f36 [4194304,4194304] 0 2024-04-24T08:24:52.813 INFO:tasks.workunit.client.1.smithi149.stdout:6/490: stat d0/f26 0 2024-04-24T08:24:52.814 INFO:tasks.workunit.client.1.smithi149.stdout:5/704: truncate db/d12/f5b 1962360 0 2024-04-24T08:24:52.815 INFO:tasks.workunit.client.1.smithi149.stdout:5/705: fsync db/f1a 0 2024-04-24T08:24:52.816 INFO:tasks.workunit.client.1.smithi149.stdout:7/594: mkdir d9/d43/d4f/dc8 0 2024-04-24T08:24:52.817 INFO:tasks.workunit.client.1.smithi149.stdout:6/491: write d0/d1a/d28/d4a/f67 [1064266,116238] 0 2024-04-24T08:24:52.818 INFO:tasks.workunit.client.0.smithi090.stdout:0/317: mkdir d1a/d2a/d69 0 2024-04-24T08:24:52.821 INFO:tasks.workunit.client.1.smithi149.stdout:4/552: unlink d2/d18/d1e/d3b/f78 0 2024-04-24T08:24:52.821 INFO:tasks.workunit.client.0.smithi090.stdout:0/318: write d1a/f24 [1424628,52422] 0 2024-04-24T08:24:52.825 INFO:tasks.workunit.client.0.smithi090.stdout:0/319: read f18 [908583,87675] 0 2024-04-24T08:24:52.827 INFO:tasks.workunit.client.1.smithi149.stdout:3/664: rename d0/d5/c2f to d0/d26/d3c/db1/ce5 0 2024-04-24T08:24:52.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/286: read d8/d1d/d2f/d30/f4b [194570,116541] 0 2024-04-24T08:24:52.830 INFO:tasks.workunit.client.1.smithi149.stdout:3/665: dread - d0/d5/da/d19/d80/dac/fc5 zero size 2024-04-24T08:24:52.833 INFO:tasks.workunit.client.1.smithi149.stdout:7/595: mkdir d9/dc/d2e/dc9 0 2024-04-24T08:24:52.835 INFO:tasks.workunit.client.0.smithi090.stdout:2/271: symlink d0/d14/d17/d2a/d31/d46/l54 0 2024-04-24T08:24:52.838 INFO:tasks.workunit.client.0.smithi090.stdout:5/341: rename d8/da/d3a/d36 to d8/d56/d6d/d7a 0 2024-04-24T08:24:52.839 INFO:tasks.workunit.client.0.smithi090.stdout:1/283: rmdir d3/d25 39 2024-04-24T08:24:52.840 INFO:tasks.workunit.client.1.smithi149.stdout:6/492: creat d0/d1a/d28/d31/d4c/d4e/d62/d7a/fa2 x:0 0 0 2024-04-24T08:24:52.841 INFO:tasks.workunit.client.1.smithi149.stdout:6/493: fdatasync d0/d1a/f27 0 2024-04-24T08:24:52.842 INFO:tasks.workunit.client.1.smithi149.stdout:4/553: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae 0 2024-04-24T08:24:52.845 INFO:tasks.workunit.client.1.smithi149.stdout:3/666: creat d0/d5/d1a/d70/fe6 x:0 0 0 2024-04-24T08:24:52.846 INFO:tasks.workunit.client.1.smithi149.stdout:2/536: write d0/f8 [219513,107140] 0 2024-04-24T08:24:52.849 INFO:tasks.workunit.client.1.smithi149.stdout:2/537: readlink d0/d6/db/d39/l62 0 2024-04-24T08:24:52.850 INFO:tasks.workunit.client.1.smithi149.stdout:7/596: rename d9/dc/d11/d83/da6/d67/la3 to d9/d43/lca 0 2024-04-24T08:24:52.850 INFO:tasks.workunit.client.1.smithi149.stdout:9/637: dwrite d1/d2/dc8/fd0 [0,4194304] 0 2024-04-24T08:24:52.853 INFO:tasks.workunit.client.0.smithi090.stdout:2/272: dread d0/d14/f2b [0,4194304] 0 2024-04-24T08:24:52.855 INFO:tasks.workunit.client.1.smithi149.stdout:2/538: fdatasync d0/d6/db/d39/d3d/d58/d98/fa5 0 2024-04-24T08:24:52.856 INFO:tasks.workunit.client.0.smithi090.stdout:8/235: link d2/da/l31 d2/da/l4f 0 2024-04-24T08:24:52.859 INFO:tasks.workunit.client.1.smithi149.stdout:6/494: symlink d0/d1a/d28/d31/d4c/d76/la3 0 2024-04-24T08:24:52.862 INFO:tasks.workunit.client.0.smithi090.stdout:0/320: rmdir d1a/d2a 39 2024-04-24T08:24:52.862 INFO:tasks.workunit.client.1.smithi149.stdout:6/495: fdatasync d0/d1a/d28/d4a/f67 0 2024-04-24T08:24:52.869 INFO:tasks.workunit.client.1.smithi149.stdout:1/726: dread d0/d5/db/d18/f29 [0,4194304] 0 2024-04-24T08:24:52.870 INFO:tasks.workunit.client.1.smithi149.stdout:8/518: sync 2024-04-24T08:24:52.871 INFO:tasks.workunit.client.0.smithi090.stdout:6/287: mknod d8/d1d/d2f/d30/c55 0 2024-04-24T08:24:52.871 INFO:tasks.workunit.client.1.smithi149.stdout:3/667: mknod d0/d5/d53/d4c/ce7 0 2024-04-24T08:24:52.871 INFO:tasks.workunit.client.1.smithi149.stdout:5/706: write db/f20 [916078,13950] 0 2024-04-24T08:24:52.873 INFO:tasks.workunit.client.0.smithi090.stdout:9/264: symlink d0/dc/d24/d3d/l4c 0 2024-04-24T08:24:52.875 INFO:tasks.workunit.client.1.smithi149.stdout:8/519: write d1/de/d18/d32/d63/d9e/fa1 [612290,106285] 0 2024-04-24T08:24:52.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/288: chown d8/d1d/d1f/d3e/d44 284 1 2024-04-24T08:24:52.879 INFO:tasks.workunit.client.1.smithi149.stdout:7/597: truncate d9/dc/d11/d83/da6/f71 5123576 0 2024-04-24T08:24:52.879 INFO:tasks.workunit.client.0.smithi090.stdout:9/265: write d0/dc/f3e [111799,16431] 0 2024-04-24T08:24:52.880 INFO:tasks.workunit.client.0.smithi090.stdout:5/342: mkdir d8/d56/d6d/d7b 0 2024-04-24T08:24:52.881 INFO:tasks.workunit.client.1.smithi149.stdout:2/539: mkdir d0/d6/db/d39/d3d/d44/d75/da7 0 2024-04-24T08:24:52.883 INFO:tasks.workunit.client.0.smithi090.stdout:8/236: dwrite d2/da/d13/d15/f49 [0,4194304] 0 2024-04-24T08:24:52.884 INFO:tasks.workunit.client.1.smithi149.stdout:6/496: readlink d0/d1a/d28/d31/d38/l3b 0 2024-04-24T08:24:52.886 INFO:tasks.workunit.client.1.smithi149.stdout:6/497: readlink d0/d1a/d28/l98 0 2024-04-24T08:24:52.887 INFO:tasks.workunit.client.1.smithi149.stdout:1/727: mknod d0/ddf/da0/cf1 0 2024-04-24T08:24:52.888 INFO:tasks.workunit.client.0.smithi090.stdout:8/237: chown d2/da/d13/d15/d24/f46 57562 1 2024-04-24T08:24:52.889 INFO:tasks.workunit.client.1.smithi149.stdout:9/638: creat d1/d2/d15/d1c/dca/fe2 x:0 0 0 2024-04-24T08:24:52.891 INFO:tasks.workunit.client.1.smithi149.stdout:1/728: dread - d0/d5/db/d18/fb9 zero size 2024-04-24T08:24:52.891 INFO:tasks.workunit.client.0.smithi090.stdout:0/321: rmdir d1a/d4b/d53/d57 39 2024-04-24T08:24:52.893 INFO:tasks.workunit.client.1.smithi149.stdout:3/668: dwrite d0/d5/f6f [0,4194304] 0 2024-04-24T08:24:52.893 INFO:tasks.workunit.client.1.smithi149.stdout:6/498: write d0/d1a/d28/d31/d33/f9b [526107,3840] 0 2024-04-24T08:24:52.894 INFO:tasks.workunit.client.0.smithi090.stdout:4/191: truncate d3/d25/f2f 683014 0 2024-04-24T08:24:52.897 INFO:tasks.workunit.client.1.smithi149.stdout:6/499: dread - d0/d1a/d28/d31/d4c/d4e/f96 zero size 2024-04-24T08:24:52.902 INFO:tasks.workunit.client.0.smithi090.stdout:1/284: mknod d3/d25/c66 0 2024-04-24T08:24:52.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/247: getdents d2/d25 0 2024-04-24T08:24:52.906 INFO:tasks.workunit.client.1.smithi149.stdout:7/598: unlink d9/dc/d11/f1e 0 2024-04-24T08:24:52.906 INFO:tasks.workunit.client.0.smithi090.stdout:5/343: mkdir d8/d7c 0 2024-04-24T08:24:52.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/328: getdents db/d17/d29/d38/d3e 0 2024-04-24T08:24:52.908 INFO:tasks.workunit.client.0.smithi090.stdout:3/248: chown d2/df/d1a/f4c 1 1 2024-04-24T08:24:52.910 INFO:tasks.workunit.client.1.smithi149.stdout:7/599: read - d9/dc/d7b/f7d zero size 2024-04-24T08:24:52.911 INFO:tasks.workunit.client.1.smithi149.stdout:9/639: symlink d1/da/d13/d67/le3 0 2024-04-24T08:24:52.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/344: fsync d8/da/d18/d21/f48 0 2024-04-24T08:24:52.912 INFO:tasks.workunit.client.0.smithi090.stdout:9/266: rename d0/d19/d20 to d0/d19/d33/d4d 0 2024-04-24T08:24:52.917 INFO:tasks.workunit.client.1.smithi149.stdout:6/500: mknod d0/d1a/d28/d31/d38/d54/ca4 0 2024-04-24T08:24:52.917 INFO:tasks.workunit.client.1.smithi149.stdout:3/669: creat d0/d5/da/d5e/fe8 x:0 0 0 2024-04-24T08:24:52.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/238: mkdir d2/da/d13/d15/d50 0 2024-04-24T08:24:52.918 INFO:tasks.workunit.client.1.smithi149.stdout:9/640: truncate d1/d2/d15/f81 2079733 0 2024-04-24T08:24:52.918 INFO:tasks.workunit.client.1.smithi149.stdout:5/707: stat db/d12/d1e/d65/l6f 0 2024-04-24T08:24:52.918 INFO:tasks.workunit.client.0.smithi090.stdout:9/267: dread - d0/d2a/f38 zero size 2024-04-24T08:24:52.921 INFO:tasks.workunit.client.1.smithi149.stdout:9/641: chown d1/da/l87 1 1 2024-04-24T08:24:52.922 INFO:tasks.workunit.client.0.smithi090.stdout:8/239: fdatasync d2/da/d13/d15/d24/f21 0 2024-04-24T08:24:52.924 INFO:tasks.workunit.client.1.smithi149.stdout:0/559: sync 2024-04-24T08:24:52.926 INFO:tasks.workunit.client.1.smithi149.stdout:4/554: dwrite d2/fc [0,4194304] 0 2024-04-24T08:24:52.927 INFO:tasks.workunit.client.0.smithi090.stdout:4/192: dwrite d3/d24/f2b [0,4194304] 0 2024-04-24T08:24:52.929 INFO:tasks.workunit.client.0.smithi090.stdout:4/193: stat d3/c19 0 2024-04-24T08:24:52.929 INFO:tasks.workunit.client.1.smithi149.stdout:7/600: dread d9/dc/d11/f6c [0,4194304] 0 2024-04-24T08:24:52.930 INFO:tasks.workunit.client.0.smithi090.stdout:2/273: creat d0/f55 x:0 0 0 2024-04-24T08:24:52.932 INFO:tasks.workunit.client.0.smithi090.stdout:0/322: read d1a/d2a/d58/f5c [203514,122776] 0 2024-04-24T08:24:52.933 INFO:tasks.workunit.client.0.smithi090.stdout:2/274: chown d0/d14/l20 1151732 1 2024-04-24T08:24:52.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/520: rename d1/d1f/f52 to d1/de/d18/d32/fb3 0 2024-04-24T08:24:52.938 INFO:tasks.workunit.client.1.smithi149.stdout:8/521: chown d1/l36 1 1 2024-04-24T08:24:52.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/285: creat d3/d1d/d54/f67 x:0 0 0 2024-04-24T08:24:52.941 INFO:tasks.workunit.client.0.smithi090.stdout:5/345: dread d8/da/d14/f28 [0,4194304] 0 2024-04-24T08:24:52.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/346: dread - d8/da/d3a/f55 zero size 2024-04-24T08:24:52.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/240: dread d2/da/d13/d15/d24/f2f [0,4194304] 0 2024-04-24T08:24:52.946 INFO:tasks.workunit.client.0.smithi090.stdout:5/347: dread - d8/d56/d6d/d7a/d50/f71 zero size 2024-04-24T08:24:52.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/329: mknod db/d54/d78/c7b 0 2024-04-24T08:24:52.949 INFO:tasks.workunit.client.1.smithi149.stdout:2/540: rmdir d0/d6/db/d39/d3d/d44/d75/da7 0 2024-04-24T08:24:52.951 INFO:tasks.workunit.client.1.smithi149.stdout:3/670: readlink d0/d5/da/d5e/la4 0 2024-04-24T08:24:52.951 INFO:tasks.workunit.client.0.smithi090.stdout:6/289: link d8/d1d/d2f/l50 d8/d1d/d2f/d30/d38/l56 0 2024-04-24T08:24:52.954 INFO:tasks.workunit.client.1.smithi149.stdout:2/541: stat d0/d6/c89 0 2024-04-24T08:24:52.956 INFO:tasks.workunit.client.1.smithi149.stdout:6/501: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 [0,4194304] 0 2024-04-24T08:24:52.964 INFO:tasks.workunit.client.0.smithi090.stdout:5/348: dwrite f7 [4194304,4194304] 0 2024-04-24T08:24:52.965 INFO:tasks.workunit.client.1.smithi149.stdout:1/729: write d0/d5/db/d63/f6b [2205722,1979] 0 2024-04-24T08:24:52.965 INFO:tasks.workunit.client.1.smithi149.stdout:4/555: dread d2/d24/d2e/f4a [0,4194304] 0 2024-04-24T08:24:52.970 INFO:tasks.workunit.client.0.smithi090.stdout:2/275: creat d0/d14/d17/d2a/d31/f56 x:0 0 0 2024-04-24T08:24:52.971 INFO:tasks.workunit.client.1.smithi149.stdout:4/556: fsync d2/d24/f9b 0 2024-04-24T08:24:52.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/671: creat d0/d5/da/d5a/d63/d6a/fe9 x:0 0 0 2024-04-24T08:24:52.973 INFO:tasks.workunit.client.0.smithi090.stdout:2/276: write d0/d7/f11 [9553102,30740] 0 2024-04-24T08:24:52.979 INFO:tasks.workunit.client.1.smithi149.stdout:0/560: dread d2/d9/df/d40/f67 [0,4194304] 0 2024-04-24T08:24:52.982 INFO:tasks.workunit.client.1.smithi149.stdout:2/542: fsync d0/d6/db/d28/f6f 0 2024-04-24T08:24:52.987 INFO:tasks.workunit.client.1.smithi149.stdout:6/502: dwrite d0/d1a/d28/d31/f5c [0,4194304] 0 2024-04-24T08:24:52.992 INFO:tasks.workunit.client.1.smithi149.stdout:1/730: mknod d0/d1b/d5d/de1/cf2 0 2024-04-24T08:24:52.993 INFO:tasks.workunit.client.0.smithi090.stdout:8/241: rmdir d2/da/d13/d15/d24 39 2024-04-24T08:24:52.996 INFO:tasks.workunit.client.0.smithi090.stdout:6/290: mknod d8/d1d/d29/c57 0 2024-04-24T08:24:53.000 INFO:tasks.workunit.client.1.smithi149.stdout:4/557: dwrite d2/d24/d2e/d3f/fa9 [0,4194304] 0 2024-04-24T08:24:53.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/268: link d0/d19/d33/f36 d0/dc/d24/f4e 0 2024-04-24T08:24:53.007 INFO:tasks.workunit.client.0.smithi090.stdout:7/330: write db/fd [1455448,44547] 0 2024-04-24T08:24:53.007 INFO:tasks.workunit.client.1.smithi149.stdout:5/708: dread db/da4/fb5 [0,4194304] 0 2024-04-24T08:24:53.009 INFO:tasks.workunit.client.1.smithi149.stdout:9/642: dread d1/de/d12/f3d [0,4194304] 0 2024-04-24T08:24:53.011 INFO:tasks.workunit.client.0.smithi090.stdout:7/331: write db/d17/d29/d38/d3e/f58 [448782,39925] 0 2024-04-24T08:24:53.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/286: mkdir d3/d68 0 2024-04-24T08:24:53.012 INFO:tasks.workunit.client.1.smithi149.stdout:7/601: dwrite d9/dc/f29 [4194304,4194304] 0 2024-04-24T08:24:53.017 INFO:tasks.workunit.client.0.smithi090.stdout:6/291: unlink d8/d1d/d1f/l25 0 2024-04-24T08:24:53.018 INFO:tasks.workunit.client.0.smithi090.stdout:3/249: getdents d2/d15/d36 0 2024-04-24T08:24:53.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/332: write db/d17/f28 [169867,124824] 0 2024-04-24T08:24:53.019 INFO:tasks.workunit.client.1.smithi149.stdout:6/503: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 [0,4194304] 0 2024-04-24T08:24:53.022 INFO:tasks.workunit.client.1.smithi149.stdout:4/558: creat d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf x:0 0 0 2024-04-24T08:24:53.023 INFO:tasks.workunit.client.1.smithi149.stdout:7/602: mknod d9/dc/d11/d83/da6/d67/d69/ccb 0 2024-04-24T08:24:53.029 INFO:tasks.workunit.client.1.smithi149.stdout:6/504: symlink d0/d1a/d7e/la5 0 2024-04-24T08:24:53.030 INFO:tasks.workunit.client.1.smithi149.stdout:9/643: rmdir d1/da/dd/d79/db6/de1 0 2024-04-24T08:24:53.032 INFO:tasks.workunit.client.1.smithi149.stdout:9/644: chown d1/d2/fba 11518912 1 2024-04-24T08:24:53.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/505: creat d0/d1a/d28/d31/d4c/d4e/d62/fa6 x:0 0 0 2024-04-24T08:24:53.039 INFO:tasks.workunit.client.1.smithi149.stdout:6/506: truncate d0/d1a/d28/d31/d4c/d8a/f9d 256837 0 2024-04-24T08:24:53.040 INFO:tasks.workunit.client.1.smithi149.stdout:7/603: dwrite d9/dc/d11/d26/fae [0,4194304] 0 2024-04-24T08:24:53.041 INFO:tasks.workunit.client.1.smithi149.stdout:6/507: write d0/d1a/f27 [8220170,102016] 0 2024-04-24T08:24:53.048 INFO:tasks.workunit.client.1.smithi149.stdout:6/508: symlink d0/d1a/d28/d31/d4c/d4e/la7 0 2024-04-24T08:24:53.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/509: truncate d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f8c 142155 0 2024-04-24T08:24:53.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/510: creat d0/d1a/d28/d31/d33/fa8 x:0 0 0 2024-04-24T08:24:53.056 INFO:tasks.workunit.client.1.smithi149.stdout:6/511: stat d0/d1a/d28/d31/d33/f9f 0 2024-04-24T08:24:53.059 INFO:tasks.workunit.client.1.smithi149.stdout:6/512: mknod d0/d1a/d28/d31/d4c/d4e/d62/d66/ca9 0 2024-04-24T08:24:53.064 INFO:tasks.workunit.client.1.smithi149.stdout:6/513: rename d0/lf to d0/d1a/d28/d31/d4c/d4e/laa 0 2024-04-24T08:24:53.066 INFO:tasks.workunit.client.1.smithi149.stdout:6/514: symlink d0/d1a/d28/d4a/lab 0 2024-04-24T08:24:53.068 INFO:tasks.workunit.client.1.smithi149.stdout:6/515: write d0/d1a/d28/d31/d33/f9b [533204,17101] 0 2024-04-24T08:24:53.070 INFO:tasks.workunit.client.1.smithi149.stdout:6/516: symlink d0/d1a/lac 0 2024-04-24T08:24:53.071 INFO:tasks.workunit.client.1.smithi149.stdout:6/517: readlink d0/d1a/d28/d31/d38/l8f 0 2024-04-24T08:24:53.072 INFO:tasks.workunit.client.1.smithi149.stdout:6/518: stat d0/d1a/d28/f95 0 2024-04-24T08:24:53.076 INFO:tasks.workunit.client.1.smithi149.stdout:6/519: link d0/f4 d0/d1a/d28/d31/d4c/d4e/d62/d7a/fad 0 2024-04-24T08:24:53.090 INFO:tasks.workunit.client.1.smithi149.stdout:6/520: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f81 [0,4194304] 0 2024-04-24T08:24:53.094 INFO:tasks.workunit.client.1.smithi149.stdout:6/521: truncate d0/d1a/d28/d31/f6c 874441 0 2024-04-24T08:24:53.107 INFO:tasks.workunit.client.1.smithi149.stdout:6/522: dwrite d0/d1a/d28/d4a/f72 [0,4194304] 0 2024-04-24T08:24:53.113 INFO:tasks.workunit.client.0.smithi090.stdout:2/277: mkdir d0/d7/d57 0 2024-04-24T08:24:53.113 INFO:tasks.workunit.client.0.smithi090.stdout:9/269: write d0/dc/d24/f4b [4720447,25202] 0 2024-04-24T08:24:53.119 INFO:tasks.workunit.client.0.smithi090.stdout:0/323: rename d1a/f38 to d1a/d4b/d53/f6a 0 2024-04-24T08:24:53.127 INFO:tasks.workunit.client.1.smithi149.stdout:6/523: dwrite d0/d1a/d28/d31/d4c/d4e/f64 [0,4194304] 0 2024-04-24T08:24:53.131 INFO:tasks.workunit.client.0.smithi090.stdout:7/333: read db/d17/d29/d32/f61 [1978360,10009] 0 2024-04-24T08:24:53.131 INFO:tasks.workunit.client.0.smithi090.stdout:9/270: read d0/d19/f25 [1977521,120745] 0 2024-04-24T08:24:53.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/292: rename d8/d1d/d2f/f39 to d8/d1d/d1f/f58 0 2024-04-24T08:24:53.131 INFO:tasks.workunit.client.1.smithi149.stdout:6/524: fsync d0/d1a/d28/f2c 0 2024-04-24T08:24:53.132 INFO:tasks.workunit.client.0.smithi090.stdout:1/287: dwrite f0 [4194304,4194304] 0 2024-04-24T08:24:53.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/525: read d0/d1a/d28/d31/f4b [11555,119719] 0 2024-04-24T08:24:53.135 INFO:tasks.workunit.client.0.smithi090.stdout:1/288: chown d3/d5/da 18320 1 2024-04-24T08:24:53.139 INFO:tasks.workunit.client.0.smithi090.stdout:8/242: dwrite d2/f3 [4194304,4194304] 0 2024-04-24T08:24:53.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/324: mknod d1a/d4b/c6b 0 2024-04-24T08:24:53.148 INFO:tasks.workunit.client.1.smithi149.stdout:6/526: getdents d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d 0 2024-04-24T08:24:53.156 INFO:tasks.workunit.client.0.smithi090.stdout:4/194: dwrite d3/d25/f2f [0,4194304] 0 2024-04-24T08:24:53.165 INFO:tasks.workunit.client.1.smithi149.stdout:5/709: write db/d12/d60/db2/f6e [5105284,21573] 0 2024-04-24T08:24:53.165 INFO:tasks.workunit.client.0.smithi090.stdout:7/334: mknod db/d17/d29/d32/d5a/c7c 0 2024-04-24T08:24:53.168 INFO:tasks.workunit.client.0.smithi090.stdout:6/293: mkdir d8/d1d/d2f/d30/d38/d59 0 2024-04-24T08:24:53.172 INFO:tasks.workunit.client.0.smithi090.stdout:8/243: mknod d2/da/d13/d15/d50/c51 0 2024-04-24T08:24:53.172 INFO:tasks.workunit.client.1.smithi149.stdout:8/522: sync 2024-04-24T08:24:53.173 INFO:tasks.workunit.client.0.smithi090.stdout:8/244: stat d2/da/d13/d15/f3f 0 2024-04-24T08:24:53.174 INFO:tasks.workunit.client.1.smithi149.stdout:6/527: symlink d0/d1a/d28/lae 0 2024-04-24T08:24:53.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/271: mknod d0/dc/d24/d3d/c4f 0 2024-04-24T08:24:53.178 INFO:tasks.workunit.client.1.smithi149.stdout:3/672: dwrite d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:53.179 INFO:tasks.workunit.client.0.smithi090.stdout:9/272: read d0/d19/f28 [1368549,121864] 0 2024-04-24T08:24:53.180 INFO:tasks.workunit.client.0.smithi090.stdout:9/273: readlink d0/l42 0 2024-04-24T08:24:53.183 INFO:tasks.workunit.client.0.smithi090.stdout:0/325: symlink d1a/d2a/d35/d62/l6c 0 2024-04-24T08:24:53.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/710: chown db/d12/d22/l7d 72026 1 2024-04-24T08:24:53.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/731: dread d0/fbd [0,4194304] 0 2024-04-24T08:24:53.186 INFO:tasks.workunit.client.1.smithi149.stdout:9/645: write d1/da/d13/f1e [2608928,23604] 0 2024-04-24T08:24:53.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/195: creat d3/f32 x:0 0 0 2024-04-24T08:24:53.188 INFO:tasks.workunit.client.1.smithi149.stdout:9/646: stat d1/d99/fd7 0 2024-04-24T08:24:53.190 INFO:tasks.workunit.client.1.smithi149.stdout:3/673: chown d0/d5/da/d19/d80/lb9 6691 1 2024-04-24T08:24:53.192 INFO:tasks.workunit.client.1.smithi149.stdout:8/523: write d1/d1f/fa6 [524515,65435] 0 2024-04-24T08:24:53.193 INFO:tasks.workunit.client.0.smithi090.stdout:6/294: rmdir d8/d1d 39 2024-04-24T08:24:53.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/289: dread d3/d5/da/dc/f27 [0,4194304] 0 2024-04-24T08:24:53.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/524: stat d1/de/d18/d91/d9f 0 2024-04-24T08:24:53.196 INFO:tasks.workunit.client.1.smithi149.stdout:4/559: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:24:53.196 INFO:tasks.workunit.client.1.smithi149.stdout:1/732: chown d0/d5/l32 24992 1 2024-04-24T08:24:53.199 INFO:tasks.workunit.client.1.smithi149.stdout:1/733: dread - d0/da/f48 zero size 2024-04-24T08:24:53.201 INFO:tasks.workunit.client.0.smithi090.stdout:1/290: write d3/f4 [9127545,55394] 0 2024-04-24T08:24:53.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/525: read d1/d1f/d22/f5f [2633790,60541] 0 2024-04-24T08:24:53.204 INFO:tasks.workunit.client.1.smithi149.stdout:6/528: symlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/laf 0 2024-04-24T08:24:53.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/274: dwrite d0/d19/f1e [0,4194304] 0 2024-04-24T08:24:53.206 INFO:tasks.workunit.client.1.smithi149.stdout:7/604: dwrite d9/dc/de/d44/f74 [0,4194304] 0 2024-04-24T08:24:53.208 INFO:tasks.workunit.client.1.smithi149.stdout:6/529: readlink d0/d1a/d28/d4a/lab 0 2024-04-24T08:24:53.208 INFO:tasks.workunit.client.0.smithi090.stdout:1/291: write d3/d25/f4a [1134919,72044] 0 2024-04-24T08:24:53.212 INFO:tasks.workunit.client.0.smithi090.stdout:1/292: stat d3/d5/d3e/c29 0 2024-04-24T08:24:53.215 INFO:tasks.workunit.client.0.smithi090.stdout:4/196: dwrite d3/f32 [0,4194304] 0 2024-04-24T08:24:53.217 INFO:tasks.workunit.client.0.smithi090.stdout:1/293: fdatasync d3/d1d/d54/f67 0 2024-04-24T08:24:53.218 INFO:tasks.workunit.client.0.smithi090.stdout:4/197: chown d3 4 1 2024-04-24T08:24:53.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/294: chown d3/d5/da/dc 316635429 1 2024-04-24T08:24:53.220 INFO:tasks.workunit.client.0.smithi090.stdout:8/245: dread d2/da/d13/d15/f26 [0,4194304] 0 2024-04-24T08:24:53.221 INFO:tasks.workunit.client.1.smithi149.stdout:3/674: rmdir d0/d5/d53/d4c/d5f 39 2024-04-24T08:24:53.224 INFO:tasks.workunit.client.0.smithi090.stdout:3/250: dwrite d2/df/d1a/f2b [0,4194304] 0 2024-04-24T08:24:53.227 INFO:tasks.workunit.client.0.smithi090.stdout:0/326: creat d1a/d2a/d69/f6d x:0 0 0 2024-04-24T08:24:53.227 INFO:tasks.workunit.client.0.smithi090.stdout:3/251: readlink d2/d25/d2c/l42 0 2024-04-24T08:24:53.228 INFO:tasks.workunit.client.1.smithi149.stdout:3/675: chown d0/d5/da/d19/d1b/d72/fdc 40787 1 2024-04-24T08:24:53.228 INFO:tasks.workunit.client.0.smithi090.stdout:3/252: readlink d2/df/l4a 0 2024-04-24T08:24:53.232 INFO:tasks.workunit.client.1.smithi149.stdout:5/711: mknod db/d12/d22/d45/d5d/de6/cef 0 2024-04-24T08:24:53.236 INFO:tasks.workunit.client.1.smithi149.stdout:5/712: stat db/da4/fb5 0 2024-04-24T08:24:53.238 INFO:tasks.workunit.client.1.smithi149.stdout:5/713: stat db/d12/d22/ld9 0 2024-04-24T08:24:53.239 INFO:tasks.workunit.client.0.smithi090.stdout:9/275: fsync d0/f2 0 2024-04-24T08:24:53.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/276: readlink d0/l9 0 2024-04-24T08:24:53.241 INFO:tasks.workunit.client.1.smithi149.stdout:4/560: chown d2/d24/d2e/la1 246 1 2024-04-24T08:24:53.241 INFO:tasks.workunit.client.1.smithi149.stdout:1/734: unlink d0/d5/db/d18/d4d/d50/ca8 0 2024-04-24T08:24:53.243 INFO:tasks.workunit.client.0.smithi090.stdout:7/335: write db/d17/d29/d32/d5a/d75/f26 [3007018,91257] 0 2024-04-24T08:24:53.245 INFO:tasks.workunit.client.0.smithi090.stdout:6/295: fsync d8/dc/d19/f34 0 2024-04-24T08:24:53.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/336: write db/fd [4167963,90435] 0 2024-04-24T08:24:53.247 INFO:tasks.workunit.client.1.smithi149.stdout:7/605: symlink d9/d43/lcc 0 2024-04-24T08:24:53.249 INFO:tasks.workunit.client.0.smithi090.stdout:4/198: mknod d3/d18/d1d/c33 0 2024-04-24T08:24:53.249 INFO:tasks.workunit.client.1.smithi149.stdout:6/530: rmdir d0/d1a/d28/d4a 39 2024-04-24T08:24:53.254 INFO:tasks.workunit.client.0.smithi090.stdout:8/246: creat d2/da/d13/f52 x:0 0 0 2024-04-24T08:24:53.254 INFO:tasks.workunit.client.1.smithi149.stdout:2/543: sync 2024-04-24T08:24:53.254 INFO:tasks.workunit.client.1.smithi149.stdout:0/561: sync 2024-04-24T08:24:53.255 INFO:tasks.workunit.client.0.smithi090.stdout:0/327: creat d1a/d2a/d35/d62/f6e x:0 0 0 2024-04-24T08:24:53.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/277: mkdir d0/d19/d33/d50 0 2024-04-24T08:24:53.258 INFO:tasks.workunit.client.0.smithi090.stdout:3/253: dwrite d2/d25/d33/f43 [0,4194304] 0 2024-04-24T08:24:53.263 INFO:tasks.workunit.client.0.smithi090.stdout:6/296: chown d8/d1d/c4a 192526 1 2024-04-24T08:24:53.264 INFO:tasks.workunit.client.1.smithi149.stdout:8/526: symlink d1/de/lb4 0 2024-04-24T08:24:53.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/647: write d1/da/dd/d79/f56 [2435481,107439] 0 2024-04-24T08:24:53.268 INFO:tasks.workunit.client.0.smithi090.stdout:4/199: dread d3/d25/f2f [0,4194304] 0 2024-04-24T08:24:53.269 INFO:tasks.workunit.client.0.smithi090.stdout:1/295: mknod d3/c69 0 2024-04-24T08:24:53.272 INFO:tasks.workunit.client.1.smithi149.stdout:0/562: creat d2/d9/d1a/d36/d85/fb2 x:0 0 0 2024-04-24T08:24:53.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:52 smithi149 ceph-mon[31282]: pgmap v19: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 249 MiB/s wr, 446 op/s 2024-04-24T08:24:53.277 INFO:tasks.workunit.client.0.smithi090.stdout:5/349: sync 2024-04-24T08:24:53.277 INFO:tasks.workunit.client.0.smithi090.stdout:2/278: sync 2024-04-24T08:24:53.277 INFO:tasks.workunit.client.1.smithi149.stdout:5/714: write db/d12/d1e/d85/d92/fcc [659010,38026] 0 2024-04-24T08:24:53.279 INFO:tasks.workunit.client.1.smithi149.stdout:4/561: unlink d2/d18/d1e/d3b/f7c 0 2024-04-24T08:24:53.279 INFO:tasks.workunit.client.1.smithi149.stdout:5/715: dread - db/d1f/feb zero size 2024-04-24T08:24:53.282 INFO:tasks.workunit.client.1.smithi149.stdout:5/716: write db/f94 [3870497,115405] 0 2024-04-24T08:24:53.284 INFO:tasks.workunit.client.1.smithi149.stdout:7/606: mkdir d9/dcd 0 2024-04-24T08:24:53.284 INFO:tasks.workunit.client.1.smithi149.stdout:8/527: dwrite d1/de/d18/d32/d65/f81 [4194304,4194304] 0 2024-04-24T08:24:53.287 INFO:tasks.workunit.client.1.smithi149.stdout:9/648: rmdir d1/d2/d15/d1c 39 2024-04-24T08:24:53.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/528: chown d1/de/l2e 19564984 1 2024-04-24T08:24:53.290 INFO:tasks.workunit.client.1.smithi149.stdout:3/676: dread d0/f4 [4194304,4194304] 0 2024-04-24T08:24:53.290 INFO:tasks.workunit.client.0.smithi090.stdout:8/247: rmdir d2/da 39 2024-04-24T08:24:53.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:52 smithi090 ceph-mon[25016]: pgmap v19: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 249 MiB/s wr, 446 op/s 2024-04-24T08:24:53.295 INFO:tasks.workunit.client.1.smithi149.stdout:1/735: rename d0/d5/ld7 to d0/lf3 0 2024-04-24T08:24:53.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/649: mkdir d1/da/d13/d67/de4 0 2024-04-24T08:24:53.300 INFO:tasks.workunit.client.0.smithi090.stdout:4/200: fdatasync d3/d18/d1d/f1f 0 2024-04-24T08:24:53.301 INFO:tasks.workunit.client.1.smithi149.stdout:8/529: dread - d1/de/d18/d32/d65/f66 zero size 2024-04-24T08:24:53.303 INFO:tasks.workunit.client.1.smithi149.stdout:8/530: stat d1/d1f/d25/f42 0 2024-04-24T08:24:53.304 INFO:tasks.workunit.client.1.smithi149.stdout:2/544: rmdir d0/d6/db/d39/d3d/d8c 0 2024-04-24T08:24:53.306 INFO:tasks.workunit.client.1.smithi149.stdout:8/531: read d1/de/d18/d32/d65/f81 [8077051,94617] 0 2024-04-24T08:24:53.307 INFO:tasks.workunit.client.0.smithi090.stdout:7/337: write db/d17/d29/f30 [1689754,31976] 0 2024-04-24T08:24:53.308 INFO:tasks.workunit.client.1.smithi149.stdout:0/563: truncate d2/d9/fe 1082541 0 2024-04-24T08:24:53.308 INFO:tasks.workunit.client.0.smithi090.stdout:0/328: write f18 [3334518,31535] 0 2024-04-24T08:24:53.309 INFO:tasks.workunit.client.1.smithi149.stdout:4/562: symlink d2/lb0 0 2024-04-24T08:24:53.311 INFO:tasks.workunit.client.0.smithi090.stdout:3/254: write d2/df/f10 [1005520,22483] 0 2024-04-24T08:24:53.312 INFO:tasks.workunit.client.1.smithi149.stdout:6/531: rename d0/d1a/d28/d4a/lab to d0/d1a/d28/d31/d38/d54/d87/lb0 0 2024-04-24T08:24:53.313 INFO:tasks.workunit.client.0.smithi090.stdout:6/297: dwrite d8/d1d/d2f/d30/f4b [0,4194304] 0 2024-04-24T08:24:53.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/279: write d0/d14/f2b [1090817,85897] 0 2024-04-24T08:24:53.320 INFO:tasks.workunit.client.0.smithi090.stdout:9/278: creat d0/d19/d33/d50/f51 x:0 0 0 2024-04-24T08:24:53.321 INFO:tasks.workunit.client.1.smithi149.stdout:9/650: creat d1/d2/d33/d5d/d83/fe5 x:0 0 0 2024-04-24T08:24:53.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/338: dwrite db/f10 [0,4194304] 0 2024-04-24T08:24:53.325 INFO:tasks.workunit.client.0.smithi090.stdout:7/339: truncate db/d17/f7a 119711 0 2024-04-24T08:24:53.326 INFO:tasks.workunit.client.1.smithi149.stdout:6/532: chown d0/d1a/d28/d31/d4c/d4e/d62/c8b 3392971 1 2024-04-24T08:24:53.326 INFO:tasks.workunit.client.1.smithi149.stdout:6/533: readlink d0/d1a/l83 0 2024-04-24T08:24:53.326 INFO:tasks.workunit.client.0.smithi090.stdout:4/201: mkdir d3/d18/d34 0 2024-04-24T08:24:53.333 INFO:tasks.workunit.client.0.smithi090.stdout:1/296: link d3/d1d/d34/f3b d3/d5/da/f6a 0 2024-04-24T08:24:53.336 INFO:tasks.workunit.client.1.smithi149.stdout:8/532: creat d1/d1f/d5e/d9c/fb5 x:0 0 0 2024-04-24T08:24:53.336 INFO:tasks.workunit.client.1.smithi149.stdout:5/717: dwrite db/d12/f3d [0,4194304] 0 2024-04-24T08:24:53.336 INFO:tasks.workunit.client.0.smithi090.stdout:5/350: getdents d8/d76 0 2024-04-24T08:24:53.336 INFO:tasks.workunit.client.1.smithi149.stdout:4/563: creat d2/d18/d1e/d3c/d59/fb1 x:0 0 0 2024-04-24T08:24:53.338 INFO:tasks.workunit.client.1.smithi149.stdout:7/607: rename d9/dc/de/l9a to d9/dc/d11/d84/lce 0 2024-04-24T08:24:53.339 INFO:tasks.workunit.client.1.smithi149.stdout:0/564: write d2/d9/d1a/d36/f47 [1866251,72489] 0 2024-04-24T08:24:53.341 INFO:tasks.workunit.client.1.smithi149.stdout:5/718: chown db/d12/d60/db2/d83/faf 7 1 2024-04-24T08:24:53.344 INFO:tasks.workunit.client.0.smithi090.stdout:7/340: dwrite db/d17/f28 [0,4194304] 0 2024-04-24T08:24:53.346 INFO:tasks.workunit.client.1.smithi149.stdout:0/565: read d2/d9/df/d40/f67 [541179,3025] 0 2024-04-24T08:24:53.346 INFO:tasks.workunit.client.1.smithi149.stdout:9/651: creat d1/da/d13/d67/fe6 x:0 0 0 2024-04-24T08:24:53.347 INFO:tasks.workunit.client.0.smithi090.stdout:9/279: creat d0/d19/d33/d4d/f52 x:0 0 0 2024-04-24T08:24:53.348 INFO:tasks.workunit.client.0.smithi090.stdout:7/341: write db/d17/d29/d32/f57 [1841240,125027] 0 2024-04-24T08:24:53.349 INFO:tasks.workunit.client.1.smithi149.stdout:1/736: dread d0/ddf/fc0 [0,4194304] 0 2024-04-24T08:24:53.354 INFO:tasks.workunit.client.1.smithi149.stdout:1/737: chown d0/d1b/d5d/de1/l75 3538375 1 2024-04-24T08:24:53.354 INFO:tasks.workunit.client.0.smithi090.stdout:1/297: rename d3/d5/f3f to d3/d1d/d34/f6b 0 2024-04-24T08:24:53.356 INFO:tasks.workunit.client.0.smithi090.stdout:1/298: readlink d3/l45 0 2024-04-24T08:24:53.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/351: chown d8/d59/l5d 1513817553 1 2024-04-24T08:24:53.361 INFO:tasks.workunit.client.0.smithi090.stdout:3/255: mkdir d2/d3a/d4f 0 2024-04-24T08:24:53.363 INFO:tasks.workunit.client.0.smithi090.stdout:6/298: dread d8/dc/d19/f27 [0,4194304] 0 2024-04-24T08:24:53.363 INFO:tasks.workunit.client.1.smithi149.stdout:2/545: truncate d0/d6/db/f1f 3956484 0 2024-04-24T08:24:53.363 INFO:tasks.workunit.client.0.smithi090.stdout:3/256: dread - d2/df/d1a/d37/f49 zero size 2024-04-24T08:24:53.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/329: truncate d1a/d4b/d53/f6a 3445989 0 2024-04-24T08:24:53.369 INFO:tasks.workunit.client.0.smithi090.stdout:0/330: write d1a/f24 [7196337,29944] 0 2024-04-24T08:24:53.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/280: dwrite d0/dc/d24/f4b [0,4194304] 0 2024-04-24T08:24:53.374 INFO:tasks.workunit.client.0.smithi090.stdout:9/281: readlink d0/d19/d33/d4d/l23 0 2024-04-24T08:24:53.376 INFO:tasks.workunit.client.0.smithi090.stdout:1/299: creat d3/d5/da/dc/f6c x:0 0 0 2024-04-24T08:24:53.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/352: mknod d8/da/d14/c7d 0 2024-04-24T08:24:53.377 INFO:tasks.workunit.client.0.smithi090.stdout:6/299: rmdir d8/d1d 39 2024-04-24T08:24:53.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/342: dwrite db/fd [4194304,4194304] 0 2024-04-24T08:24:53.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/202: dwrite d3/f16 [4194304,4194304] 0 2024-04-24T08:24:53.388 INFO:tasks.workunit.client.0.smithi090.stdout:2/280: creat d0/d14/f58 x:0 0 0 2024-04-24T08:24:53.389 INFO:tasks.workunit.client.1.smithi149.stdout:6/534: symlink d0/d1a/d28/d31/d4c/lb1 0 2024-04-24T08:24:53.393 INFO:tasks.workunit.client.1.smithi149.stdout:8/533: fdatasync d1/de/d18/d32/d65/d8b/f7c 0 2024-04-24T08:24:53.394 INFO:tasks.workunit.client.1.smithi149.stdout:6/535: write d0/d1a/d28/d31/d33/f9f [520522,109687] 0 2024-04-24T08:24:53.394 INFO:tasks.workunit.client.0.smithi090.stdout:9/282: rmdir d0/d2a 39 2024-04-24T08:24:53.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/257: fsync d2/df/f19 0 2024-04-24T08:24:53.397 INFO:tasks.workunit.client.0.smithi090.stdout:9/283: write d0/dc/f29 [592089,86085] 0 2024-04-24T08:24:53.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/300: creat d3/d25/f6d x:0 0 0 2024-04-24T08:24:53.405 INFO:tasks.workunit.client.1.smithi149.stdout:1/738: dread d0/d1b/d5d/f9a [0,4194304] 0 2024-04-24T08:24:53.406 INFO:tasks.workunit.client.1.smithi149.stdout:3/677: rename d0/d5/da/d19/d76/fd8 to d0/d5/d53/d6d/fea 0 2024-04-24T08:24:53.408 INFO:tasks.workunit.client.0.smithi090.stdout:6/300: stat d8/dc/d19/l2d 0 2024-04-24T08:24:53.409 INFO:tasks.workunit.client.1.smithi149.stdout:7/608: fdatasync d9/dc/d11/d38/f56 0 2024-04-24T08:24:53.418 INFO:tasks.workunit.client.0.smithi090.stdout:4/203: creat d3/d18/d1d/d22/f35 x:0 0 0 2024-04-24T08:24:53.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/204: chown d3/l29 12 1 2024-04-24T08:24:53.420 INFO:tasks.workunit.client.0.smithi090.stdout:9/284: dread d0/fb [0,4194304] 0 2024-04-24T08:24:53.420 INFO:tasks.workunit.client.1.smithi149.stdout:4/564: dwrite d2/d10/d91/f9d [0,4194304] 0 2024-04-24T08:24:53.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/205: write d3/d18/d1d/f27 [784610,112715] 0 2024-04-24T08:24:53.424 INFO:tasks.workunit.client.1.smithi149.stdout:4/565: read d2/d24/f9b [5684723,21262] 0 2024-04-24T08:24:53.424 INFO:tasks.workunit.client.0.smithi090.stdout:9/285: truncate d0/d19/d33/d50/f51 678685 0 2024-04-24T08:24:53.426 INFO:tasks.workunit.client.1.smithi149.stdout:1/739: dwrite d0/ddf/d53/d61/fc7 [0,4194304] 0 2024-04-24T08:24:53.427 INFO:tasks.workunit.client.1.smithi149.stdout:2/546: rmdir d0/d6 39 2024-04-24T08:24:53.433 INFO:tasks.workunit.client.0.smithi090.stdout:9/286: dread d0/dc/d24/d2d/f3f [0,4194304] 0 2024-04-24T08:24:53.434 INFO:tasks.workunit.client.1.smithi149.stdout:6/536: creat d0/d1a/d28/d4a/fb2 x:0 0 0 2024-04-24T08:24:53.435 INFO:tasks.workunit.client.0.smithi090.stdout:9/287: dread - d0/d19/d33/d4d/f52 zero size 2024-04-24T08:24:53.437 INFO:tasks.workunit.client.0.smithi090.stdout:3/258: creat d2/d25/f50 x:0 0 0 2024-04-24T08:24:53.439 INFO:tasks.workunit.client.0.smithi090.stdout:3/259: dread - d2/d25/f48 zero size 2024-04-24T08:24:53.440 INFO:tasks.workunit.client.1.smithi149.stdout:8/534: rename d1/de/f1a to d1/de/d18/d43/fb6 0 2024-04-24T08:24:53.441 INFO:tasks.workunit.client.0.smithi090.stdout:3/260: write d2/f17 [2196765,73890] 0 2024-04-24T08:24:53.442 INFO:tasks.workunit.client.0.smithi090.stdout:8/248: link d2/l29 d2/da/d13/d15/l53 0 2024-04-24T08:24:53.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/678: mkdir d0/d26/d3c/db1/deb 0 2024-04-24T08:24:53.447 INFO:tasks.workunit.client.0.smithi090.stdout:0/331: creat d1a/d4b/f6f x:0 0 0 2024-04-24T08:24:53.449 INFO:tasks.workunit.client.1.smithi149.stdout:7/609: unlink d9/dc/de/lbd 0 2024-04-24T08:24:53.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/281: mknod d0/d7/d57/c59 0 2024-04-24T08:24:53.454 INFO:tasks.workunit.client.0.smithi090.stdout:9/288: dwrite d0/d19/f1c [4194304,4194304] 0 2024-04-24T08:24:53.454 INFO:tasks.workunit.client.1.smithi149.stdout:0/566: creat d2/d17/d41/fb3 x:0 0 0 2024-04-24T08:24:53.456 INFO:tasks.workunit.client.1.smithi149.stdout:9/652: mkdir d1/d2/d15/de7 0 2024-04-24T08:24:53.457 INFO:tasks.workunit.client.0.smithi090.stdout:4/206: unlink d3/d18/d1d/c33 0 2024-04-24T08:24:53.459 INFO:tasks.workunit.client.1.smithi149.stdout:6/537: dwrite d0/d1a/d28/d4a/f72 [0,4194304] 0 2024-04-24T08:24:53.461 INFO:tasks.workunit.client.1.smithi149.stdout:7/610: dread - d9/dc/d11/d83/da6/d67/d69/dad/fb9 zero size 2024-04-24T08:24:53.461 INFO:tasks.workunit.client.0.smithi090.stdout:7/343: dread db/d17/d29/d32/f3d [0,4194304] 0 2024-04-24T08:24:53.468 INFO:tasks.workunit.client.0.smithi090.stdout:6/301: write d8/d1d/d29/f40 [2730,83022] 0 2024-04-24T08:24:53.470 INFO:tasks.workunit.client.1.smithi149.stdout:3/679: dwrite d0/d5/da/d59/fc9 [0,4194304] 0 2024-04-24T08:24:53.473 INFO:tasks.workunit.client.0.smithi090.stdout:5/353: rename d8/da/f1d to d8/d59/f7e 0 2024-04-24T08:24:53.480 INFO:tasks.workunit.client.1.smithi149.stdout:5/719: link db/d12/d60/c96 db/d12/d60/db2/cf0 0 2024-04-24T08:24:53.486 INFO:tasks.workunit.client.0.smithi090.stdout:6/302: dwrite d8/d1d/f24 [0,4194304] 0 2024-04-24T08:24:53.492 INFO:tasks.workunit.client.1.smithi149.stdout:7/611: dwrite d9/dc/de/f31 [0,4194304] 0 2024-04-24T08:24:53.493 INFO:tasks.workunit.client.1.smithi149.stdout:0/567: creat d2/d9/df/d40/fb4 x:0 0 0 2024-04-24T08:24:53.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/261: dread d2/d15/f16 [0,4194304] 0 2024-04-24T08:24:53.499 INFO:tasks.workunit.client.1.smithi149.stdout:6/538: symlink d0/d1a/d28/d31/d38/lb3 0 2024-04-24T08:24:53.502 INFO:tasks.workunit.client.1.smithi149.stdout:3/680: dwrite d0/d5/da/fa7 [0,4194304] 0 2024-04-24T08:24:53.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/344: unlink db/d17/l27 0 2024-04-24T08:24:53.506 INFO:tasks.workunit.client.1.smithi149.stdout:3/681: stat d0/d5/d1a/lb4 0 2024-04-24T08:24:53.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/301: creat d3/d1d/f6e x:0 0 0 2024-04-24T08:24:53.508 INFO:tasks.workunit.client.0.smithi090.stdout:7/345: dread - db/d17/d29/f55 zero size 2024-04-24T08:24:53.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/302: fsync d3/d5/f1a 0 2024-04-24T08:24:53.511 INFO:tasks.workunit.client.1.smithi149.stdout:1/740: mknod d0/d5/db/d63/cf4 0 2024-04-24T08:24:53.513 INFO:tasks.workunit.client.1.smithi149.stdout:2/547: rename d0/d6/f41 to d0/d6c/fa8 0 2024-04-24T08:24:53.516 INFO:tasks.workunit.client.0.smithi090.stdout:3/262: dwrite d2/d25/f48 [0,4194304] 0 2024-04-24T08:24:53.518 INFO:tasks.workunit.client.0.smithi090.stdout:1/303: truncate d3/d1d/f55 999004 0 2024-04-24T08:24:53.520 INFO:tasks.workunit.client.1.smithi149.stdout:8/535: write d1/de/d18/d43/f55 [2367495,62164] 0 2024-04-24T08:24:53.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/354: unlink d8/d59/f7e 0 2024-04-24T08:24:53.529 INFO:tasks.workunit.client.1.smithi149.stdout:6/539: dwrite d0/d1a/d28/d31/f5c [0,4194304] 0 2024-04-24T08:24:53.530 INFO:tasks.workunit.client.0.smithi090.stdout:5/355: write d8/da/d18/f1e [2460631,75926] 0 2024-04-24T08:24:53.538 INFO:tasks.workunit.client.1.smithi149.stdout:9/653: write d1/f59 [1323018,102041] 0 2024-04-24T08:24:53.538 INFO:tasks.workunit.client.1.smithi149.stdout:4/566: truncate d2/d10/f17 2204375 0 2024-04-24T08:24:53.540 INFO:tasks.workunit.client.1.smithi149.stdout:4/567: chown d2/d10 7297101 1 2024-04-24T08:24:53.550 INFO:tasks.workunit.client.0.smithi090.stdout:2/282: mknod d0/d14/d4d/d50/c5a 0 2024-04-24T08:24:53.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/283: stat d0/d2e/f49 0 2024-04-24T08:24:53.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/284: fsync d0/f3b 0 2024-04-24T08:24:53.553 INFO:tasks.workunit.client.1.smithi149.stdout:3/682: creat d0/d5/da/d19/d1b/d22/dd4/fec x:0 0 0 2024-04-24T08:24:53.557 INFO:tasks.workunit.client.1.smithi149.stdout:5/720: dread db/f1a [0,4194304] 0 2024-04-24T08:24:53.557 INFO:tasks.workunit.client.1.smithi149.stdout:6/540: creat d0/d1a/d28/d31/d4c/d76/fb4 x:0 0 0 2024-04-24T08:24:53.559 INFO:tasks.workunit.client.1.smithi149.stdout:9/654: fdatasync d1/da/d4a/dad/fc2 0 2024-04-24T08:24:53.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/263: symlink d2/d25/d2c/l51 0 2024-04-24T08:24:53.560 INFO:tasks.workunit.client.0.smithi090.stdout:0/332: link d1a/d2a/d35/c36 d1a/d2a/d3c/c70 0 2024-04-24T08:24:53.562 INFO:tasks.workunit.client.1.smithi149.stdout:2/548: fdatasync d0/d6/db/d39/f4e 0 2024-04-24T08:24:53.567 INFO:tasks.workunit.client.0.smithi090.stdout:4/207: creat d3/df/f36 x:0 0 0 2024-04-24T08:24:53.568 INFO:tasks.workunit.client.1.smithi149.stdout:4/568: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2 0 2024-04-24T08:24:53.572 INFO:tasks.workunit.client.0.smithi090.stdout:8/249: link d2/f10 d2/f54 0 2024-04-24T08:24:53.573 INFO:tasks.workunit.client.1.smithi149.stdout:3/683: dread d0/d5/da/f24 [0,4194304] 0 2024-04-24T08:24:53.574 INFO:tasks.workunit.client.1.smithi149.stdout:6/541: unlink d0/d1a/d28/d31/d4c/d8a/f9d 0 2024-04-24T08:24:53.575 INFO:tasks.workunit.client.1.smithi149.stdout:6/542: readlink d0/d1a/d7e/la5 0 2024-04-24T08:24:53.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/264: unlink d2/df/d1a/c1f 0 2024-04-24T08:24:53.578 INFO:tasks.workunit.client.1.smithi149.stdout:8/536: creat d1/de/d18/fb7 x:0 0 0 2024-04-24T08:24:53.580 INFO:tasks.workunit.client.0.smithi090.stdout:9/289: rename d0/d2a/f43 to d0/f53 0 2024-04-24T08:24:53.581 INFO:tasks.workunit.client.1.smithi149.stdout:4/569: readlink d2/d18/l50 0 2024-04-24T08:24:53.581 INFO:tasks.workunit.client.0.smithi090.stdout:6/303: creat d8/d1d/f5a x:0 0 0 2024-04-24T08:24:53.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/208: mknod d3/d24/c37 0 2024-04-24T08:24:53.582 INFO:tasks.workunit.client.0.smithi090.stdout:6/304: stat d8/cd 0 2024-04-24T08:24:53.583 INFO:tasks.workunit.client.0.smithi090.stdout:4/209: stat d3/df/c14 0 2024-04-24T08:24:53.585 INFO:tasks.workunit.client.1.smithi149.stdout:3/684: creat d0/d5/d53/d34/d3a/d93/fed x:0 0 0 2024-04-24T08:24:53.586 INFO:tasks.workunit.client.0.smithi090.stdout:2/285: dread d0/d7/f11 [0,4194304] 0 2024-04-24T08:24:53.587 INFO:tasks.workunit.client.0.smithi090.stdout:3/265: creat d2/d15/f52 x:0 0 0 2024-04-24T08:24:53.587 INFO:tasks.workunit.client.1.smithi149.stdout:6/543: creat d0/d1a/d28/d31/d4c/d8a/fb5 x:0 0 0 2024-04-24T08:24:53.587 INFO:tasks.workunit.client.1.smithi149.stdout:5/721: dread db/d35/f7c [0,4194304] 0 2024-04-24T08:24:53.588 INFO:tasks.workunit.client.0.smithi090.stdout:8/250: fdatasync d2/da/d13/d15/d24/f46 0 2024-04-24T08:24:53.588 INFO:tasks.workunit.client.0.smithi090.stdout:0/333: truncate d1a/f1b 661155 0 2024-04-24T08:24:53.588 INFO:tasks.workunit.client.0.smithi090.stdout:2/286: dread - d0/d2c/f43 zero size 2024-04-24T08:24:53.589 INFO:tasks.workunit.client.0.smithi090.stdout:2/287: stat d0/d7 0 2024-04-24T08:24:53.591 INFO:tasks.workunit.client.0.smithi090.stdout:2/288: chown d0/d2e/l4a 410995234 1 2024-04-24T08:24:53.591 INFO:tasks.workunit.client.0.smithi090.stdout:7/346: getdents db/d17/d29/d38 0 2024-04-24T08:24:53.592 INFO:tasks.workunit.client.1.smithi149.stdout:3/685: readlink d0/d5/da/d19/d1b/d22/d38/ld7 0 2024-04-24T08:24:53.593 INFO:tasks.workunit.client.0.smithi090.stdout:2/289: write d0/d2e/f49 [1675653,40332] 0 2024-04-24T08:24:53.594 INFO:tasks.workunit.client.0.smithi090.stdout:4/210: rmdir d3/d18/d1d 39 2024-04-24T08:24:53.594 INFO:tasks.workunit.client.1.smithi149.stdout:8/537: mknod d1/de/d18/daf/cb8 0 2024-04-24T08:24:53.597 INFO:tasks.workunit.client.1.smithi149.stdout:5/722: readlink db/d12/d2f/d74/lb3 0 2024-04-24T08:24:53.597 INFO:tasks.workunit.client.0.smithi090.stdout:6/305: symlink d8/dc/d19/l5b 0 2024-04-24T08:24:53.599 INFO:tasks.workunit.client.1.smithi149.stdout:4/570: creat d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 x:0 0 0 2024-04-24T08:24:53.599 INFO:tasks.workunit.client.1.smithi149.stdout:5/723: write db/d12/d22/d45/f21 [7987124,126584] 0 2024-04-24T08:24:53.601 INFO:tasks.workunit.client.1.smithi149.stdout:9/655: getdents d1/d4c/d91 0 2024-04-24T08:24:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:8/538: readlink d1/de/lb2 0 2024-04-24T08:24:53.606 INFO:tasks.workunit.client.1.smithi149.stdout:3/686: write d0/d5/d53/d4c/d5f/f8b [258424,105343] 0 2024-04-24T08:24:53.606 INFO:tasks.workunit.client.0.smithi090.stdout:3/266: dwrite d2/df/d21/f3e [0,4194304] 0 2024-04-24T08:24:53.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/347: symlink db/d54/d78/l7d 0 2024-04-24T08:24:53.610 INFO:tasks.workunit.client.1.smithi149.stdout:3/687: chown d0/d88 22148271 1 2024-04-24T08:24:53.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/251: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:24:53.611 INFO:tasks.workunit.client.0.smithi090.stdout:6/306: truncate d8/f1b 707951 0 2024-04-24T08:24:53.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/539: mknod d1/de/cb9 0 2024-04-24T08:24:53.614 INFO:tasks.workunit.client.1.smithi149.stdout:4/571: getdents d2/d18/d1e/d3c/d48/d69 0 2024-04-24T08:24:53.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/290: dwrite d0/dc/fd [0,4194304] 0 2024-04-24T08:24:53.615 INFO:tasks.workunit.client.0.smithi090.stdout:6/307: dread - d8/d1d/d2f/f3c zero size 2024-04-24T08:24:53.617 INFO:tasks.workunit.client.1.smithi149.stdout:3/688: creat d0/d26/fee x:0 0 0 2024-04-24T08:24:53.618 INFO:tasks.workunit.client.1.smithi149.stdout:8/540: rmdir d1/de/d18/d43 39 2024-04-24T08:24:53.619 INFO:tasks.workunit.client.0.smithi090.stdout:6/308: read f6 [1926239,103964] 0 2024-04-24T08:24:53.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/724: dwrite db/d12/d60/f67 [0,4194304] 0 2024-04-24T08:24:53.620 INFO:tasks.workunit.client.0.smithi090.stdout:3/267: mkdir d2/df/d1a/d37/d53 0 2024-04-24T08:24:53.621 INFO:tasks.workunit.client.0.smithi090.stdout:6/309: dread - d8/d1d/d1f/f2e zero size 2024-04-24T08:24:53.622 INFO:tasks.workunit.client.0.smithi090.stdout:8/252: symlink d2/da/d13/d34/l55 0 2024-04-24T08:24:53.623 INFO:tasks.workunit.client.0.smithi090.stdout:7/348: mknod db/d17/d29/d38/d3e/d46/c7e 0 2024-04-24T08:24:53.625 INFO:tasks.workunit.client.1.smithi149.stdout:4/572: symlink d2/da8/lb4 0 2024-04-24T08:24:53.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/268: mknod d2/df/d1a/c54 0 2024-04-24T08:24:53.629 INFO:tasks.workunit.client.1.smithi149.stdout:4/573: write d2/d18/d1e/d3c/f80 [3428536,113634] 0 2024-04-24T08:24:53.631 INFO:tasks.workunit.client.1.smithi149.stdout:4/574: chown d2/d18/d1e/f2c 31241479 1 2024-04-24T08:24:53.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/253: symlink d2/da/d13/d15/d24/l56 0 2024-04-24T08:24:53.635 INFO:tasks.workunit.client.0.smithi090.stdout:7/349: rename db/d17/d29/d32/f73 to db/d17/d29/d32/d5a/d75/f7f 0 2024-04-24T08:24:53.637 INFO:tasks.workunit.client.1.smithi149.stdout:4/575: symlink d2/d18/d1e/d3c/d48/d69/lb5 0 2024-04-24T08:24:53.638 INFO:tasks.workunit.client.0.smithi090.stdout:8/254: creat d2/da/d13/d15/d50/f57 x:0 0 0 2024-04-24T08:24:53.639 INFO:tasks.workunit.client.0.smithi090.stdout:6/310: rmdir d8/d1d/d2f/d30/d38/d59 0 2024-04-24T08:24:53.640 INFO:tasks.workunit.client.0.smithi090.stdout:8/255: write d2/da/d13/f18 [498774,29743] 0 2024-04-24T08:24:53.645 INFO:tasks.workunit.client.0.smithi090.stdout:6/311: fdatasync d8/d1d/f1e 0 2024-04-24T08:24:53.655 INFO:tasks.workunit.client.0.smithi090.stdout:6/312: link d8/d1d/d29/f2c d8/d1d/d1f/d3e/f5c 0 2024-04-24T08:24:53.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/350: dwrite db/d17/f3a [0,4194304] 0 2024-04-24T08:24:53.658 INFO:tasks.workunit.client.1.smithi149.stdout:4/576: dwrite d2/fc [0,4194304] 0 2024-04-24T08:24:53.659 INFO:tasks.workunit.client.0.smithi090.stdout:7/351: write db/d17/d29/f69 [309520,71085] 0 2024-04-24T08:24:53.662 INFO:tasks.workunit.client.1.smithi149.stdout:4/577: write d2/d10/d1a/f6f [445642,54242] 0 2024-04-24T08:24:53.664 INFO:tasks.workunit.client.0.smithi090.stdout:6/313: creat d8/d1d/d1f/f5d x:0 0 0 2024-04-24T08:24:53.664 INFO:tasks.workunit.client.1.smithi149.stdout:4/578: chown d2/d24/d2e/f98 1 1 2024-04-24T08:24:53.666 INFO:tasks.workunit.client.0.smithi090.stdout:7/352: mknod db/c80 0 2024-04-24T08:24:53.674 INFO:tasks.workunit.client.0.smithi090.stdout:6/314: truncate d8/dc/d19/f3d 311248 0 2024-04-24T08:24:53.674 INFO:tasks.workunit.client.0.smithi090.stdout:3/269: dread d2/f9 [0,4194304] 0 2024-04-24T08:24:53.678 INFO:tasks.workunit.client.1.smithi149.stdout:8/541: dread d1/d1f/d22/f5f [0,4194304] 0 2024-04-24T08:24:53.678 INFO:tasks.workunit.client.0.smithi090.stdout:7/353: creat db/d17/d29/d32/d5a/d75/f81 x:0 0 0 2024-04-24T08:24:53.680 INFO:tasks.workunit.client.1.smithi149.stdout:3/689: dread d0/d5/da/d19/d1b/d22/d38/f50 [4194304,4194304] 0 2024-04-24T08:24:53.683 INFO:tasks.workunit.client.0.smithi090.stdout:2/290: getdents d0/d14/d4d/d50 0 2024-04-24T08:24:53.691 INFO:tasks.workunit.client.0.smithi090.stdout:3/270: rename d2/d15/f52 to d2/df/d1a/d37/f55 0 2024-04-24T08:24:53.693 INFO:tasks.workunit.client.0.smithi090.stdout:6/315: mknod d8/c5e 0 2024-04-24T08:24:53.695 INFO:tasks.workunit.client.1.smithi149.stdout:0/568: dwrite d2/d9/df/d2f/d6c/f75 [0,4194304] 0 2024-04-24T08:24:53.696 INFO:tasks.workunit.client.0.smithi090.stdout:8/256: dread d2/f54 [0,4194304] 0 2024-04-24T08:24:53.697 INFO:tasks.workunit.client.0.smithi090.stdout:8/257: read d2/f54 [374649,110853] 0 2024-04-24T08:24:53.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/258: fsync d2/da/d13/d15/f4e 0 2024-04-24T08:24:53.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/579: link d2/d18/f88 d2/d18/d1e/d3c/d48/d69/fb6 0 2024-04-24T08:24:53.699 INFO:tasks.workunit.client.1.smithi149.stdout:3/690: creat d0/d5/da/d5a/d9c/fef x:0 0 0 2024-04-24T08:24:53.701 INFO:tasks.workunit.client.0.smithi090.stdout:8/259: chown d2/da/d13/l1d 110184339 1 2024-04-24T08:24:53.703 INFO:tasks.workunit.client.0.smithi090.stdout:8/260: write d2/f12 [296628,118271] 0 2024-04-24T08:24:53.703 INFO:tasks.workunit.client.1.smithi149.stdout:8/542: rename d1/de/d18/d32/d63/d9e/fa1 to d1/de/d18/d32/d63/d9e/fba 0 2024-04-24T08:24:53.704 INFO:tasks.workunit.client.0.smithi090.stdout:3/271: dread d2/d15/f16 [0,4194304] 0 2024-04-24T08:24:53.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/272: chown d2/df/d1a 66684 1 2024-04-24T08:24:53.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/261: chown d2/da/d13/l25 8202603 1 2024-04-24T08:24:53.708 INFO:tasks.workunit.client.1.smithi149.stdout:0/569: readlink d2/d9/d1a/d36/d85/l9f 0 2024-04-24T08:24:53.708 INFO:tasks.workunit.client.1.smithi149.stdout:8/543: write d1/d1f/f68 [2645137,116217] 0 2024-04-24T08:24:53.710 INFO:tasks.workunit.client.0.smithi090.stdout:8/262: read - d2/da/f38 zero size 2024-04-24T08:24:53.711 INFO:tasks.workunit.client.1.smithi149.stdout:4/580: symlink d2/d18/da4/lb7 0 2024-04-24T08:24:53.713 INFO:tasks.workunit.client.0.smithi090.stdout:7/354: dwrite db/d17/d29/d38/f4e [0,4194304] 0 2024-04-24T08:24:53.713 INFO:tasks.workunit.client.0.smithi090.stdout:3/273: mknod d2/d15/d36/c56 0 2024-04-24T08:24:53.713 INFO:tasks.workunit.client.0.smithi090.stdout:8/263: chown d2/da/d13/d34 3 1 2024-04-24T08:24:53.715 INFO:tasks.workunit.client.0.smithi090.stdout:3/274: readlink d2/ld 0 2024-04-24T08:24:53.715 INFO:tasks.workunit.client.1.smithi149.stdout:0/570: dread - d2/d9/df/d40/d4a/d4e/d6f/d9e/fac zero size 2024-04-24T08:24:53.716 INFO:tasks.workunit.client.1.smithi149.stdout:8/544: chown d1/de/d18/l60 15270576 1 2024-04-24T08:24:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:1/741: dwrite d0/da/f3f [0,4194304] 0 2024-04-24T08:24:53.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/356: truncate d8/f9 7616518 0 2024-04-24T08:24:53.719 INFO:tasks.workunit.client.0.smithi090.stdout:6/316: dwrite d8/dc/d19/f3d [0,4194304] 0 2024-04-24T08:24:53.720 INFO:tasks.workunit.client.0.smithi090.stdout:6/317: read - d8/f46 zero size 2024-04-24T08:24:53.720 INFO:tasks.workunit.client.1.smithi149.stdout:4/581: unlink d2/d24/d2e/d38/f71 0 2024-04-24T08:24:53.721 INFO:tasks.workunit.client.1.smithi149.stdout:0/571: write d2/d3c/d57/f78 [4655955,15748] 0 2024-04-24T08:24:53.724 INFO:tasks.workunit.client.1.smithi149.stdout:0/572: stat d2/d9/df/d40/fb4 0 2024-04-24T08:24:53.724 INFO:tasks.workunit.client.0.smithi090.stdout:6/318: chown d8/l35 154 1 2024-04-24T08:24:53.726 INFO:tasks.workunit.client.1.smithi149.stdout:3/691: dwrite d0/d26/fee [0,4194304] 0 2024-04-24T08:24:53.728 INFO:tasks.workunit.client.0.smithi090.stdout:7/355: symlink db/d17/d29/d32/l82 0 2024-04-24T08:24:53.732 INFO:tasks.workunit.client.1.smithi149.stdout:3/692: truncate d0/d5/da/d5e/fe8 855368 0 2024-04-24T08:24:53.742 INFO:tasks.workunit.client.1.smithi149.stdout:2/549: truncate d0/d6/db/d39/d3d/d44/f61 1472526 0 2024-04-24T08:24:53.748 INFO:tasks.workunit.client.0.smithi090.stdout:3/275: getdents d2/df/d1a/d37/d53 0 2024-04-24T08:24:53.750 INFO:tasks.workunit.client.0.smithi090.stdout:3/276: write d2/d25/f48 [1339703,122228] 0 2024-04-24T08:24:53.754 INFO:tasks.workunit.client.1.smithi149.stdout:0/573: dwrite d2/d9/df/d40/d4a/d4e/d6f/d9e/fad [0,4194304] 0 2024-04-24T08:24:53.755 INFO:tasks.workunit.client.0.smithi090.stdout:0/334: write d1a/f3b [333994,87701] 0 2024-04-24T08:24:53.756 INFO:tasks.workunit.client.1.smithi149.stdout:6/544: dwrite d0/f73 [0,4194304] 0 2024-04-24T08:24:53.762 INFO:tasks.workunit.client.1.smithi149.stdout:1/742: dwrite d0/fbd [0,4194304] 0 2024-04-24T08:24:53.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/335: read d1a/f2c [2340440,122646] 0 2024-04-24T08:24:53.767 INFO:tasks.workunit.client.0.smithi090.stdout:4/211: truncate d3/d24/f2b 3753862 0 2024-04-24T08:24:53.771 INFO:tasks.workunit.client.0.smithi090.stdout:3/277: dwrite d2/df/d1a/d37/f49 [0,4194304] 0 2024-04-24T08:24:53.782 INFO:tasks.workunit.client.1.smithi149.stdout:3/693: mknod d0/d5/d53/d4c/cf0 0 2024-04-24T08:24:53.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/725: write db/d12/d2f/d95/fe5 [98860,125759] 0 2024-04-24T08:24:53.784 INFO:tasks.workunit.client.0.smithi090.stdout:9/291: write d0/fb [1523327,100347] 0 2024-04-24T08:24:53.787 INFO:tasks.workunit.client.1.smithi149.stdout:0/574: rmdir d2/d17/d41 39 2024-04-24T08:24:53.789 INFO:tasks.workunit.client.1.smithi149.stdout:0/575: chown d2/f7f 115 1 2024-04-24T08:24:53.793 INFO:tasks.workunit.client.1.smithi149.stdout:1/743: dwrite d0/d5/db/d18/fc1 [0,4194304] 0 2024-04-24T08:24:53.793 INFO:tasks.workunit.client.0.smithi090.stdout:0/336: chown d1a/f20 664427075 1 2024-04-24T08:24:53.794 INFO:tasks.workunit.client.1.smithi149.stdout:6/545: rmdir d0/d1a/d28/d31/d4c/d80/d6f 39 2024-04-24T08:24:53.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/337: read - d1a/d4b/d53/f5d zero size 2024-04-24T08:24:53.796 INFO:tasks.workunit.client.0.smithi090.stdout:0/338: dread - d1a/f64 zero size 2024-04-24T08:24:53.799 INFO:tasks.workunit.client.0.smithi090.stdout:3/278: dwrite d2/df/d1a/f4c [0,4194304] 0 2024-04-24T08:24:53.801 INFO:tasks.workunit.client.1.smithi149.stdout:4/582: link d2/d18/d1e/d3b/f79 d2/d18/d1e/d3c/d48/d69/d7d/d8d/fb8 0 2024-04-24T08:24:53.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/304: sync 2024-04-24T08:24:53.805 INFO:tasks.workunit.client.1.smithi149.stdout:3/694: creat d0/d5/d53/d34/d3a/d93/ff1 x:0 0 0 2024-04-24T08:24:53.805 INFO:tasks.workunit.client.0.smithi090.stdout:1/305: write d3/d5/f17 [3408027,19939] 0 2024-04-24T08:24:53.810 INFO:tasks.workunit.client.0.smithi090.stdout:0/339: fsync d1a/f20 0 2024-04-24T08:24:53.811 INFO:tasks.workunit.client.1.smithi149.stdout:5/726: unlink db/d12/d60/db2/f6e 0 2024-04-24T08:24:53.811 INFO:tasks.workunit.client.1.smithi149.stdout:5/727: chown db 5 1 2024-04-24T08:24:53.812 INFO:tasks.workunit.client.1.smithi149.stdout:5/728: chown db/d12/d1e 41263 1 2024-04-24T08:24:53.813 INFO:tasks.workunit.client.0.smithi090.stdout:0/340: write d1a/d4b/d53/f5d [55693,70594] 0 2024-04-24T08:24:53.814 INFO:tasks.workunit.client.1.smithi149.stdout:0/576: mkdir d2/d9/d1a/d36/d85/db5 0 2024-04-24T08:24:53.815 INFO:tasks.workunit.client.0.smithi090.stdout:4/212: rmdir d3/d18/d1d 39 2024-04-24T08:24:53.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/341: write d1a/d2a/d35/d62/f6e [438709,35842] 0 2024-04-24T08:24:53.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/744: creat d0/d1b/d4b/da1/ff5 x:0 0 0 2024-04-24T08:24:53.819 INFO:tasks.workunit.client.0.smithi090.stdout:0/342: write f8 [1413475,122667] 0 2024-04-24T08:24:53.820 INFO:tasks.workunit.client.0.smithi090.stdout:9/292: symlink d0/l54 0 2024-04-24T08:24:53.820 INFO:tasks.workunit.client.0.smithi090.stdout:6/319: dread d8/d1d/d29/f2c [0,4194304] 0 2024-04-24T08:24:53.820 INFO:tasks.workunit.client.1.smithi149.stdout:1/745: read d0/d5/db/d18/fc1 [3670486,36330] 0 2024-04-24T08:24:53.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/291: truncate d0/d14/f1d 3898212 0 2024-04-24T08:24:53.824 INFO:tasks.workunit.client.1.smithi149.stdout:4/583: fsync d2/d18/d1e/d3c/d48/d69/f70 0 2024-04-24T08:24:53.830 INFO:tasks.workunit.client.0.smithi090.stdout:1/306: dwrite d3/f41 [0,4194304] 0 2024-04-24T08:24:53.834 INFO:tasks.workunit.client.1.smithi149.stdout:5/729: creat db/da4/ff1 x:0 0 0 2024-04-24T08:24:53.840 INFO:tasks.workunit.client.1.smithi149.stdout:7/612: sync 2024-04-24T08:24:53.842 INFO:tasks.workunit.client.1.smithi149.stdout:0/577: mkdir d2/d9/df/d40/d4a/d4e/d6f/db6 0 2024-04-24T08:24:53.842 INFO:tasks.workunit.client.1.smithi149.stdout:3/695: dwrite d0/d5/da/f28 [0,4194304] 0 2024-04-24T08:24:53.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/357: dwrite d8/d56/d6d/d7a/d50/f71 [0,4194304] 0 2024-04-24T08:24:53.847 INFO:tasks.workunit.client.1.smithi149.stdout:0/578: chown d2/d9/df/d56/f5f 231554 1 2024-04-24T08:24:53.848 INFO:tasks.workunit.client.1.smithi149.stdout:3/696: read - d0/d88/fbe zero size 2024-04-24T08:24:53.849 INFO:tasks.workunit.client.1.smithi149.stdout:6/546: link d0/f1 d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fb6 0 2024-04-24T08:24:53.850 INFO:tasks.workunit.client.1.smithi149.stdout:8/545: write d1/d1f/f28 [1728814,52187] 0 2024-04-24T08:24:53.850 INFO:tasks.workunit.client.0.smithi090.stdout:5/358: dread - d8/d56/d6d/d7a/f4e zero size 2024-04-24T08:24:53.851 INFO:tasks.workunit.client.1.smithi149.stdout:1/746: truncate d0/d5/f41 10643166 0 2024-04-24T08:24:53.858 INFO:tasks.workunit.client.0.smithi090.stdout:4/213: dread d3/d18/d1d/f27 [0,4194304] 0 2024-04-24T08:24:53.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/584: mkdir d2/d18/db9 0 2024-04-24T08:24:53.863 INFO:tasks.workunit.client.1.smithi149.stdout:5/730: creat db/d12/d1e/d85/ff2 x:0 0 0 2024-04-24T08:24:53.865 INFO:tasks.workunit.client.1.smithi149.stdout:7/613: readlink d9/dc/d11/l65 0 2024-04-24T08:24:53.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/614: fdatasync d9/dc/d7b/f7d 0 2024-04-24T08:24:53.869 INFO:tasks.workunit.client.0.smithi090.stdout:8/264: dwrite d2/f22 [0,4194304] 0 2024-04-24T08:24:53.869 INFO:tasks.workunit.client.1.smithi149.stdout:3/697: dwrite d0/d5/d73/fad [0,4194304] 0 2024-04-24T08:24:53.873 INFO:tasks.workunit.client.1.smithi149.stdout:2/550: dwrite d0/d6/db/d27/f37 [0,4194304] 0 2024-04-24T08:24:53.875 INFO:tasks.workunit.client.0.smithi090.stdout:9/293: dwrite d0/f53 [0,4194304] 0 2024-04-24T08:24:53.880 INFO:tasks.workunit.client.1.smithi149.stdout:8/546: creat d1/d1f/d31/fbb x:0 0 0 2024-04-24T08:24:53.881 INFO:tasks.workunit.client.0.smithi090.stdout:7/356: dwrite db/d17/d39/d59/f6d [0,4194304] 0 2024-04-24T08:24:53.895 INFO:tasks.workunit.client.0.smithi090.stdout:4/214: mkdir d3/d38 0 2024-04-24T08:24:53.898 INFO:tasks.workunit.client.1.smithi149.stdout:7/615: fdatasync d9/dc/d11/d38/f56 0 2024-04-24T08:24:53.900 INFO:tasks.workunit.client.0.smithi090.stdout:3/279: getdents d2 0 2024-04-24T08:24:53.903 INFO:tasks.workunit.client.1.smithi149.stdout:4/585: dread d2/d18/d1e/f2c [0,4194304] 0 2024-04-24T08:24:53.906 INFO:tasks.workunit.client.1.smithi149.stdout:3/698: mkdir d0/d88/df2 0 2024-04-24T08:24:53.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/294: dwrite d0/f1b [0,4194304] 0 2024-04-24T08:24:53.908 INFO:tasks.workunit.client.0.smithi090.stdout:8/265: rmdir d2/da/d13/d34 39 2024-04-24T08:24:53.909 INFO:tasks.workunit.client.1.smithi149.stdout:2/551: fsync d0/d6/db/d27/d64/f6e 0 2024-04-24T08:24:53.912 INFO:tasks.workunit.client.0.smithi090.stdout:9/295: write d0/d19/f1a [967592,122667] 0 2024-04-24T08:24:53.912 INFO:tasks.workunit.client.1.smithi149.stdout:2/552: write d0/d6/db/d39/f90 [270557,120840] 0 2024-04-24T08:24:53.912 INFO:tasks.workunit.client.1.smithi149.stdout:9/656: sync 2024-04-24T08:24:53.913 INFO:tasks.workunit.client.1.smithi149.stdout:8/547: mknod d1/de/d18/daf/cbc 0 2024-04-24T08:24:53.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/548: fsync d1/de/d18/d32/d63/f85 0 2024-04-24T08:24:53.917 INFO:tasks.workunit.client.0.smithi090.stdout:0/343: rename d1a/d2a/f27 to d1a/f71 0 2024-04-24T08:24:53.917 INFO:tasks.workunit.client.1.smithi149.stdout:1/747: symlink d0/d1b/d5d/de1/dd3/lf6 0 2024-04-24T08:24:53.920 INFO:tasks.workunit.client.0.smithi090.stdout:7/357: dwrite db/d17/d29/d38/f53 [0,4194304] 0 2024-04-24T08:24:53.926 INFO:tasks.workunit.client.0.smithi090.stdout:2/292: link d0/d7/l2f d0/d7/d24/l5b 0 2024-04-24T08:24:53.927 INFO:tasks.workunit.client.1.smithi149.stdout:0/579: fsync d2/d17/f24 0 2024-04-24T08:24:53.929 INFO:tasks.workunit.client.1.smithi149.stdout:0/580: stat d2/d9/df/d40/l86 0 2024-04-24T08:24:53.932 INFO:tasks.workunit.client.1.smithi149.stdout:3/699: write d0/d5/da/d59/fbf [108605,120240] 0 2024-04-24T08:24:53.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/553: readlink d0/d6/db/l6a 0 2024-04-24T08:24:53.938 INFO:tasks.workunit.client.1.smithi149.stdout:2/554: dread - d0/d6/db/d39/d3d/d44/f96 zero size 2024-04-24T08:24:53.939 INFO:tasks.workunit.client.0.smithi090.stdout:3/280: dwrite d2/df/d1a/f2b [4194304,4194304] 0 2024-04-24T08:24:53.941 INFO:tasks.workunit.client.1.smithi149.stdout:2/555: stat d0/d6/db/d39/f90 0 2024-04-24T08:24:53.942 INFO:tasks.workunit.client.0.smithi090.stdout:6/320: getdents d8/d1d/d2f 0 2024-04-24T08:24:53.942 INFO:tasks.workunit.client.1.smithi149.stdout:8/549: rename d1/de/d18/l59 to d1/lbd 0 2024-04-24T08:24:53.943 INFO:tasks.workunit.client.1.smithi149.stdout:2/556: chown d0/d6c 0 1 2024-04-24T08:24:53.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/281: write d2/df/d21/f3e [3200458,76664] 0 2024-04-24T08:24:53.945 INFO:tasks.workunit.client.0.smithi090.stdout:0/344: rmdir d1a/d2a/d35 39 2024-04-24T08:24:53.948 INFO:tasks.workunit.client.0.smithi090.stdout:5/359: dwrite d8/d25/f3d [0,4194304] 0 2024-04-24T08:24:53.950 INFO:tasks.workunit.client.1.smithi149.stdout:7/616: getdents d9/dcd 0 2024-04-24T08:24:53.954 INFO:tasks.workunit.client.0.smithi090.stdout:4/215: link d3/f16 d3/d38/f39 0 2024-04-24T08:24:53.954 INFO:tasks.workunit.client.0.smithi090.stdout:5/360: dread - d8/da/d14/d2f/f66 zero size 2024-04-24T08:24:53.954 INFO:tasks.workunit.client.0.smithi090.stdout:2/293: write d0/d7/fd [5904856,38831] 0 2024-04-24T08:24:53.956 INFO:tasks.workunit.client.0.smithi090.stdout:7/358: mkdir db/d54/d83 0 2024-04-24T08:24:53.957 INFO:tasks.workunit.client.1.smithi149.stdout:0/581: creat d2/d9/df/d48/fb7 x:0 0 0 2024-04-24T08:24:53.959 INFO:tasks.workunit.client.0.smithi090.stdout:8/266: symlink d2/da/d13/d34/l58 0 2024-04-24T08:24:53.960 INFO:tasks.workunit.client.1.smithi149.stdout:9/657: getdents d1/da/dd/d55/d72 0 2024-04-24T08:24:53.963 INFO:tasks.workunit.client.0.smithi090.stdout:8/267: write d2/da/d13/f3a [1705476,99692] 0 2024-04-24T08:24:53.966 INFO:tasks.workunit.client.1.smithi149.stdout:6/547: truncate d0/d1a/f29 3320977 0 2024-04-24T08:24:53.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/321: creat d8/dc/f5f x:0 0 0 2024-04-24T08:24:53.968 INFO:tasks.workunit.client.1.smithi149.stdout:3/700: rename d0/d26/c7d to d0/d26/d3c/cf3 0 2024-04-24T08:24:53.969 INFO:tasks.workunit.client.1.smithi149.stdout:4/586: dread d2/d24/d2e/f66 [0,4194304] 0 2024-04-24T08:24:53.972 INFO:tasks.workunit.client.0.smithi090.stdout:1/307: rename d3/d25/d30/l49 to d3/d5/d3e/l6f 0 2024-04-24T08:24:53.972 INFO:tasks.workunit.client.1.smithi149.stdout:2/557: truncate d0/d6/db/f42 823439 0 2024-04-24T08:24:53.974 INFO:tasks.workunit.client.1.smithi149.stdout:5/731: getdents db/d12/d2f/d95 0 2024-04-24T08:24:53.975 INFO:tasks.workunit.client.0.smithi090.stdout:0/345: chown d1a/d2a/d35/f3e 13102 1 2024-04-24T08:24:53.976 INFO:tasks.workunit.client.1.smithi149.stdout:5/732: write db/d12/d22/d45/f21 [4076891,11068] 0 2024-04-24T08:24:53.977 INFO:tasks.workunit.client.0.smithi090.stdout:4/216: creat d3/d25/f3a x:0 0 0 2024-04-24T08:24:53.979 INFO:tasks.workunit.client.1.smithi149.stdout:7/617: creat d9/dc/d11/d83/da6/d67/d69/fcf x:0 0 0 2024-04-24T08:24:53.979 INFO:tasks.workunit.client.0.smithi090.stdout:4/217: chown d3/d18/d1d 7673 1 2024-04-24T08:24:53.979 INFO:tasks.workunit.client.0.smithi090.stdout:1/308: write d3/d1d/d34/d5a/f5d [1085846,103834] 0 2024-04-24T08:24:53.981 INFO:tasks.workunit.client.1.smithi149.stdout:7/618: stat d9/dc/d11/d38/dac/lbc 0 2024-04-24T08:24:53.983 INFO:tasks.workunit.client.1.smithi149.stdout:6/548: mkdir d0/db7 0 2024-04-24T08:24:53.985 INFO:tasks.workunit.client.0.smithi090.stdout:2/294: dwrite d0/d14/f15 [0,4194304] 0 2024-04-24T08:24:53.987 INFO:tasks.workunit.client.1.smithi149.stdout:3/701: mknod d0/d5/d53/d34/cf4 0 2024-04-24T08:24:53.990 INFO:tasks.workunit.client.1.smithi149.stdout:4/587: creat d2/d10/d1a/fba x:0 0 0 2024-04-24T08:24:53.994 INFO:tasks.workunit.client.0.smithi090.stdout:2/295: fdatasync d0/d7/fd 0 2024-04-24T08:24:53.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/346: rmdir d1a/d4b/d53 39 2024-04-24T08:24:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:4/588: chown d2/d18/d1e/d3c/d48/d69/lb5 597915093 1 2024-04-24T08:24:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:1/748: dwrite d0/d5/d4f/f77 [0,4194304] 0 2024-04-24T08:24:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:7/619: dread d9/dc/de/d44/f74 [0,4194304] 0 2024-04-24T08:24:53.996 INFO:tasks.workunit.client.0.smithi090.stdout:7/359: mknod db/d54/d83/c84 0 2024-04-24T08:24:53.999 INFO:tasks.workunit.client.1.smithi149.stdout:7/620: stat d9/d43/d4f/l85 0 2024-04-24T08:24:53.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/749: chown d0/ddd/fa2 21234 1 2024-04-24T08:24:53.999 INFO:tasks.workunit.client.1.smithi149.stdout:4/589: chown d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf 1 1 2024-04-24T08:24:54.000 INFO:tasks.workunit.client.0.smithi090.stdout:1/309: rmdir d3/d1d/d54 39 2024-04-24T08:24:54.003 INFO:tasks.workunit.client.1.smithi149.stdout:0/582: symlink d2/d9/lb8 0 2024-04-24T08:24:54.005 INFO:tasks.workunit.client.1.smithi149.stdout:9/658: mkdir d1/da/de8 0 2024-04-24T08:24:54.006 INFO:tasks.workunit.client.0.smithi090.stdout:9/296: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:24:54.009 INFO:tasks.workunit.client.0.smithi090.stdout:1/310: stat d3/f41 0 2024-04-24T08:24:54.014 INFO:tasks.workunit.client.1.smithi149.stdout:3/702: dwrite d0/d5/d73/fad [0,4194304] 0 2024-04-24T08:24:54.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/361: dread d8/da/d14/d2f/d44/f5b [0,4194304] 0 2024-04-24T08:24:54.018 INFO:tasks.workunit.client.0.smithi090.stdout:4/218: dwrite d3/d18/d1d/f1f [0,4194304] 0 2024-04-24T08:24:54.024 INFO:tasks.workunit.client.0.smithi090.stdout:8/268: creat d2/f59 x:0 0 0 2024-04-24T08:24:54.026 INFO:tasks.workunit.client.1.smithi149.stdout:2/558: write d0/f70 [948996,2448] 0 2024-04-24T08:24:54.032 INFO:tasks.workunit.client.0.smithi090.stdout:3/282: rename d2/d3a to d2/df/d57 0 2024-04-24T08:24:54.033 INFO:tasks.workunit.client.1.smithi149.stdout:7/621: creat d9/d43/d95/fd0 x:0 0 0 2024-04-24T08:24:54.037 INFO:tasks.workunit.client.1.smithi149.stdout:4/590: symlink d2/d24/lbb 0 2024-04-24T08:24:54.038 INFO:tasks.workunit.client.0.smithi090.stdout:2/296: creat d0/d14/d4d/d50/f5c x:0 0 0 2024-04-24T08:24:54.038 INFO:tasks.workunit.client.1.smithi149.stdout:7/622: chown d9/dc/d11/d83/da6/d67/d69/dad/fc6 12834 1 2024-04-24T08:24:54.040 INFO:tasks.workunit.client.1.smithi149.stdout:5/733: creat db/d12/d60/db2/de2/ff3 x:0 0 0 2024-04-24T08:24:54.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/311: creat d3/d5/da/f70 x:0 0 0 2024-04-24T08:24:54.041 INFO:tasks.workunit.client.0.smithi090.stdout:2/297: fdatasync d0/d7/d24/f29 0 2024-04-24T08:24:54.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/623: truncate d9/dc/d11/d26/f9b 486136 0 2024-04-24T08:24:54.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/347: symlink d1a/d2a/d35/d68/l72 0 2024-04-24T08:24:54.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/312: readlink d3/d5/da/dc/l20 0 2024-04-24T08:24:54.045 INFO:tasks.workunit.client.0.smithi090.stdout:4/219: symlink d3/d18/d1d/l3b 0 2024-04-24T08:24:54.046 INFO:tasks.workunit.client.1.smithi149.stdout:8/550: mknod d1/d1f/d25/cbe 0 2024-04-24T08:24:54.047 INFO:tasks.workunit.client.1.smithi149.stdout:6/549: mkdir d0/d1a/d28/d31/d4c/d4e/db8 0 2024-04-24T08:24:54.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/703: mkdir d0/d5/da/d5a/d63/d6a/df5 0 2024-04-24T08:24:54.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/322: dwrite d8/d1d/d29/f2c [0,4194304] 0 2024-04-24T08:24:54.050 INFO:tasks.workunit.client.0.smithi090.stdout:5/362: mknod d8/da/c7f 0 2024-04-24T08:24:54.053 INFO:tasks.workunit.client.1.smithi149.stdout:3/704: chown d0/d5/da/d19/d1b/d22/d38 127734 1 2024-04-24T08:24:54.056 INFO:tasks.workunit.client.0.smithi090.stdout:1/313: creat d3/d5/da/dc/f71 x:0 0 0 2024-04-24T08:24:54.061 INFO:tasks.workunit.client.0.smithi090.stdout:8/269: mknod d2/da/d13/d15/c5a 0 2024-04-24T08:24:54.064 INFO:tasks.workunit.client.0.smithi090.stdout:6/323: creat d8/d1d/d1f/d3e/f60 x:0 0 0 2024-04-24T08:24:54.064 INFO:tasks.workunit.client.1.smithi149.stdout:5/734: dwrite db/d12/d60/db2/f6d [0,4194304] 0 2024-04-24T08:24:54.068 INFO:tasks.workunit.client.0.smithi090.stdout:4/220: mknod d3/d38/c3c 0 2024-04-24T08:24:54.069 INFO:tasks.workunit.client.1.smithi149.stdout:8/551: write d1/de/d18/fae [10434,106852] 0 2024-04-24T08:24:54.069 INFO:tasks.workunit.client.0.smithi090.stdout:9/297: write d0/d19/f28 [8450789,109645] 0 2024-04-24T08:24:54.072 INFO:tasks.workunit.client.1.smithi149.stdout:4/591: symlink d2/d24/da7/lbc 0 2024-04-24T08:24:54.079 INFO:tasks.workunit.client.1.smithi149.stdout:3/705: dread d0/d5/d53/d39/f8a [0,4194304] 0 2024-04-24T08:24:54.082 INFO:tasks.workunit.client.1.smithi149.stdout:9/659: mknod d1/d2/d15/d1c/ce9 0 2024-04-24T08:24:54.086 INFO:tasks.workunit.client.1.smithi149.stdout:8/552: write d1/d1f/d5e/f73 [2858050,28851] 0 2024-04-24T08:24:54.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/363: dwrite d8/da/f35 [4194304,4194304] 0 2024-04-24T08:24:54.089 INFO:tasks.workunit.client.1.smithi149.stdout:1/750: truncate d0/d5/f8 2603073 0 2024-04-24T08:24:54.089 INFO:tasks.workunit.client.1.smithi149.stdout:2/559: write d0/d6/db/d39/f4e [3698318,74267] 0 2024-04-24T08:24:54.089 INFO:tasks.workunit.client.0.smithi090.stdout:8/270: creat d2/da/d13/d15/d24/f5b x:0 0 0 2024-04-24T08:24:54.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/324: rename d8/d1d/f5a to d8/d1d/d29/f61 0 2024-04-24T08:24:54.090 INFO:tasks.workunit.client.0.smithi090.stdout:3/283: chown d2/df/l2d 93 1 2024-04-24T08:24:54.090 INFO:tasks.workunit.client.1.smithi149.stdout:0/583: link d2/d9/d1a/d36/d85/fb2 d2/d17/fb9 0 2024-04-24T08:24:54.091 INFO:tasks.workunit.client.0.smithi090.stdout:2/298: dwrite d0/d14/f58 [0,4194304] 0 2024-04-24T08:24:54.093 INFO:tasks.workunit.client.1.smithi149.stdout:1/751: write d0/ddd/fe9 [828521,84561] 0 2024-04-24T08:24:54.094 INFO:tasks.workunit.client.1.smithi149.stdout:7/624: write d9/dc/f6b [871203,36191] 0 2024-04-24T08:24:54.097 INFO:tasks.workunit.client.1.smithi149.stdout:6/550: link d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 d0/db7/fb9 0 2024-04-24T08:24:54.098 INFO:tasks.workunit.client.1.smithi149.stdout:6/551: stat d0/fc 0 2024-04-24T08:24:54.099 INFO:tasks.workunit.client.1.smithi149.stdout:1/752: truncate d0/d1b/d4b/da1/ff5 951870 0 2024-04-24T08:24:54.100 INFO:tasks.workunit.client.0.smithi090.stdout:1/314: rename d3/d1d/d34/d5a/f5d to d3/d1d/d54/f72 0 2024-04-24T08:24:54.102 INFO:tasks.workunit.client.1.smithi149.stdout:9/660: mkdir d1/d2/d33/dea 0 2024-04-24T08:24:54.107 INFO:tasks.workunit.client.0.smithi090.stdout:8/271: dread d2/da/f35 [0,4194304] 0 2024-04-24T08:24:54.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/364: dread - d8/d56/d6d/d7a/d50/f68 zero size 2024-04-24T08:24:54.108 INFO:tasks.workunit.client.0.smithi090.stdout:3/284: symlink d2/df/d21/l58 0 2024-04-24T08:24:54.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/298: dwrite d0/dc/f2c [0,4194304] 0 2024-04-24T08:24:54.110 INFO:tasks.workunit.client.1.smithi149.stdout:5/735: link db/d1f/d7f/fd0 db/d12/d60/db2/d83/ff4 0 2024-04-24T08:24:54.110 INFO:tasks.workunit.client.0.smithi090.stdout:3/285: write d2/df/d1a/f4c [521161,102872] 0 2024-04-24T08:24:54.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/360: dread db/f12 [0,4194304] 0 2024-04-24T08:24:54.112 INFO:tasks.workunit.client.1.smithi149.stdout:0/584: mkdir d2/d9/df/d48/dba 0 2024-04-24T08:24:54.114 INFO:tasks.workunit.client.1.smithi149.stdout:2/560: write d0/d6/db/d27/d64/f74 [2807845,8397] 0 2024-04-24T08:24:54.118 INFO:tasks.workunit.client.1.smithi149.stdout:1/753: creat d0/d5/d4f/ff7 x:0 0 0 2024-04-24T08:24:54.119 INFO:tasks.workunit.client.1.smithi149.stdout:9/661: fsync d1/d2/fa0 0 2024-04-24T08:24:54.121 INFO:tasks.workunit.client.1.smithi149.stdout:0/585: creat d2/d9/df/d2f/d6c/fbb x:0 0 0 2024-04-24T08:24:54.122 INFO:tasks.workunit.client.0.smithi090.stdout:1/315: dwrite d3/d5/da/f59 [0,4194304] 0 2024-04-24T08:24:54.123 INFO:tasks.workunit.client.1.smithi149.stdout:3/706: link d0/d5/f6f d0/d5/da/ff6 0 2024-04-24T08:24:54.124 INFO:tasks.workunit.client.0.smithi090.stdout:2/299: rmdir d0 39 2024-04-24T08:24:54.125 INFO:tasks.workunit.client.0.smithi090.stdout:0/348: dread d1a/f24 [4194304,4194304] 0 2024-04-24T08:24:54.125 INFO:tasks.workunit.client.0.smithi090.stdout:9/299: mknod d0/dc/d24/d2d/c55 0 2024-04-24T08:24:54.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/736: creat db/d12/d1e/d65/de8/ff5 x:0 0 0 2024-04-24T08:24:54.130 INFO:tasks.workunit.client.0.smithi090.stdout:5/365: write d8/da/d14/f17 [1950998,20012] 0 2024-04-24T08:24:54.131 INFO:tasks.workunit.client.1.smithi149.stdout:3/707: write d0/d5/da/d5a/d9c/fef [986092,115665] 0 2024-04-24T08:24:54.132 INFO:tasks.workunit.client.1.smithi149.stdout:5/737: chown db/d12/d22/ld9 2894 1 2024-04-24T08:24:54.133 INFO:tasks.workunit.client.1.smithi149.stdout:9/662: creat d1/da/dd/d79/db6/feb x:0 0 0 2024-04-24T08:24:54.136 INFO:tasks.workunit.client.1.smithi149.stdout:5/738: fdatasync db/da4/ff1 0 2024-04-24T08:24:54.138 INFO:tasks.workunit.client.1.smithi149.stdout:3/708: write d0/d5/d53/d4c/fc2 [1037596,37409] 0 2024-04-24T08:24:54.139 INFO:tasks.workunit.client.1.smithi149.stdout:8/553: rename d1/f64 to d1/fbf 0 2024-04-24T08:24:54.142 INFO:tasks.workunit.client.1.smithi149.stdout:9/663: truncate d1/d2/d15/d1c/dca/fe2 148227 0 2024-04-24T08:24:54.151 INFO:tasks.workunit.client.0.smithi090.stdout:9/300: creat d0/d19/f56 x:0 0 0 2024-04-24T08:24:54.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/709: mknod d0/d5/da/d5a/d9c/cf7 0 2024-04-24T08:24:54.157 INFO:tasks.workunit.client.1.smithi149.stdout:5/739: rename db/d12/d60/d8f/da0 to db/d12/d2f/d74/df6 0 2024-04-24T08:24:54.163 INFO:tasks.workunit.client.1.smithi149.stdout:0/586: creat d2/d9/d1a/fbc x:0 0 0 2024-04-24T08:24:54.167 INFO:tasks.workunit.client.0.smithi090.stdout:2/300: creat d0/d14/d17/d2a/f5d x:0 0 0 2024-04-24T08:24:54.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/361: dread db/d17/d29/d32/f57 [0,4194304] 0 2024-04-24T08:24:54.169 INFO:tasks.workunit.client.0.smithi090.stdout:8/272: creat d2/da/d13/f5c x:0 0 0 2024-04-24T08:24:54.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/301: dread - d0/d2e/f37 zero size 2024-04-24T08:24:54.170 INFO:tasks.workunit.client.1.smithi149.stdout:9/664: mknod d1/d2/d15/de7/cec 0 2024-04-24T08:24:54.171 INFO:tasks.workunit.client.1.smithi149.stdout:0/587: mknod d2/d9/df/d92/cbd 0 2024-04-24T08:24:54.172 INFO:tasks.workunit.client.0.smithi090.stdout:6/325: dwrite d8/d1d/f1e [0,4194304] 0 2024-04-24T08:24:54.173 INFO:tasks.workunit.client.0.smithi090.stdout:9/301: creat d0/d19/d33/f57 x:0 0 0 2024-04-24T08:24:54.173 INFO:tasks.workunit.client.1.smithi149.stdout:6/552: write d0/d1a/f2e [1574559,78321] 0 2024-04-24T08:24:54.177 INFO:tasks.workunit.client.0.smithi090.stdout:1/316: rmdir d3/d1d/d4c 0 2024-04-24T08:24:54.180 INFO:tasks.workunit.client.1.smithi149.stdout:2/561: write d0/f55 [2330512,41185] 0 2024-04-24T08:24:54.184 INFO:tasks.workunit.client.0.smithi090.stdout:8/273: rmdir d2/da/d13/d15/d50 39 2024-04-24T08:24:54.185 INFO:tasks.workunit.client.1.smithi149.stdout:7/625: dwrite f1 [0,4194304] 0 2024-04-24T08:24:54.190 INFO:tasks.workunit.client.0.smithi090.stdout:7/362: dwrite db/d17/d39/f65 [0,4194304] 0 2024-04-24T08:24:54.190 INFO:tasks.workunit.client.1.smithi149.stdout:9/665: rename d1/da/d4a/fd3 to d1/da/d13/d67/de4/fed 0 2024-04-24T08:24:54.191 INFO:tasks.workunit.client.0.smithi090.stdout:3/286: dread d2/d15/f31 [0,4194304] 0 2024-04-24T08:24:54.196 INFO:tasks.workunit.client.0.smithi090.stdout:2/302: dwrite d0/d14/d17/d2a/d31/f41 [0,4194304] 0 2024-04-24T08:24:54.197 INFO:tasks.workunit.client.1.smithi149.stdout:6/553: rmdir d0/d1a/d28/d31/d4c/d4e/d62/d66 39 2024-04-24T08:24:54.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/317: dwrite d3/d5/f32 [0,4194304] 0 2024-04-24T08:24:54.205 INFO:tasks.workunit.client.1.smithi149.stdout:0/588: dwrite d2/d9/d69/fa7 [0,4194304] 0 2024-04-24T08:24:54.207 INFO:tasks.workunit.client.0.smithi090.stdout:0/349: getdents d1a 0 2024-04-24T08:24:54.208 INFO:tasks.workunit.client.0.smithi090.stdout:0/350: stat d1a/l45 0 2024-04-24T08:24:54.216 INFO:tasks.workunit.client.1.smithi149.stdout:5/740: getdents db/d23 0 2024-04-24T08:24:54.217 INFO:tasks.workunit.client.1.smithi149.stdout:7/626: creat d9/d43/d4f/fd1 x:0 0 0 2024-04-24T08:24:54.218 INFO:tasks.workunit.client.0.smithi090.stdout:5/366: write d8/d25/f26 [295350,129083] 0 2024-04-24T08:24:54.221 INFO:tasks.workunit.client.0.smithi090.stdout:5/367: write d8/da/f35 [3551513,54371] 0 2024-04-24T08:24:54.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/363: rmdir db/d54/d83 39 2024-04-24T08:24:54.229 INFO:tasks.workunit.client.1.smithi149.stdout:3/710: write d0/d5/d53/d34/d3a/d93/f77 [1159714,17526] 0 2024-04-24T08:24:54.229 INFO:tasks.workunit.client.1.smithi149.stdout:9/666: creat d1/da/d4a/fee x:0 0 0 2024-04-24T08:24:54.231 INFO:tasks.workunit.client.0.smithi090.stdout:3/287: mkdir d2/df/d1a/d37/d59 0 2024-04-24T08:24:54.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/667: chown d1/da/dd/d79/db6/feb 1443290987 1 2024-04-24T08:24:54.235 INFO:tasks.workunit.client.0.smithi090.stdout:1/318: unlink d3/d5/da/f70 0 2024-04-24T08:24:54.235 INFO:tasks.workunit.client.1.smithi149.stdout:9/668: truncate d1/da/dd/d79/db6/feb 877246 0 2024-04-24T08:24:54.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/326: dread d8/dc/d19/f34 [0,4194304] 0 2024-04-24T08:24:54.238 INFO:tasks.workunit.client.0.smithi090.stdout:0/351: mknod d1a/d2a/d35/c73 0 2024-04-24T08:24:54.239 INFO:tasks.workunit.client.1.smithi149.stdout:8/554: truncate d1/d1f/d31/f95 849438 0 2024-04-24T08:24:54.239 INFO:tasks.workunit.client.1.smithi149.stdout:2/562: write d0/f21 [43116,47813] 0 2024-04-24T08:24:54.240 INFO:tasks.workunit.client.1.smithi149.stdout:7/627: symlink d9/d43/d64/ld2 0 2024-04-24T08:24:54.242 INFO:tasks.workunit.client.1.smithi149.stdout:7/628: fsync d9/dc/de/d44/fbe 0 2024-04-24T08:24:54.243 INFO:tasks.workunit.client.1.smithi149.stdout:5/741: rename db/d12/d60/db2/d83/c89 to db/d12/d2f/d74/dc6/cf7 0 2024-04-24T08:24:54.245 INFO:tasks.workunit.client.1.smithi149.stdout:6/554: mkdir d0/d1a/d28/d31/d4c/d4e/dba 0 2024-04-24T08:24:54.250 INFO:tasks.workunit.client.0.smithi090.stdout:1/319: mkdir d3/d1d/d34/d73 0 2024-04-24T08:24:54.252 INFO:tasks.workunit.client.1.smithi149.stdout:8/555: fdatasync d1/de/d18/f45 0 2024-04-24T08:24:54.253 INFO:tasks.workunit.client.1.smithi149.stdout:2/563: rmdir d0/d6/db/d39/d3d/d44 39 2024-04-24T08:24:54.256 INFO:tasks.workunit.client.0.smithi090.stdout:9/302: truncate d0/fb 946268 0 2024-04-24T08:24:54.258 INFO:tasks.workunit.client.0.smithi090.stdout:9/303: write d0/d19/f1d [773907,44996] 0 2024-04-24T08:24:54.260 INFO:tasks.workunit.client.0.smithi090.stdout:9/304: chown d0/l54 1727 1 2024-04-24T08:24:54.260 INFO:tasks.workunit.client.1.smithi149.stdout:3/711: dwrite d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:54.261 INFO:tasks.workunit.client.0.smithi090.stdout:3/288: stat d2/d25/d2c/l41 0 2024-04-24T08:24:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:9/669: creat d1/d2/d33/dea/fef x:0 0 0 2024-04-24T08:24:54.263 INFO:tasks.workunit.client.0.smithi090.stdout:9/305: write d0/d19/d33/d4d/d32/f3c [824493,60037] 0 2024-04-24T08:24:54.265 INFO:tasks.workunit.client.1.smithi149.stdout:9/670: stat d1/d4c/d91/fd4 0 2024-04-24T08:24:54.268 INFO:tasks.workunit.client.1.smithi149.stdout:0/589: link d2/d9/df/d2f/d6c/l8e d2/d9/d1a/d36/d85/lbe 0 2024-04-24T08:24:54.269 INFO:tasks.workunit.client.0.smithi090.stdout:3/289: dread d2/d15/f1d [0,4194304] 0 2024-04-24T08:24:54.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/364: dwrite db/f34 [0,4194304] 0 2024-04-24T08:24:54.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/320: rmdir d3/d1d 39 2024-04-24T08:24:54.274 INFO:tasks.workunit.client.0.smithi090.stdout:5/368: truncate d8/da/fc 4121483 0 2024-04-24T08:24:54.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/556: creat d1/d1f/d31/fc0 x:0 0 0 2024-04-24T08:24:54.276 INFO:tasks.workunit.client.0.smithi090.stdout:6/327: getdents d8/d1d/d1f/d3e/d44 0 2024-04-24T08:24:54.279 INFO:tasks.workunit.client.0.smithi090.stdout:6/328: chown d8/d1d/d1f/c33 3708 1 2024-04-24T08:24:54.280 INFO:tasks.workunit.client.1.smithi149.stdout:6/555: fsync d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fb6 0 2024-04-24T08:24:54.282 INFO:tasks.workunit.client.1.smithi149.stdout:9/671: creat d1/d2/dc8/ff0 x:0 0 0 2024-04-24T08:24:54.283 INFO:tasks.workunit.client.1.smithi149.stdout:6/556: write d0/d1a/d28/d31/d33/fa8 [141004,82614] 0 2024-04-24T08:24:54.285 INFO:tasks.workunit.client.1.smithi149.stdout:2/564: mknod d0/d6/db/d39/d3d/ca9 0 2024-04-24T08:24:54.289 INFO:tasks.workunit.client.0.smithi090.stdout:7/365: dread db/fd [4194304,4194304] 0 2024-04-24T08:24:54.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/321: dread d3/f41 [0,4194304] 0 2024-04-24T08:24:54.294 INFO:tasks.workunit.client.0.smithi090.stdout:2/303: getdents d0/d7/d24 0 2024-04-24T08:24:54.295 INFO:tasks.workunit.client.0.smithi090.stdout:9/306: fdatasync d0/d19/d33/f36 0 2024-04-24T08:24:54.296 INFO:tasks.workunit.client.0.smithi090.stdout:3/290: dwrite d2/d25/f48 [0,4194304] 0 2024-04-24T08:24:54.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/307: dread - d0/d19/f56 zero size 2024-04-24T08:24:54.298 INFO:tasks.workunit.client.0.smithi090.stdout:3/291: readlink d2/df/d21/l58 0 2024-04-24T08:24:54.303 INFO:tasks.workunit.client.1.smithi149.stdout:7/629: getdents d9/dc/d11/d83/da6 0 2024-04-24T08:24:54.305 INFO:tasks.workunit.client.0.smithi090.stdout:0/352: link d1a/f2c d1a/d2a/d35/f74 0 2024-04-24T08:24:54.312 INFO:tasks.workunit.client.1.smithi149.stdout:3/712: getdents d0/d5/d1a/d70 0 2024-04-24T08:24:54.313 INFO:tasks.workunit.client.1.smithi149.stdout:5/742: dwrite db/d12/d22/d45/f42 [0,4194304] 0 2024-04-24T08:24:54.317 INFO:tasks.workunit.client.1.smithi149.stdout:6/557: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 [4194304,4194304] 0 2024-04-24T08:24:54.319 INFO:tasks.workunit.client.1.smithi149.stdout:6/558: readlink d0/d1a/d28/d4a/l85 0 2024-04-24T08:24:54.320 INFO:tasks.workunit.client.0.smithi090.stdout:5/369: creat d8/d56/d6d/d7b/f80 x:0 0 0 2024-04-24T08:24:54.321 INFO:tasks.workunit.client.0.smithi090.stdout:9/308: dwrite d0/d19/d33/d4d/f52 [0,4194304] 0 2024-04-24T08:24:54.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/292: symlink d2/df/d57/l5a 0 2024-04-24T08:24:54.322 INFO:tasks.workunit.client.1.smithi149.stdout:9/672: dread d1/da/f8 [0,4194304] 0 2024-04-24T08:24:54.323 INFO:tasks.workunit.client.0.smithi090.stdout:6/329: dwrite d8/d1d/d1f/f2e [0,4194304] 0 2024-04-24T08:24:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:5/743: symlink db/d12/d1e/d65/lf8 0 2024-04-24T08:24:54.329 INFO:tasks.workunit.client.0.smithi090.stdout:2/304: mkdir d0/d5e 0 2024-04-24T08:24:54.331 INFO:tasks.workunit.client.0.smithi090.stdout:2/305: fsync d0/f22 0 2024-04-24T08:24:54.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/306: stat d0/d14/d17/f18 0 2024-04-24T08:24:54.335 INFO:tasks.workunit.client.0.smithi090.stdout:1/322: rmdir d3/d1d/d34/d5a 39 2024-04-24T08:24:54.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/307: stat d0/d7/f11 0 2024-04-24T08:24:54.343 INFO:tasks.workunit.client.0.smithi090.stdout:5/370: chown d8/da/d14/d2f/l6e 3 1 2024-04-24T08:24:54.347 INFO:tasks.workunit.client.1.smithi149.stdout:6/559: symlink d0/d1a/d28/d31/d4c/d4e/lbb 0 2024-04-24T08:24:54.347 INFO:tasks.workunit.client.1.smithi149.stdout:5/744: rename db/d12/d2f/c8e to db/d23/daa/cf9 0 2024-04-24T08:24:54.348 INFO:tasks.workunit.client.1.smithi149.stdout:6/560: dread - d0/d1a/d28/d31/f82 zero size 2024-04-24T08:24:54.348 INFO:tasks.workunit.client.0.smithi090.stdout:6/330: creat d8/d1d/d2f/d30/f62 x:0 0 0 2024-04-24T08:24:54.354 INFO:tasks.workunit.client.0.smithi090.stdout:2/308: rmdir d0/d2e 39 2024-04-24T08:24:54.358 INFO:tasks.workunit.client.0.smithi090.stdout:2/309: write d0/d7/fd [4978692,8156] 0 2024-04-24T08:24:54.358 INFO:tasks.workunit.client.1.smithi149.stdout:2/565: write d0/d6/f12 [4050465,11192] 0 2024-04-24T08:24:54.361 INFO:tasks.workunit.client.0.smithi090.stdout:9/309: dread d0/d19/f1d [0,4194304] 0 2024-04-24T08:24:54.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/366: write db/d17/f2e [3153738,21768] 0 2024-04-24T08:24:54.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/371: unlink d8/da/c7f 0 2024-04-24T08:24:54.367 INFO:tasks.workunit.client.1.smithi149.stdout:9/673: dread d1/da/dd/fac [0,4194304] 0 2024-04-24T08:24:54.367 INFO:tasks.workunit.client.1.smithi149.stdout:8/557: dwrite d1/d1f/d22/f5f [0,4194304] 0 2024-04-24T08:24:54.368 INFO:tasks.workunit.client.0.smithi090.stdout:5/372: dread d8/da/d14/f28 [0,4194304] 0 2024-04-24T08:24:54.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/293: mkdir d2/df/d5b 0 2024-04-24T08:24:54.371 INFO:tasks.workunit.client.1.smithi149.stdout:7/630: truncate d9/dc/f35 2764788 0 2024-04-24T08:24:54.373 INFO:tasks.workunit.client.0.smithi090.stdout:6/331: dread d8/f1b [0,4194304] 0 2024-04-24T08:24:54.373 INFO:tasks.workunit.client.1.smithi149.stdout:3/713: dread d0/d88/f8e [0,4194304] 0 2024-04-24T08:24:54.375 INFO:tasks.workunit.client.0.smithi090.stdout:6/332: stat d8/d1d/l42 0 2024-04-24T08:24:54.375 INFO:tasks.workunit.client.0.smithi090.stdout:2/310: mknod d0/d14/d4d/d50/c5f 0 2024-04-24T08:24:54.378 INFO:tasks.workunit.client.0.smithi090.stdout:6/333: write d8/dc/d19/f34 [2453309,41767] 0 2024-04-24T08:24:54.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/311: mkdir d0/d14/d17/d2a/d31/d46/d60 0 2024-04-24T08:24:54.386 INFO:tasks.workunit.client.0.smithi090.stdout:0/353: dwrite d1a/f1b [0,4194304] 0 2024-04-24T08:24:54.386 INFO:tasks.workunit.client.1.smithi149.stdout:6/561: dread d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fb6 [0,4194304] 0 2024-04-24T08:24:54.388 INFO:tasks.workunit.client.1.smithi149.stdout:6/562: chown d0/l55 44 1 2024-04-24T08:24:54.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/294: rename d2/d15/d36/c45 to d2/df/d1a/c5c 0 2024-04-24T08:24:54.401 INFO:tasks.workunit.client.0.smithi090.stdout:7/367: creat db/d17/f85 x:0 0 0 2024-04-24T08:24:54.409 INFO:tasks.workunit.client.0.smithi090.stdout:1/323: truncate d3/f2e 207546 0 2024-04-24T08:24:54.410 INFO:tasks.workunit.client.0.smithi090.stdout:7/368: mkdir db/d17/d39/d86 0 2024-04-24T08:24:54.414 INFO:tasks.workunit.client.0.smithi090.stdout:5/373: write d8/da/d3a/d2d/f4d [21606,83501] 0 2024-04-24T08:24:54.416 INFO:tasks.workunit.client.0.smithi090.stdout:6/334: dread d8/f16 [0,4194304] 0 2024-04-24T08:24:54.416 INFO:tasks.workunit.client.0.smithi090.stdout:3/295: dwrite d2/df/d1a/f2b [4194304,4194304] 0 2024-04-24T08:24:54.418 INFO:tasks.workunit.client.0.smithi090.stdout:9/310: truncate d0/d19/f46 2348459 0 2024-04-24T08:24:54.422 INFO:tasks.workunit.client.0.smithi090.stdout:0/354: dread f8 [0,4194304] 0 2024-04-24T08:24:54.424 INFO:tasks.workunit.client.0.smithi090.stdout:5/374: mkdir d8/da/d14/d2f/d81 0 2024-04-24T08:24:54.426 INFO:tasks.workunit.client.1.smithi149.stdout:5/745: mkdir db/d12/d1e/d65/de8/dfa 0 2024-04-24T08:24:54.426 INFO:tasks.workunit.client.0.smithi090.stdout:2/312: link d0/d14/l20 d0/d7/l61 0 2024-04-24T08:24:54.428 INFO:tasks.workunit.client.1.smithi149.stdout:5/746: dread - db/d12/d60/fe7 zero size 2024-04-24T08:24:54.429 INFO:tasks.workunit.client.0.smithi090.stdout:6/335: mknod d8/d1d/d2f/c63 0 2024-04-24T08:24:54.431 INFO:tasks.workunit.client.0.smithi090.stdout:3/296: dwrite d2/df/d21/f32 [0,4194304] 0 2024-04-24T08:24:54.433 INFO:tasks.workunit.client.1.smithi149.stdout:9/674: rename d1/de/ca9 to d1/da/dd/d79/db6/cf1 0 2024-04-24T08:24:54.436 INFO:tasks.workunit.client.1.smithi149.stdout:7/631: creat d9/dc/d11/d84/da7/fd3 x:0 0 0 2024-04-24T08:24:54.438 INFO:tasks.workunit.client.1.smithi149.stdout:7/632: chown d9/d43/d64/c8d 15 1 2024-04-24T08:24:54.443 INFO:tasks.workunit.client.0.smithi090.stdout:5/375: stat d8/da/fc 0 2024-04-24T08:24:54.444 INFO:tasks.workunit.client.0.smithi090.stdout:5/376: stat d8/da/d3a/l64 0 2024-04-24T08:24:54.444 INFO:tasks.workunit.client.0.smithi090.stdout:5/377: chown d8/d56/d6d 1849 1 2024-04-24T08:24:54.448 INFO:tasks.workunit.client.1.smithi149.stdout:1/754: sync 2024-04-24T08:24:54.448 INFO:tasks.workunit.client.1.smithi149.stdout:4/592: sync 2024-04-24T08:24:54.448 INFO:tasks.workunit.client.1.smithi149.stdout:0/590: sync 2024-04-24T08:24:54.450 INFO:tasks.workunit.client.0.smithi090.stdout:7/369: rename db/d17/f7a to db/d17/d29/f87 0 2024-04-24T08:24:54.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/370: write db/d17/d29/d32/d5a/d75/f70 [3326823,80123] 0 2024-04-24T08:24:54.453 INFO:tasks.workunit.client.1.smithi149.stdout:8/558: mknod d1/de/cc1 0 2024-04-24T08:24:54.456 INFO:tasks.workunit.client.1.smithi149.stdout:5/747: mkdir db/d12/d60/db2/d83/dfb 0 2024-04-24T08:24:54.459 INFO:tasks.workunit.client.1.smithi149.stdout:5/748: truncate db/d12/d2f/d74/fa2 660564 0 2024-04-24T08:24:54.460 INFO:tasks.workunit.client.1.smithi149.stdout:9/675: creat d1/d2/d34/ff2 x:0 0 0 2024-04-24T08:24:54.461 INFO:tasks.workunit.client.0.smithi090.stdout:5/378: creat d8/da/d3a/d2d/f82 x:0 0 0 2024-04-24T08:24:54.471 INFO:tasks.workunit.client.0.smithi090.stdout:3/297: symlink d2/df/d1a/d37/d59/l5d 0 2024-04-24T08:24:54.471 INFO:tasks.workunit.client.1.smithi149.stdout:0/591: mknod d2/d3c/d57/cbf 0 2024-04-24T08:24:54.472 INFO:tasks.workunit.client.0.smithi090.stdout:6/336: dread d8/d1d/d2f/d30/f52 [0,4194304] 0 2024-04-24T08:24:54.473 INFO:tasks.workunit.client.0.smithi090.stdout:3/298: readlink d2/d25/d33/l3f 0 2024-04-24T08:24:54.474 INFO:tasks.workunit.client.0.smithi090.stdout:1/324: getdents d3/d5/da/dc 0 2024-04-24T08:24:54.478 INFO:tasks.workunit.client.0.smithi090.stdout:1/325: readlink d3/d5/da/l22 0 2024-04-24T08:24:54.479 INFO:tasks.workunit.client.0.smithi090.stdout:9/311: dwrite d0/d19/d33/f36 [0,4194304] 0 2024-04-24T08:24:54.479 INFO:tasks.workunit.client.1.smithi149.stdout:3/714: dwrite d0/d5/d53/d39/f8a [0,4194304] 0 2024-04-24T08:24:54.481 INFO:tasks.workunit.client.0.smithi090.stdout:5/379: dwrite d8/da/f35 [0,4194304] 0 2024-04-24T08:24:54.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/312: readlink d0/d19/d33/d4d/l23 0 2024-04-24T08:24:54.483 INFO:tasks.workunit.client.0.smithi090.stdout:9/313: write d0/dc/fd [3332807,26703] 0 2024-04-24T08:24:54.484 INFO:tasks.workunit.client.0.smithi090.stdout:5/380: chown d8/da/d14/c7d 6 1 2024-04-24T08:24:54.489 INFO:tasks.workunit.client.0.smithi090.stdout:0/355: rename d1a/d2a/d35/l40 to d1a/d2a/d3c/l75 0 2024-04-24T08:24:54.491 INFO:tasks.workunit.client.1.smithi149.stdout:9/676: mkdir d1/d2/d15/d1c/d42/df3 0 2024-04-24T08:24:54.500 INFO:tasks.workunit.client.1.smithi149.stdout:6/563: creat d0/d1a/d28/d31/d4c/d4e/d62/fbc x:0 0 0 2024-04-24T08:24:54.503 INFO:tasks.workunit.client.1.smithi149.stdout:5/749: dread db/d12/d2f/f57 [4194304,4194304] 0 2024-04-24T08:24:54.508 INFO:tasks.workunit.client.1.smithi149.stdout:0/592: rmdir d2/d9 39 2024-04-24T08:24:54.508 INFO:tasks.workunit.client.1.smithi149.stdout:5/750: write db/d12/f84 [4431226,65880] 0 2024-04-24T08:24:54.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/221: sync 2024-04-24T08:24:54.510 INFO:tasks.workunit.client.0.smithi090.stdout:8/274: sync 2024-04-24T08:24:54.512 INFO:tasks.workunit.client.1.smithi149.stdout:2/566: link d0/d6/db/d39/d78/d8a/f99 d0/d6/db/d39/d3d/d44/faa 0 2024-04-24T08:24:54.512 INFO:tasks.workunit.client.0.smithi090.stdout:4/222: truncate d3/d18/d1d/d22/f35 128281 0 2024-04-24T08:24:54.513 INFO:tasks.workunit.client.0.smithi090.stdout:7/371: symlink db/d17/d29/l88 0 2024-04-24T08:24:54.513 INFO:tasks.workunit.client.1.smithi149.stdout:1/755: dwrite d0/da/f48 [0,4194304] 0 2024-04-24T08:24:54.515 INFO:tasks.workunit.client.0.smithi090.stdout:3/299: creat d2/df/d1a/d37/d59/f5e x:0 0 0 2024-04-24T08:24:54.518 INFO:tasks.workunit.client.1.smithi149.stdout:2/567: write d0/d6/db/d28/f48 [211,125565] 0 2024-04-24T08:24:54.518 INFO:tasks.workunit.client.1.smithi149.stdout:7/633: truncate d9/dc/d11/d38/f4e 1500436 0 2024-04-24T08:24:54.520 INFO:tasks.workunit.client.0.smithi090.stdout:1/326: creat d3/d1d/d54/f74 x:0 0 0 2024-04-24T08:24:54.521 INFO:tasks.workunit.client.1.smithi149.stdout:3/715: rmdir d0/d88 39 2024-04-24T08:24:54.526 INFO:tasks.workunit.client.0.smithi090.stdout:2/313: rename d0/d14/f1f to d0/d14/d17/f62 0 2024-04-24T08:24:54.527 INFO:tasks.workunit.client.1.smithi149.stdout:4/593: dwrite d2/d18/d1e/d3c/f43 [0,4194304] 0 2024-04-24T08:24:54.541 INFO:tasks.workunit.client.1.smithi149.stdout:8/559: write d1/d1f/d25/f8f [2117586,75910] 0 2024-04-24T08:24:54.541 INFO:tasks.workunit.client.0.smithi090.stdout:7/372: mknod db/c89 0 2024-04-24T08:24:54.542 INFO:tasks.workunit.client.1.smithi149.stdout:9/677: write d1/da/dd/f9e [502935,48601] 0 2024-04-24T08:24:54.543 INFO:tasks.workunit.client.0.smithi090.stdout:3/300: chown d2/df/d1a/c5c 0 1 2024-04-24T08:24:54.546 INFO:tasks.workunit.client.1.smithi149.stdout:1/756: rename d0/d5/df/f28 to d0/ddf/da0/db8/dc5/ff8 0 2024-04-24T08:24:54.546 INFO:tasks.workunit.client.1.smithi149.stdout:8/560: dread - d1/d1f/d31/fbb zero size 2024-04-24T08:24:54.548 INFO:tasks.workunit.client.0.smithi090.stdout:1/327: write d3/d1d/d34/f6b [306068,114429] 0 2024-04-24T08:24:54.549 INFO:tasks.workunit.client.1.smithi149.stdout:2/568: dwrite d0/d6/db/d39/d3d/d58/d98/fa5 [0,4194304] 0 2024-04-24T08:24:54.550 INFO:tasks.workunit.client.0.smithi090.stdout:1/328: dread - d3/d5/d58/f60 zero size 2024-04-24T08:24:54.554 INFO:tasks.workunit.client.0.smithi090.stdout:0/356: dwrite d1a/d2a/d35/f42 [0,4194304] 0 2024-04-24T08:24:54.555 INFO:tasks.workunit.client.0.smithi090.stdout:0/357: chown f8 2598591 1 2024-04-24T08:24:54.557 INFO:tasks.workunit.client.1.smithi149.stdout:3/716: creat d0/d5/da/d19/d1b/d22/dd4/ff8 x:0 0 0 2024-04-24T08:24:54.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/381: rename d8/d25/c46 to d8/da/d14/d2f/d44/c83 0 2024-04-24T08:24:54.559 INFO:tasks.workunit.client.0.smithi090.stdout:5/382: fsync d8/d56/d6d/d7a/f4e 0 2024-04-24T08:24:54.561 INFO:tasks.workunit.client.1.smithi149.stdout:7/634: write d9/dc/d11/d83/da6/d67/d69/f98 [345208,89916] 0 2024-04-24T08:24:54.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/314: chown d0/d7/l2f 1 1 2024-04-24T08:24:54.564 INFO:tasks.workunit.client.1.smithi149.stdout:0/593: unlink d2/d9/df/d2f/c31 0 2024-04-24T08:24:54.565 INFO:tasks.workunit.client.1.smithi149.stdout:5/751: dread db/d12/d60/db2/d83/ff4 [0,4194304] 0 2024-04-24T08:24:54.570 INFO:tasks.workunit.client.0.smithi090.stdout:6/337: link d8/f16 d8/dc/d19/f64 0 2024-04-24T08:24:54.571 INFO:tasks.workunit.client.0.smithi090.stdout:6/338: readlink d8/d1d/d2f/l31 0 2024-04-24T08:24:54.571 INFO:tasks.workunit.client.0.smithi090.stdout:8/275: creat d2/da/d13/d15/d50/f5d x:0 0 0 2024-04-24T08:24:54.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/223: dread d3/df/f1e [0,4194304] 0 2024-04-24T08:24:54.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/373: unlink db/d17/d29/l88 0 2024-04-24T08:24:54.572 INFO:tasks.workunit.client.1.smithi149.stdout:4/594: stat d2/d18/d1e/d3c/d48/d69/d7d/d8d/fb8 0 2024-04-24T08:24:54.573 INFO:tasks.workunit.client.1.smithi149.stdout:9/678: mkdir d1/d2/d33/d5d/d83/df4 0 2024-04-24T08:24:54.573 INFO:tasks.workunit.client.1.smithi149.stdout:5/752: truncate db/d12/f84 5495388 0 2024-04-24T08:24:54.574 INFO:tasks.workunit.client.1.smithi149.stdout:8/561: mknod d1/d1f/d22/cc2 0 2024-04-24T08:24:54.575 INFO:tasks.workunit.client.1.smithi149.stdout:2/569: creat d0/d6/db/d27/fab x:0 0 0 2024-04-24T08:24:54.576 INFO:tasks.workunit.client.0.smithi090.stdout:1/329: symlink d3/d25/l75 0 2024-04-24T08:24:54.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/314: creat d0/dc/f58 x:0 0 0 2024-04-24T08:24:54.590 INFO:tasks.workunit.client.0.smithi090.stdout:3/301: dwrite d2/f17 [0,4194304] 0 2024-04-24T08:24:54.591 INFO:tasks.workunit.client.1.smithi149.stdout:2/570: dwrite d0/f70 [0,4194304] 0 2024-04-24T08:24:54.593 INFO:tasks.workunit.client.0.smithi090.stdout:8/276: mknod d2/da/d13/d34/c5e 0 2024-04-24T08:24:54.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/753: creat db/d12/d1e/d65/ffc x:0 0 0 2024-04-24T08:24:54.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/330: dwrite d3/d5/d58/f60 [0,4194304] 0 2024-04-24T08:24:54.598 INFO:tasks.workunit.client.1.smithi149.stdout:7/635: write d9/d43/d64/f73 [1430558,76717] 0 2024-04-24T08:24:54.599 INFO:tasks.workunit.client.0.smithi090.stdout:7/374: rmdir db/d17/d29/d32 39 2024-04-24T08:24:54.603 INFO:tasks.workunit.client.1.smithi149.stdout:4/595: write d2/d18/d1e/f2c [1069158,83080] 0 2024-04-24T08:24:54.605 INFO:tasks.workunit.client.1.smithi149.stdout:9/679: rename d1/d2/d15/d1c/d42/f4b to d1/d2/d33/d5d/d83/dbf/dc4/ff5 0 2024-04-24T08:24:54.606 INFO:tasks.workunit.client.0.smithi090.stdout:1/331: chown d3/d1d/d34/f6b 124 1 2024-04-24T08:24:54.606 INFO:tasks.workunit.client.0.smithi090.stdout:5/383: dwrite d8/da/fc [0,4194304] 0 2024-04-24T08:24:54.610 INFO:tasks.workunit.client.0.smithi090.stdout:2/315: dread d0/d14/f38 [0,4194304] 0 2024-04-24T08:24:54.611 INFO:tasks.workunit.client.1.smithi149.stdout:3/717: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:24:54.612 INFO:tasks.workunit.client.0.smithi090.stdout:6/339: unlink l7 0 2024-04-24T08:24:54.617 INFO:tasks.workunit.client.0.smithi090.stdout:8/277: dwrite d2/da/d13/f18 [0,4194304] 0 2024-04-24T08:24:54.618 INFO:tasks.workunit.client.0.smithi090.stdout:7/375: readlink db/d17/l2b 0 2024-04-24T08:24:54.618 INFO:tasks.workunit.client.0.smithi090.stdout:4/224: creat d3/d18/d34/f3d x:0 0 0 2024-04-24T08:24:54.620 INFO:tasks.workunit.client.0.smithi090.stdout:5/384: mknod d8/d56/d6d/d7b/c84 0 2024-04-24T08:24:54.621 INFO:tasks.workunit.client.1.smithi149.stdout:4/596: write d2/d18/d1e/f2c [2188976,79751] 0 2024-04-24T08:24:54.622 INFO:tasks.workunit.client.1.smithi149.stdout:6/564: getdents d0/d1a/d28/d31/d38 0 2024-04-24T08:24:54.623 INFO:tasks.workunit.client.0.smithi090.stdout:1/332: rename d3/d5/da/dc/f71 to d3/d5/d3e/f76 0 2024-04-24T08:24:54.623 INFO:tasks.workunit.client.1.smithi149.stdout:1/757: write d0/d1b/db5/fb1 [5995,119074] 0 2024-04-24T08:24:54.623 INFO:tasks.workunit.client.1.smithi149.stdout:2/571: unlink d0/d6/db/d39/d3d/d58/f6d 0 2024-04-24T08:24:54.625 INFO:tasks.workunit.client.0.smithi090.stdout:0/358: write d1a/d2a/f32 [1071865,123146] 0 2024-04-24T08:24:54.630 INFO:tasks.workunit.client.0.smithi090.stdout:7/376: write db/d17/d29/f87 [33757,48712] 0 2024-04-24T08:24:54.631 INFO:tasks.workunit.client.1.smithi149.stdout:1/758: write d0/da/fe4 [446030,79713] 0 2024-04-24T08:24:54.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/636: symlink d9/dc/d11/d84/da7/ld4 0 2024-04-24T08:24:54.631 INFO:tasks.workunit.client.1.smithi149.stdout:2/572: readlink d0/d6/db/d39/l62 0 2024-04-24T08:24:54.633 INFO:tasks.workunit.client.1.smithi149.stdout:8/562: write d1/d1f/d25/f34 [658267,37638] 0 2024-04-24T08:24:54.638 INFO:tasks.workunit.client.0.smithi090.stdout:6/340: rename d8/d1d/d1f/c21 to d8/d1d/d2f/d30/d38/c65 0 2024-04-24T08:24:54.639 INFO:tasks.workunit.client.0.smithi090.stdout:0/359: rmdir d1a/d2a/d69 39 2024-04-24T08:24:54.640 INFO:tasks.workunit.client.1.smithi149.stdout:3/718: creat d0/d5/d53/d34/d3a/d93/ff9 x:0 0 0 2024-04-24T08:24:54.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/333: dread d3/f4 [8388608,4194304] 0 2024-04-24T08:24:54.641 INFO:tasks.workunit.client.0.smithi090.stdout:0/360: chown d1a/d2a/d3c/l4f 54 1 2024-04-24T08:24:54.641 INFO:tasks.workunit.client.0.smithi090.stdout:0/361: readlink d1a/l2d 0 2024-04-24T08:24:54.642 INFO:tasks.workunit.client.0.smithi090.stdout:1/334: write d3/d1d/f37 [136305,78828] 0 2024-04-24T08:24:54.645 INFO:tasks.workunit.client.1.smithi149.stdout:2/573: write d0/d6/db/d39/d3d/d58/f5e [1669727,103881] 0 2024-04-24T08:24:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:5/754: symlink db/d12/lfd 0 2024-04-24T08:24:54.654 INFO:tasks.workunit.client.1.smithi149.stdout:9/680: dwrite d1/da/f3e [0,4194304] 0 2024-04-24T08:24:54.655 INFO:tasks.workunit.client.1.smithi149.stdout:7/637: dread - d9/dc/d11/d83/da6/d67/fa8 zero size 2024-04-24T08:24:54.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/385: rename d8/da/d14/d2f/c3e to d8/da/d3a/d2d/d37/c85 0 2024-04-24T08:24:54.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/377: dwrite db/d17/d29/d32/d5a/d75/f81 [0,4194304] 0 2024-04-24T08:24:54.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/681: write d1/d2/d15/fbc [1359067,105803] 0 2024-04-24T08:24:54.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/563: creat d1/d1f/d31/fc3 x:0 0 0 2024-04-24T08:24:54.666 INFO:tasks.workunit.client.1.smithi149.stdout:4/597: dwrite d2/fc [0,4194304] 0 2024-04-24T08:24:54.671 INFO:tasks.workunit.client.1.smithi149.stdout:3/719: mknod d0/d5/d53/d34/d3a/cfa 0 2024-04-24T08:24:54.672 INFO:tasks.workunit.client.1.smithi149.stdout:8/564: readlink d1/de/d18/l1e 0 2024-04-24T08:24:54.672 INFO:tasks.workunit.client.1.smithi149.stdout:0/594: getdents d2 0 2024-04-24T08:24:54.674 INFO:tasks.workunit.client.1.smithi149.stdout:2/574: creat d0/d6/db/d39/d78/fac x:0 0 0 2024-04-24T08:24:54.678 INFO:tasks.workunit.client.1.smithi149.stdout:1/759: truncate d0/d5/db/d18/f8a 351740 0 2024-04-24T08:24:54.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/595: dread - d2/d9/df/d40/d4a/d4e/d6f/f76 zero size 2024-04-24T08:24:54.690 INFO:tasks.workunit.client.1.smithi149.stdout:1/760: write d0/d1b/d5d/fdc [397735,60874] 0 2024-04-24T08:24:54.691 INFO:tasks.workunit.client.0.smithi090.stdout:0/362: rename d1a/d2a/d58/f5c to d1a/d2a/d35/d68/f76 0 2024-04-24T08:24:54.695 INFO:tasks.workunit.client.0.smithi090.stdout:2/316: truncate d0/d7/f11 10451100 0 2024-04-24T08:24:54.697 INFO:tasks.workunit.client.1.smithi149.stdout:6/565: mkdir d0/d1a/d28/d31/d4c/d80/dbd 0 2024-04-24T08:24:54.697 INFO:tasks.workunit.client.1.smithi149.stdout:5/755: dwrite db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:54.697 INFO:tasks.workunit.client.1.smithi149.stdout:7/638: dwrite d9/fd [0,4194304] 0 2024-04-24T08:24:54.715 INFO:tasks.workunit.client.0.smithi090.stdout:4/225: getdents d3/d18/d34 0 2024-04-24T08:24:54.721 INFO:tasks.workunit.client.1.smithi149.stdout:9/682: symlink d1/d99/lf6 0 2024-04-24T08:24:54.723 INFO:tasks.workunit.client.1.smithi149.stdout:9/683: chown d1/d2/c9 413 1 2024-04-24T08:24:54.727 INFO:tasks.workunit.client.1.smithi149.stdout:9/684: write d1/d2/d34/f40 [4916500,97657] 0 2024-04-24T08:24:54.728 INFO:tasks.workunit.client.0.smithi090.stdout:4/226: dread d3/d18/d1d/f1f [0,4194304] 0 2024-04-24T08:24:54.740 INFO:tasks.workunit.client.0.smithi090.stdout:1/335: rename d3/d5/d58/l5c to d3/d5/l77 0 2024-04-24T08:24:54.741 INFO:tasks.workunit.client.1.smithi149.stdout:0/596: mknod d2/d9/df/d48/cc0 0 2024-04-24T08:24:54.742 INFO:tasks.workunit.client.0.smithi090.stdout:0/363: rmdir d1a/d2a/d69 39 2024-04-24T08:24:54.744 INFO:tasks.workunit.client.1.smithi149.stdout:1/761: creat d0/d1b/d2a/ff9 x:0 0 0 2024-04-24T08:24:54.744 INFO:tasks.workunit.client.0.smithi090.stdout:4/227: dwrite d3/d18/d34/f3d [0,4194304] 0 2024-04-24T08:24:54.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/364: chown d1a/f64 3 1 2024-04-24T08:24:54.749 INFO:tasks.workunit.client.1.smithi149.stdout:7/639: creat d9/dc/de/d44/fd5 x:0 0 0 2024-04-24T08:24:54.749 INFO:tasks.workunit.client.0.smithi090.stdout:2/317: creat d0/d5e/f63 x:0 0 0 2024-04-24T08:24:54.750 INFO:tasks.workunit.client.0.smithi090.stdout:7/378: dwrite db/f12 [4194304,4194304] 0 2024-04-24T08:24:54.751 INFO:tasks.workunit.client.1.smithi149.stdout:5/756: dread db/d12/d1e/f41 [0,4194304] 0 2024-04-24T08:24:54.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/336: rename d3/d5/da/dc/d65 to d3/d25/d78 0 2024-04-24T08:24:54.755 INFO:tasks.workunit.client.0.smithi090.stdout:2/318: truncate d0/d14/d4d/d50/f5c 649410 0 2024-04-24T08:24:54.756 INFO:tasks.workunit.client.1.smithi149.stdout:3/720: mkdir d0/d5/d53/d4c/dca/dfb 0 2024-04-24T08:24:54.758 INFO:tasks.workunit.client.0.smithi090.stdout:5/386: dwrite d8/da/d14/d2f/f42 [0,4194304] 0 2024-04-24T08:24:54.761 INFO:tasks.workunit.client.0.smithi090.stdout:2/319: chown d0/d14/d17/f62 11352 1 2024-04-24T08:24:54.761 INFO:tasks.workunit.client.0.smithi090.stdout:7/379: unlink fa 0 2024-04-24T08:24:54.762 INFO:tasks.workunit.client.1.smithi149.stdout:6/566: write d0/f4 [1369434,114435] 0 2024-04-24T08:24:54.763 INFO:tasks.workunit.client.1.smithi149.stdout:1/762: rename d0/ddf/da0/f96 to d0/d1b/d2a/ffa 0 2024-04-24T08:24:54.765 INFO:tasks.workunit.client.0.smithi090.stdout:1/337: creat d3/d25/d30/f79 x:0 0 0 2024-04-24T08:24:54.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/380: read db/d17/f3a [3363087,84435] 0 2024-04-24T08:24:54.771 INFO:tasks.workunit.client.1.smithi149.stdout:3/721: chown d0/d5/da/d59/fc9 24077 1 2024-04-24T08:24:54.776 INFO:tasks.workunit.client.1.smithi149.stdout:6/567: truncate d0/d1a/d28/d31/d4c/d4e/d62/fbc 286238 0 2024-04-24T08:24:54.781 INFO:tasks.workunit.client.1.smithi149.stdout:5/757: dwrite db/d12/d1e/d65/fb4 [0,4194304] 0 2024-04-24T08:24:54.787 INFO:tasks.workunit.client.0.smithi090.stdout:4/228: dread d3/d25/f2f [0,4194304] 0 2024-04-24T08:24:54.788 INFO:tasks.workunit.client.1.smithi149.stdout:7/640: rename d9/dc/de/d44/c97 to d9/dc/d11/d83/da6/d67/d69/dad/cd6 0 2024-04-24T08:24:54.790 INFO:tasks.workunit.client.0.smithi090.stdout:4/229: write d3/df/f30 [890350,1795] 0 2024-04-24T08:24:54.790 INFO:tasks.workunit.client.0.smithi090.stdout:7/381: rename db/d17/d29/d32/d5a/d75/f77 to db/d17/d29/d32/d5a/d75/f8a 0 2024-04-24T08:24:54.790 INFO:tasks.workunit.client.1.smithi149.stdout:6/568: write d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 [5274310,104890] 0 2024-04-24T08:24:54.792 INFO:tasks.workunit.client.0.smithi090.stdout:0/365: fsync d1a/d4b/d53/f6a 0 2024-04-24T08:24:54.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/366: write d1a/d4b/f6f [727493,66161] 0 2024-04-24T08:24:54.796 INFO:tasks.workunit.client.1.smithi149.stdout:2/575: dwrite d0/d6/db/f4a [0,4194304] 0 2024-04-24T08:24:54.798 INFO:tasks.workunit.client.0.smithi090.stdout:7/382: dread db/d17/d29/d38/d3e/f3f [0,4194304] 0 2024-04-24T08:24:54.799 INFO:tasks.workunit.client.1.smithi149.stdout:6/569: write d0/d1a/d28/d31/d33/f9b [1591817,75026] 0 2024-04-24T08:24:54.801 INFO:tasks.workunit.client.0.smithi090.stdout:7/383: read db/d17/d29/d32/d5a/d75/f81 [3842540,41324] 0 2024-04-24T08:24:54.802 INFO:tasks.workunit.client.1.smithi149.stdout:6/570: chown d0/fa 188243 1 2024-04-24T08:24:54.802 INFO:tasks.workunit.client.1.smithi149.stdout:8/565: link d1/l12 d1/de/d18/d32/d63/d9e/lc4 0 2024-04-24T08:24:54.803 INFO:tasks.workunit.client.0.smithi090.stdout:7/384: chown db/d17/d29/d32/f57 761 1 2024-04-24T08:24:54.809 INFO:tasks.workunit.client.1.smithi149.stdout:0/597: truncate d2/d9/d1a/f6b 3661102 0 2024-04-24T08:24:54.809 INFO:tasks.workunit.client.1.smithi149.stdout:8/566: read d1/de/d18/d32/d63/d76/f9a [13339,42064] 0 2024-04-24T08:24:54.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/567: write d1/de/d18/f45 [2266042,121647] 0 2024-04-24T08:24:54.814 INFO:tasks.workunit.client.1.smithi149.stdout:1/763: write d0/d5/df/f94 [1299432,113107] 0 2024-04-24T08:24:54.814 INFO:tasks.workunit.client.0.smithi090.stdout:1/338: dread d3/d5/da/dc/f2b [0,4194304] 0 2024-04-24T08:24:54.815 INFO:tasks.workunit.client.0.smithi090.stdout:4/230: mknod d3/c3e 0 2024-04-24T08:24:54.815 INFO:tasks.workunit.client.1.smithi149.stdout:1/764: stat d0/d5/f9 0 2024-04-24T08:24:54.817 INFO:tasks.workunit.client.1.smithi149.stdout:5/758: creat db/d12/d60/db2/d83/dc8/ffe x:0 0 0 2024-04-24T08:24:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/765: write d0/d5/db/d18/d2f/fd1 [988959,75583] 0 2024-04-24T08:24:54.819 INFO:tasks.workunit.client.1.smithi149.stdout:5/759: readlink db/d12/d60/d8f/lba 0 2024-04-24T08:24:54.820 INFO:tasks.workunit.client.1.smithi149.stdout:5/760: stat db/d12/d60/fe7 0 2024-04-24T08:24:54.821 INFO:tasks.workunit.client.0.smithi090.stdout:0/367: symlink d1a/d2a/d35/d68/l77 0 2024-04-24T08:24:54.822 INFO:tasks.workunit.client.1.smithi149.stdout:2/576: unlink d0/d6/db/d39/f80 0 2024-04-24T08:24:54.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/368: dread - d1a/d2a/d3c/f5b zero size 2024-04-24T08:24:54.824 INFO:tasks.workunit.client.1.smithi149.stdout:7/641: truncate d9/dc/de/d44/f91 572669 0 2024-04-24T08:24:54.825 INFO:tasks.workunit.client.1.smithi149.stdout:7/642: readlink d9/dc/d11/d84/da7/ld4 0 2024-04-24T08:24:54.826 INFO:tasks.workunit.client.0.smithi090.stdout:5/387: creat d8/d59/f86 x:0 0 0 2024-04-24T08:24:54.828 INFO:tasks.workunit.client.0.smithi090.stdout:2/320: rename d0/d7/d24/f27 to d0/d7/f64 0 2024-04-24T08:24:54.830 INFO:tasks.workunit.client.1.smithi149.stdout:9/685: getdents d1/da/d13/d67/de4 0 2024-04-24T08:24:54.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/766: mkdir d0/d5/db/d18/d4d/de8/dfb 0 2024-04-24T08:24:54.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/385: creat db/d54/d83/f8b x:0 0 0 2024-04-24T08:24:54.841 INFO:tasks.workunit.client.0.smithi090.stdout:2/321: dwrite d0/d14/f15 [0,4194304] 0 2024-04-24T08:24:54.841 INFO:tasks.workunit.client.1.smithi149.stdout:9/686: dread d1/da/f8 [0,4194304] 0 2024-04-24T08:24:54.841 INFO:tasks.workunit.client.1.smithi149.stdout:5/761: symlink db/d12/d2f/d78/lff 0 2024-04-24T08:24:54.846 INFO:tasks.workunit.client.1.smithi149.stdout:6/571: symlink d0/d1a/d28/lbe 0 2024-04-24T08:24:54.848 INFO:tasks.workunit.client.0.smithi090.stdout:4/231: link d3/df/f36 d3/f3f 0 2024-04-24T08:24:54.850 INFO:tasks.workunit.client.0.smithi090.stdout:4/232: read d3/f16 [3567242,25506] 0 2024-04-24T08:24:54.852 INFO:tasks.workunit.client.1.smithi149.stdout:7/643: symlink d9/dc/d11/d84/ld7 0 2024-04-24T08:24:54.853 INFO:tasks.workunit.client.1.smithi149.stdout:8/568: write d1/de/d18/d32/d65/d8b/f7c [3720937,46613] 0 2024-04-24T08:24:54.854 INFO:tasks.workunit.client.1.smithi149.stdout:3/722: link d0/d5/da/d5a/d63/d6a/f7c d0/d26/d3c/db1/ffc 0 2024-04-24T08:24:54.857 INFO:tasks.workunit.client.0.smithi090.stdout:7/386: unlink db/d17/d29/d38/d3e/d46/f6f 0 2024-04-24T08:24:54.858 INFO:tasks.workunit.client.0.smithi090.stdout:7/387: write db/f12 [4956587,80619] 0 2024-04-24T08:24:54.859 INFO:tasks.workunit.client.0.smithi090.stdout:5/388: unlink d8/da/d18/d21/c62 0 2024-04-24T08:24:54.860 INFO:tasks.workunit.client.1.smithi149.stdout:3/723: write d0/d5/d53/d34/d3a/d93/ff9 [144385,12730] 0 2024-04-24T08:24:54.861 INFO:tasks.workunit.client.0.smithi090.stdout:5/389: chown d8/da/d3a/d2d/d37/f6a 471356 1 2024-04-24T08:24:54.862 INFO:tasks.workunit.client.1.smithi149.stdout:8/569: fsync d1/de/d18/d32/d65/d8b/f7c 0 2024-04-24T08:24:54.864 INFO:tasks.workunit.client.1.smithi149.stdout:9/687: dwrite d1/d2/d33/dea/fef [0,4194304] 0 2024-04-24T08:24:54.865 INFO:tasks.workunit.client.1.smithi149.stdout:5/762: fdatasync db/d35/f7c 0 2024-04-24T08:24:54.866 INFO:tasks.workunit.client.1.smithi149.stdout:1/767: creat d0/d5/db/d18/d4d/de8/ffc x:0 0 0 2024-04-24T08:24:54.866 INFO:tasks.workunit.client.1.smithi149.stdout:0/598: write d2/d17/f98 [2053713,5010] 0 2024-04-24T08:24:54.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/233: creat d3/d18/d1d/f40 x:0 0 0 2024-04-24T08:24:54.868 INFO:tasks.workunit.client.1.smithi149.stdout:8/570: write d1/d1f/d31/f67 [1248996,79199] 0 2024-04-24T08:24:54.872 INFO:tasks.workunit.client.1.smithi149.stdout:2/577: dread d0/d6/db/d39/d3d/f6b [0,4194304] 0 2024-04-24T08:24:54.872 INFO:tasks.workunit.client.1.smithi149.stdout:3/724: fsync d0/d5/da/d59/fc9 0 2024-04-24T08:24:54.876 INFO:tasks.workunit.client.1.smithi149.stdout:3/725: fdatasync d0/d5/da/d5e/f30 0 2024-04-24T08:24:54.876 INFO:tasks.workunit.client.1.smithi149.stdout:2/578: write d0/d6/f16 [4965712,19266] 0 2024-04-24T08:24:54.877 INFO:tasks.workunit.client.0.smithi090.stdout:2/322: mkdir d0/d14/d17/d2a/d31/d46/d60/d65 0 2024-04-24T08:24:54.879 INFO:tasks.workunit.client.1.smithi149.stdout:3/726: read - d0/d5/da/d19/d80/dac/fc5 zero size 2024-04-24T08:24:54.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.878+0000 7fc7b4000700 1 -- 172.21.15.90:0/3207952834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 msgr2=0x7fc7ac073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:54.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.878+0000 7fc7b4000700 1 --2- 172.21.15.90:0/3207952834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac073c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc798009910 tx=0x7fc798009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:54.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.879+0000 7fc7b4000700 1 -- 172.21.15.90:0/3207952834 shutdown_connections 2024-04-24T08:24:54.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.879+0000 7fc7b4000700 1 --2- 172.21.15.90:0/3207952834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac073c20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:54.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.879+0000 7fc7b4000700 1 --2- 172.21.15.90:0/3207952834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 0x7fc7ac0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:54.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.879+0000 7fc7b4000700 1 -- 172.21.15.90:0/3207952834 >> 172.21.15.90:0/3207952834 conn(0x7fc7ac0fb7f0 msgr2=0x7fc7ac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:54.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.880+0000 7fc7b4000700 1 -- 172.21.15.90:0/3207952834 shutdown_connections 2024-04-24T08:24:54.883 INFO:tasks.workunit.client.0.smithi090.stdout:7/388: symlink db/d17/d29/d32/l8c 0 2024-04-24T08:24:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.880+0000 7fc7b4000700 1 -- 172.21.15.90:0/3207952834 wait complete. 2024-04-24T08:24:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.881+0000 7fc7b4000700 1 Processor -- start 2024-04-24T08:24:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.881+0000 7fc7b4000700 1 -- start start 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b4000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b4000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 0x7fc7ac072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b4000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0728b0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b4000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1a9e20 con 0x7fc7ac074d80 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b1600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b1600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44474/0 (socket says 172.21.15.90:44474) 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b1600700 1 -- 172.21.15.90:0/842269599 learned_addr learned my addr 172.21.15.90:0/842269599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.882+0000 7fc7b0c00700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 0x7fc7ac072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.883+0000 7fc7b1600700 1 -- 172.21.15.90:0/842269599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 msgr2=0x7fc7ac072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.883+0000 7fc7b1600700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 0x7fc7ac072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.883+0000 7fc7b1600700 1 -- 172.21.15.90:0/842269599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980095f0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.886 INFO:tasks.workunit.client.0.smithi090.stdout:1/339: getdents d3/d5/d3e 0 2024-04-24T08:24:54.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.883+0000 7fc7b1600700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d8d0 tx=0x7fc7a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:54.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.884+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a00098a0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.884+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7a0010460 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.884+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a000f5f0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.884+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac1a9fc0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.884+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac1aa490 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.885+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c005320 con 0x7fc7ac0737b0 2024-04-24T08:24:54.887 INFO:tasks.workunit.client.0.smithi090.stdout:7/389: truncate db/d17/d29/d32/f4c 431726 0 2024-04-24T08:24:54.888 INFO:tasks.workunit.client.0.smithi090.stdout:4/234: mknod d3/d38/c41 0 2024-04-24T08:24:54.889 INFO:tasks.workunit.client.1.smithi149.stdout:8/571: write d1/f9 [2887505,15294] 0 2024-04-24T08:24:54.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.889+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7fc7a0009a00 con 0x7fc7ac0737b0 2024-04-24T08:24:54.890 INFO:tasks.workunit.client.0.smithi090.stdout:0/369: getdents d1a/d2a/d35/d62 0 2024-04-24T08:24:54.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.890+0000 7fc7aa000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 0x7fc794070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:54.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.891+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc7a008e940 con 0x7fc7ac0737b0 2024-04-24T08:24:54.892 INFO:tasks.workunit.client.1.smithi149.stdout:9/688: rename d1/d2/dc8/fd0 to d1/da/de8/ff7 0 2024-04-24T08:24:54.893 INFO:tasks.workunit.client.0.smithi090.stdout:0/370: chown d1a/d4b/f4d 25910 1 2024-04-24T08:24:54.894 INFO:tasks.workunit.client.0.smithi090.stdout:2/323: creat d0/d7/d57/f66 x:0 0 0 2024-04-24T08:24:54.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.895+0000 7fc7b0c00700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 0x7fc794070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:54.898 INFO:tasks.workunit.client.1.smithi149.stdout:1/768: mknod d0/d5/d4f/d60/cfd 0 2024-04-24T08:24:54.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/235: unlink d3/l29 0 2024-04-24T08:24:54.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.900+0000 7fc7b0c00700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 0x7fc794070f70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc798005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:54.901 INFO:tasks.workunit.client.1.smithi149.stdout:6/572: write d0/fc [1042527,94178] 0 2024-04-24T08:24:54.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:54.902+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fc7a0057ac0 con 0x7fc7ac0737b0 2024-04-24T08:24:54.903 INFO:tasks.workunit.client.0.smithi090.stdout:7/390: unlink db/d17/d29/d32/d5a/d75/l25 0 2024-04-24T08:24:54.904 INFO:tasks.workunit.client.1.smithi149.stdout:6/573: dread - d0/d1a/d28/d31/d38/f63 zero size 2024-04-24T08:24:54.905 INFO:tasks.workunit.client.0.smithi090.stdout:7/391: readlink db/d17/d39/d59/l74 0 2024-04-24T08:24:54.905 INFO:tasks.workunit.client.1.smithi149.stdout:6/574: chown d0 228 1 2024-04-24T08:24:54.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/392: chown f4 233026184 1 2024-04-24T08:24:54.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/371: chown d1a/d2a/c29 3559 1 2024-04-24T08:24:54.908 INFO:tasks.workunit.client.1.smithi149.stdout:0/599: truncate d2/d9/df/f16 1698853 0 2024-04-24T08:24:54.911 INFO:tasks.workunit.client.0.smithi090.stdout:7/393: write db/d17/d29/d32/f36 [1515712,105608] 0 2024-04-24T08:24:54.912 INFO:tasks.workunit.client.1.smithi149.stdout:8/572: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:24:54.915 INFO:tasks.workunit.client.0.smithi090.stdout:4/236: write d3/df/f1e [3689600,1048] 0 2024-04-24T08:24:54.917 INFO:tasks.workunit.client.0.smithi090.stdout:0/372: truncate d1a/d4b/d53/f6a 2454143 0 2024-04-24T08:24:54.917 INFO:tasks.workunit.client.1.smithi149.stdout:3/727: fdatasync d0/d5/da/d19/d80/f8c 0 2024-04-24T08:24:54.917 INFO:tasks.workunit.client.0.smithi090.stdout:7/394: dread - db/d17/d29/d32/d5a/d75/f8a zero size 2024-04-24T08:24:54.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/689: symlink d1/da/d13/d74/lf8 0 2024-04-24T08:24:54.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/324: dwrite d0/d7/f33 [0,4194304] 0 2024-04-24T08:24:54.922 INFO:tasks.workunit.client.1.smithi149.stdout:1/769: creat d0/ddf/d76/ffe x:0 0 0 2024-04-24T08:24:54.922 INFO:tasks.workunit.client.1.smithi149.stdout:5/763: getdents db/d12/d60/db2/d83/dfb 0 2024-04-24T08:24:54.923 INFO:tasks.workunit.client.0.smithi090.stdout:4/237: write d3/d25/f3a [895133,76253] 0 2024-04-24T08:24:54.924 INFO:tasks.workunit.client.0.smithi090.stdout:0/373: write d1a/d2a/d3c/f5b [423239,110371] 0 2024-04-24T08:24:54.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/600: rmdir d2/d9/df/d48 39 2024-04-24T08:24:54.926 INFO:tasks.workunit.client.0.smithi090.stdout:7/395: write db/d17/d29/f69 [1058063,31543] 0 2024-04-24T08:24:54.926 INFO:tasks.workunit.client.0.smithi090.stdout:5/390: dwrite d8/da/d3a/f43 [0,4194304] 0 2024-04-24T08:24:54.926 INFO:tasks.workunit.client.1.smithi149.stdout:6/575: creat d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fbf x:0 0 0 2024-04-24T08:24:54.932 INFO:tasks.workunit.client.1.smithi149.stdout:7/644: creat d9/dc/d11/d83/da6/d67/fd8 x:0 0 0 2024-04-24T08:24:54.934 INFO:tasks.workunit.client.1.smithi149.stdout:7/645: chown c4 259632 1 2024-04-24T08:24:54.940 INFO:tasks.workunit.client.1.smithi149.stdout:4/598: sync 2024-04-24T08:24:54.941 INFO:tasks.workunit.client.1.smithi149.stdout:2/579: dread d0/d6/db/f9e [0,4194304] 0 2024-04-24T08:24:54.941 INFO:tasks.workunit.client.1.smithi149.stdout:9/690: creat d1/da/d4a/ff9 x:0 0 0 2024-04-24T08:24:54.944 INFO:tasks.workunit.client.1.smithi149.stdout:4/599: write d2/d18/d1e/d3c/f80 [5108590,75689] 0 2024-04-24T08:24:54.944 INFO:tasks.workunit.client.0.smithi090.stdout:4/238: creat d3/d38/f42 x:0 0 0 2024-04-24T08:24:54.946 INFO:tasks.workunit.client.0.smithi090.stdout:4/239: readlink d3/l26 0 2024-04-24T08:24:54.949 INFO:tasks.workunit.client.0.smithi090.stdout:4/240: write d3/d18/d1d/f40 [636508,125341] 0 2024-04-24T08:24:54.951 INFO:tasks.workunit.client.0.smithi090.stdout:5/391: creat d8/d56/d6d/d7a/f87 x:0 0 0 2024-04-24T08:24:54.951 INFO:tasks.workunit.client.1.smithi149.stdout:8/573: fdatasync d1/fbf 0 2024-04-24T08:24:54.953 INFO:tasks.workunit.client.1.smithi149.stdout:8/574: chown d1/d1f/d25/l83 0 1 2024-04-24T08:24:54.953 INFO:tasks.workunit.client.0.smithi090.stdout:2/325: mkdir d0/d7/d67 0 2024-04-24T08:24:54.954 INFO:tasks.workunit.client.1.smithi149.stdout:6/576: unlink d0/d1a/d28/d31/d33/c3e 0 2024-04-24T08:24:54.957 INFO:tasks.workunit.client.0.smithi090.stdout:0/374: mkdir d1a/d2a/d69/d78 0 2024-04-24T08:24:54.959 INFO:tasks.workunit.client.0.smithi090.stdout:1/340: dread d3/d5/d3e/f18 [8388608,4194304] 0 2024-04-24T08:24:54.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/601: rename d2/d9/df/d40/d4a to d2/d9/df/d2f/d6c/dc1 0 2024-04-24T08:24:54.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/646: fsync d9/dc/de/f3a 0 2024-04-24T08:24:54.964 INFO:tasks.workunit.client.0.smithi090.stdout:4/241: creat d3/d38/f43 x:0 0 0 2024-04-24T08:24:54.965 INFO:tasks.workunit.client.1.smithi149.stdout:3/728: truncate d0/d5/da/d19/d1b/d22/d38/f64 676213 0 2024-04-24T08:24:54.968 INFO:tasks.workunit.client.1.smithi149.stdout:9/691: read - d1/de/fa8 zero size 2024-04-24T08:24:54.969 INFO:tasks.workunit.client.0.smithi090.stdout:1/341: chown d3/d1d/d34/d5a/f5b 4691777 1 2024-04-24T08:24:54.971 INFO:tasks.workunit.client.0.smithi090.stdout:1/342: dread - d3/d5/da/f64 zero size 2024-04-24T08:24:54.971 INFO:tasks.workunit.client.1.smithi149.stdout:4/600: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/dbd 0 2024-04-24T08:24:54.973 INFO:tasks.workunit.client.0.smithi090.stdout:1/343: chown d3/d25/l5f 0 1 2024-04-24T08:24:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:4/601: dread - d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 zero size 2024-04-24T08:24:54.977 INFO:tasks.workunit.client.0.smithi090.stdout:2/326: symlink d0/d14/d17/d2a/d31/d46/d60/d65/l68 0 2024-04-24T08:24:54.978 INFO:tasks.workunit.client.0.smithi090.stdout:2/327: dread - d0/d2c/f43 zero size 2024-04-24T08:24:54.980 INFO:tasks.workunit.client.1.smithi149.stdout:6/577: unlink d0/c8 0 2024-04-24T08:24:54.984 INFO:tasks.workunit.client.0.smithi090.stdout:7/396: dwrite db/f15 [0,4194304] 0 2024-04-24T08:24:54.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/344: truncate d3/d5/da/dc/f35 769279 0 2024-04-24T08:24:54.987 INFO:tasks.workunit.client.1.smithi149.stdout:7/647: creat d9/dc/d11/d83/fd9 x:0 0 0 2024-04-24T08:24:54.988 INFO:tasks.workunit.client.1.smithi149.stdout:1/770: dwrite d0/d1b/d5d/f9a [4194304,4194304] 0 2024-04-24T08:24:54.988 INFO:tasks.workunit.client.0.smithi090.stdout:7/397: truncate db/d17/d39/f41 633659 0 2024-04-24T08:24:54.991 INFO:tasks.workunit.client.0.smithi090.stdout:0/375: rename d1a/d2a to d1a/d4b/d53/d79 0 2024-04-24T08:24:54.994 INFO:tasks.workunit.client.1.smithi149.stdout:3/729: dread d0/d5/da/d19/d1b/d72/fbb [0,4194304] 0 2024-04-24T08:24:54.994 INFO:tasks.workunit.client.1.smithi149.stdout:2/580: unlink d0/d6/l43 0 2024-04-24T08:24:54.996 INFO:tasks.workunit.client.1.smithi149.stdout:2/581: write d0/f21 [226261,75160] 0 2024-04-24T08:24:54.998 INFO:tasks.workunit.client.1.smithi149.stdout:9/692: creat d1/da/dd/d79/db6/ffa x:0 0 0 2024-04-24T08:24:54.999 INFO:tasks.workunit.client.1.smithi149.stdout:2/582: stat d0/d6/db/d39/f90 0 2024-04-24T08:24:54.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/345: dread d3/d5/f16 [0,4194304] 0 2024-04-24T08:24:55.000 INFO:tasks.workunit.client.1.smithi149.stdout:4/602: dread - d2/d10/d1a/d34/f62 zero size 2024-04-24T08:24:55.001 INFO:tasks.workunit.client.0.smithi090.stdout:5/392: getdents d8/da/d14 0 2024-04-24T08:24:55.002 INFO:tasks.workunit.client.0.smithi090.stdout:4/242: write d3/d38/f39 [3573270,20332] 0 2024-04-24T08:24:55.003 INFO:tasks.workunit.client.0.smithi090.stdout:1/346: read - d3/d5/da/dc/f51 zero size 2024-04-24T08:24:55.005 INFO:tasks.workunit.client.1.smithi149.stdout:0/602: mknod d2/d9/df/cc2 0 2024-04-24T08:24:55.005 INFO:tasks.workunit.client.0.smithi090.stdout:1/347: readlink d3/l45 0 2024-04-24T08:24:55.007 INFO:tasks.workunit.client.0.smithi090.stdout:2/328: dwrite d0/d7/f32 [4194304,4194304] 0 2024-04-24T08:24:55.008 INFO:tasks.workunit.client.0.smithi090.stdout:4/243: write d3/f32 [187805,7350] 0 2024-04-24T08:24:55.009 INFO:tasks.workunit.client.1.smithi149.stdout:7/648: mknod d9/d43/cda 0 2024-04-24T08:24:55.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/329: write d0/f3b [432692,130250] 0 2024-04-24T08:24:55.010 INFO:tasks.workunit.client.0.smithi090.stdout:4/244: chown d3/c19 115 1 2024-04-24T08:24:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:5/764: dwrite db/d12/d22/f31 [0,4194304] 0 2024-04-24T08:24:55.017 INFO:tasks.workunit.client.0.smithi090.stdout:5/393: creat d8/da/d3a/d2d/d37/f88 x:0 0 0 2024-04-24T08:24:55.023 INFO:tasks.workunit.client.0.smithi090.stdout:1/348: mknod d3/d25/d78/c7a 0 2024-04-24T08:24:55.029 INFO:tasks.workunit.client.0.smithi090.stdout:4/245: creat d3/d24/f44 x:0 0 0 2024-04-24T08:24:55.031 INFO:tasks.workunit.client.0.smithi090.stdout:5/394: mknod d8/da/d14/c89 0 2024-04-24T08:24:55.034 INFO:tasks.workunit.client.1.smithi149.stdout:2/583: write d0/f4d [4093684,17609] 0 2024-04-24T08:24:55.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/578: rename d0/d1a/d7e/c89 to d0/d1a/d28/d31/d4c/d80/d6f/d97/cc0 0 2024-04-24T08:24:55.035 INFO:tasks.workunit.client.0.smithi090.stdout:1/349: symlink d3/d25/l7b 0 2024-04-24T08:24:55.038 INFO:tasks.workunit.client.0.smithi090.stdout:4/246: symlink d3/l45 0 2024-04-24T08:24:55.040 INFO:tasks.workunit.client.1.smithi149.stdout:1/771: getdents d0/d1b/d5d/de5 0 2024-04-24T08:24:55.040 INFO:tasks.workunit.client.1.smithi149.stdout:3/730: dread - d0/d88/fa5 zero size 2024-04-24T08:24:55.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/575: getdents d1/d1f 0 2024-04-24T08:24:55.046 INFO:tasks.workunit.client.0.smithi090.stdout:4/247: rename d3/df/f1e to d3/d18/d1d/f46 0 2024-04-24T08:24:55.047 INFO:tasks.workunit.client.0.smithi090.stdout:4/248: chown d3/d38/f39 1258983433 1 2024-04-24T08:24:55.048 INFO:tasks.workunit.client.1.smithi149.stdout:0/603: dwrite d2/d9/d1a/fbc [0,4194304] 0 2024-04-24T08:24:55.050 INFO:tasks.workunit.client.1.smithi149.stdout:8/576: chown d1/de/d18/d32/d65/d8b/d7d/f96 221621676 1 2024-04-24T08:24:55.052 INFO:tasks.workunit.client.0.smithi090.stdout:2/330: link d0/l12 d0/d7/d28/l69 0 2024-04-24T08:24:55.053 INFO:tasks.workunit.client.1.smithi149.stdout:7/649: fsync d9/d43/f7e 0 2024-04-24T08:24:55.053 INFO:tasks.workunit.client.0.smithi090.stdout:2/331: stat d0/f55 0 2024-04-24T08:24:55.054 INFO:tasks.workunit.client.1.smithi149.stdout:0/604: dread - d2/d9/df/d40/fb4 zero size 2024-04-24T08:24:55.054 INFO:tasks.workunit.client.1.smithi149.stdout:1/772: dread d0/d1b/d5d/fdc [0,4194304] 0 2024-04-24T08:24:55.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/579: symlink d0/d1a/d28/d31/d38/d54/d87/lc1 0 2024-04-24T08:24:55.055 INFO:tasks.workunit.client.0.smithi090.stdout:2/332: write d0/d2c/f43 [594822,864] 0 2024-04-24T08:24:55.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/333: readlink d0/d14/d17/l25 0 2024-04-24T08:24:55.057 INFO:tasks.workunit.client.1.smithi149.stdout:5/765: dwrite db/d12/d60/fe7 [0,4194304] 0 2024-04-24T08:24:55.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/580: write d0/f4 [2241073,108545] 0 2024-04-24T08:24:55.062 INFO:tasks.workunit.client.1.smithi149.stdout:7/650: dread - d9/dc/d11/d83/da6/d67/d69/dad/fc6 zero size 2024-04-24T08:24:55.063 INFO:tasks.workunit.client.1.smithi149.stdout:9/693: write d1/d2/d34/f35 [597376,29110] 0 2024-04-24T08:24:55.066 INFO:tasks.workunit.client.0.smithi090.stdout:4/249: creat d3/d18/d34/f47 x:0 0 0 2024-04-24T08:24:55.068 INFO:tasks.workunit.client.1.smithi149.stdout:9/694: chown d1/da/f3e 86003 1 2024-04-24T08:24:55.070 INFO:tasks.workunit.client.1.smithi149.stdout:7/651: chown d9/dc/d11/d26/f4a 941406454 1 2024-04-24T08:24:55.071 INFO:tasks.workunit.client.1.smithi149.stdout:2/584: dwrite d0/d6/fa0 [0,4194304] 0 2024-04-24T08:24:55.072 INFO:tasks.workunit.client.0.smithi090.stdout:5/395: creat d8/d56/d6d/d7a/d50/f8a x:0 0 0 2024-04-24T08:24:55.075 INFO:tasks.workunit.client.0.smithi090.stdout:0/376: dwrite d1a/f20 [4194304,4194304] 0 2024-04-24T08:24:55.075 INFO:tasks.workunit.client.1.smithi149.stdout:7/652: write f1 [3836557,48235] 0 2024-04-24T08:24:55.083 INFO:tasks.workunit.client.0.smithi090.stdout:1/350: getdents d3/d1d 0 2024-04-24T08:24:55.085 INFO:tasks.workunit.client.1.smithi149.stdout:3/731: mknod d0/d5/da/d19/d1b/d22/dc6/cfd 0 2024-04-24T08:24:55.087 INFO:tasks.workunit.client.0.smithi090.stdout:5/396: creat d8/d56/d6d/d7b/f8b x:0 0 0 2024-04-24T08:24:55.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/397: stat d8/da/d14/d2f/d44/f5b 0 2024-04-24T08:24:55.089 INFO:tasks.workunit.client.1.smithi149.stdout:6/581: mkdir d0/d1a/d28/d31/d4c/d80/d6f/dc2 0 2024-04-24T08:24:55.089 INFO:tasks.workunit.client.0.smithi090.stdout:2/334: creat d0/d2e/f6a x:0 0 0 2024-04-24T08:24:55.090 INFO:tasks.workunit.client.0.smithi090.stdout:1/351: dread - d3/d5/da/dc/f6c zero size 2024-04-24T08:24:55.092 INFO:tasks.workunit.client.1.smithi149.stdout:9/695: mkdir d1/d99/dfb 0 2024-04-24T08:24:55.092 INFO:tasks.workunit.client.1.smithi149.stdout:3/732: chown d0/d26/d3c/db1/fd0 11128 1 2024-04-24T08:24:55.092 INFO:tasks.workunit.client.0.smithi090.stdout:2/335: fsync d0/d7/d57/f66 0 2024-04-24T08:24:55.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/577: fdatasync d1/de/d18/d43/fb6 0 2024-04-24T08:24:55.094 INFO:tasks.workunit.client.0.smithi090.stdout:0/377: dread ff [0,4194304] 0 2024-04-24T08:24:55.094 INFO:tasks.workunit.client.0.smithi090.stdout:5/398: mkdir d8/da/d18/d8c 0 2024-04-24T08:24:55.094 INFO:tasks.workunit.client.1.smithi149.stdout:3/733: readlink d0/d5/l40 0 2024-04-24T08:24:55.095 INFO:tasks.workunit.client.1.smithi149.stdout:7/653: dwrite f1 [4194304,4194304] 0 2024-04-24T08:24:55.096 INFO:tasks.workunit.client.1.smithi149.stdout:2/585: symlink d0/d6/db/d28/lad 0 2024-04-24T08:24:55.100 INFO:tasks.workunit.client.0.smithi090.stdout:1/352: creat d3/d5/d3e/f7c x:0 0 0 2024-04-24T08:24:55.100 INFO:tasks.workunit.client.0.smithi090.stdout:0/378: chown d1a/d4b/l4e 1 1 2024-04-24T08:24:55.101 INFO:tasks.workunit.client.1.smithi149.stdout:2/586: write d0/d6/db/d39/d3d/d44/d57/f71 [178073,71784] 0 2024-04-24T08:24:55.101 INFO:tasks.workunit.client.0.smithi090.stdout:4/250: link d3/df/l31 d3/d25/l48 0 2024-04-24T08:24:55.102 INFO:tasks.workunit.client.0.smithi090.stdout:4/251: fsync d3/d38/f42 0 2024-04-24T08:24:55.102 INFO:tasks.workunit.client.1.smithi149.stdout:7/654: dread d9/dc/de/d44/fab [0,4194304] 0 2024-04-24T08:24:55.104 INFO:tasks.workunit.client.1.smithi149.stdout:9/696: creat d1/de/d12/ffc x:0 0 0 2024-04-24T08:24:55.106 INFO:tasks.workunit.client.1.smithi149.stdout:7/655: write d9/dc/de/d44/fbe [809730,16976] 0 2024-04-24T08:24:55.106 INFO:tasks.workunit.client.1.smithi149.stdout:8/578: unlink d1/de/d18/d32/d65/f66 0 2024-04-24T08:24:55.107 INFO:tasks.workunit.client.1.smithi149.stdout:9/697: chown d1/da/d4a 5 1 2024-04-24T08:24:55.108 INFO:tasks.workunit.client.1.smithi149.stdout:2/587: chown d0/d6/db/d28/f6f 2 1 2024-04-24T08:24:55.112 INFO:tasks.workunit.client.1.smithi149.stdout:1/773: creat d0/ddf/d53/fff x:0 0 0 2024-04-24T08:24:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:1/774: stat d0/d5/db/d18/l9c 0 2024-04-24T08:24:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/766: dread db/f20 [0,4194304] 0 2024-04-24T08:24:55.117 INFO:tasks.workunit.client.1.smithi149.stdout:1/775: fdatasync d0/d5/f9 0 2024-04-24T08:24:55.117 INFO:tasks.workunit.client.0.smithi090.stdout:4/252: mkdir d3/d38/d49 0 2024-04-24T08:24:55.119 INFO:tasks.workunit.client.1.smithi149.stdout:3/734: truncate d0/d5/f16 697169 0 2024-04-24T08:24:55.126 INFO:tasks.workunit.client.0.smithi090.stdout:0/379: unlink d1a/f5f 0 2024-04-24T08:24:55.130 INFO:tasks.workunit.client.0.smithi090.stdout:5/399: rename d8/da/d3a/l5c to d8/d59/l8d 0 2024-04-24T08:24:55.132 INFO:tasks.workunit.client.1.smithi149.stdout:6/582: write d0/f17 [5122772,82315] 0 2024-04-24T08:24:55.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/253: rename d3/d25/f3a to d3/d18/d34/f4a 0 2024-04-24T08:24:55.135 INFO:tasks.workunit.client.1.smithi149.stdout:6/583: chown d0/d1a/d28/d4a/f77 12 1 2024-04-24T08:24:55.138 INFO:tasks.workunit.client.0.smithi090.stdout:1/353: dread d3/d1d/d34/f6b [0,4194304] 0 2024-04-24T08:24:55.138 INFO:tasks.workunit.client.1.smithi149.stdout:3/735: dwrite d0/d88/fbe [0,4194304] 0 2024-04-24T08:24:55.140 INFO:tasks.workunit.client.1.smithi149.stdout:6/584: write d0/d1a/d28/d31/d4c/d4e/f8d [151760,123155] 0 2024-04-24T08:24:55.142 INFO:tasks.workunit.client.1.smithi149.stdout:2/588: rename d0/f8 to d0/d6/fae 0 2024-04-24T08:24:55.143 INFO:tasks.workunit.client.1.smithi149.stdout:3/736: readlink d0/d5/da/d5a/l74 0 2024-04-24T08:24:55.145 INFO:tasks.workunit.client.0.smithi090.stdout:5/400: truncate d8/da/d18/f1f 880480 0 2024-04-24T08:24:55.151 INFO:tasks.workunit.client.1.smithi149.stdout:8/579: dread d1/d1f/f28 [0,4194304] 0 2024-04-24T08:24:55.154 INFO:tasks.workunit.client.0.smithi090.stdout:1/354: write d3/d1d/d34/f6b [1139835,90626] 0 2024-04-24T08:24:55.154 INFO:tasks.workunit.client.1.smithi149.stdout:8/580: write d1/d1f/f8d [3631995,84338] 0 2024-04-24T08:24:55.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/355: fdatasync d3/d1d/d34/d5a/f5b 0 2024-04-24T08:24:55.158 INFO:tasks.workunit.client.1.smithi149.stdout:9/698: mkdir d1/de/d9d/dfd 0 2024-04-24T08:24:55.161 INFO:tasks.workunit.client.1.smithi149.stdout:9/699: chown d1/d2/dc8/dc9/lda 0 1 2024-04-24T08:24:55.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/380: write d1a/f52 [492968,80702] 0 2024-04-24T08:24:55.167 INFO:tasks.workunit.client.1.smithi149.stdout:6/585: dwrite d0/d1a/d28/d31/d4c/d4e/f96 [0,4194304] 0 2024-04-24T08:24:55.167 INFO:tasks.workunit.client.1.smithi149.stdout:7/656: creat d9/dc/d2e/d3c/db8/fdb x:0 0 0 2024-04-24T08:24:55.168 INFO:tasks.workunit.client.1.smithi149.stdout:9/700: truncate d1/da/d13/d74/dbb/fcd 564089 0 2024-04-24T08:24:55.169 INFO:tasks.workunit.client.0.smithi090.stdout:4/254: getdents d3/d38 0 2024-04-24T08:24:55.171 INFO:tasks.workunit.client.0.smithi090.stdout:5/401: dwrite d8/d56/d6d/d7a/d50/f71 [4194304,4194304] 0 2024-04-24T08:24:55.174 INFO:tasks.workunit.client.1.smithi149.stdout:1/776: creat d0/ddf/da0/db8/dec/f100 x:0 0 0 2024-04-24T08:24:55.175 INFO:tasks.workunit.client.0.smithi090.stdout:4/255: mkdir d3/d18/d1d/d4b 0 2024-04-24T08:24:55.175 INFO:tasks.workunit.client.1.smithi149.stdout:9/701: write d1/d2/d33/d5d/d83/dbf/fc1 [159233,4552] 0 2024-04-24T08:24:55.176 INFO:tasks.workunit.client.1.smithi149.stdout:2/589: rename d0/d6/db/d27/d64/f76 to d0/d6/db/d39/d3d/d58/d98/faf 0 2024-04-24T08:24:55.181 INFO:tasks.workunit.client.0.smithi090.stdout:0/381: creat d1a/d4b/d53/d79/d35/d62/f7a x:0 0 0 2024-04-24T08:24:55.184 INFO:tasks.workunit.client.0.smithi090.stdout:1/356: dwrite d3/d1d/d34/d5a/f5e [0,4194304] 0 2024-04-24T08:24:55.184 INFO:tasks.workunit.client.1.smithi149.stdout:8/581: creat d1/de/d18/d91/fc5 x:0 0 0 2024-04-24T08:24:55.185 INFO:tasks.workunit.client.0.smithi090.stdout:0/382: chown d1a/d4b/d53/d79/c29 934 1 2024-04-24T08:24:55.188 INFO:tasks.workunit.client.0.smithi090.stdout:4/256: readlink d3/d25/l48 0 2024-04-24T08:24:55.196 INFO:tasks.workunit.client.1.smithi149.stdout:5/767: dwrite db/d12/fb0 [4194304,4194304] 0 2024-04-24T08:24:55.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/737: creat d0/d5/da/d5a/d63/d6a/dd9/ffe x:0 0 0 2024-04-24T08:24:55.199 INFO:tasks.workunit.client.1.smithi149.stdout:2/590: mkdir d0/d6/db/d28/db0 0 2024-04-24T08:24:55.200 INFO:tasks.workunit.client.1.smithi149.stdout:6/586: dwrite d0/d1a/d28/d31/d33/f9f [0,4194304] 0 2024-04-24T08:24:55.205 INFO:tasks.workunit.client.0.smithi090.stdout:5/402: creat d8/d56/d6d/d7a/d50/f8e x:0 0 0 2024-04-24T08:24:55.207 INFO:tasks.workunit.client.1.smithi149.stdout:9/702: write d1/da/dd/f32 [399894,50221] 0 2024-04-24T08:24:55.216 INFO:tasks.workunit.client.1.smithi149.stdout:8/582: dwrite d1/d1f/d25/f34 [0,4194304] 0 2024-04-24T08:24:55.217 INFO:tasks.workunit.client.0.smithi090.stdout:6/341: sync 2024-04-24T08:24:55.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/278: sync 2024-04-24T08:24:55.218 INFO:tasks.workunit.client.0.smithi090.stdout:9/315: sync 2024-04-24T08:24:55.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/302: sync 2024-04-24T08:24:55.220 INFO:tasks.workunit.client.0.smithi090.stdout:3/303: chown d2/df 4300913 1 2024-04-24T08:24:55.221 INFO:tasks.workunit.client.0.smithi090.stdout:3/304: chown d2/d25/d33/c46 218 1 2024-04-24T08:24:55.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/342: dread d8/f1b [0,4194304] 0 2024-04-24T08:24:55.224 INFO:tasks.workunit.client.1.smithi149.stdout:7/657: write d9/dc/f18 [3052956,118121] 0 2024-04-24T08:24:55.226 INFO:tasks.workunit.client.1.smithi149.stdout:5/768: unlink db/d1f/f30 0 2024-04-24T08:24:55.227 INFO:tasks.workunit.client.1.smithi149.stdout:7/658: readlink d9/dc/l13 0 2024-04-24T08:24:55.232 INFO:tasks.workunit.client.0.smithi090.stdout:1/357: getdents d3/d5/da/dc 0 2024-04-24T08:24:55.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/279: rmdir d2/da/d13/d15/d50 39 2024-04-24T08:24:55.234 INFO:tasks.workunit.client.0.smithi090.stdout:4/257: write d3/df/f36 [554423,8001] 0 2024-04-24T08:24:55.240 INFO:tasks.workunit.client.0.smithi090.stdout:6/343: rename d8/d1d/d2f/c63 to d8/d1d/d1f/d3e/d4f/c66 0 2024-04-24T08:24:55.241 INFO:tasks.workunit.client.1.smithi149.stdout:5/769: creat db/d12/d1e/d65/de8/f100 x:0 0 0 2024-04-24T08:24:55.243 INFO:tasks.workunit.client.1.smithi149.stdout:7/659: creat d9/dc/d2e/d3c/db8/fdc x:0 0 0 2024-04-24T08:24:55.244 INFO:tasks.workunit.client.0.smithi090.stdout:0/383: dwrite ff [0,4194304] 0 2024-04-24T08:24:55.245 INFO:tasks.workunit.client.1.smithi149.stdout:1/777: dwrite d0/d5/d4f/d60/f78 [0,4194304] 0 2024-04-24T08:24:55.246 INFO:tasks.workunit.client.0.smithi090.stdout:8/280: chown d2/da/d13/d15/d50 6 1 2024-04-24T08:24:55.247 INFO:tasks.workunit.client.1.smithi149.stdout:2/591: mkdir d0/d6/db/d27/d64/d66/db1 0 2024-04-24T08:24:55.248 INFO:tasks.workunit.client.0.smithi090.stdout:6/344: chown d8/l35 11327 1 2024-04-24T08:24:55.251 INFO:tasks.workunit.client.0.smithi090.stdout:9/316: dwrite d0/d19/f56 [0,4194304] 0 2024-04-24T08:24:55.252 INFO:tasks.workunit.client.1.smithi149.stdout:2/592: truncate d0/d6/db/d39/d3d/d44/f96 116090 0 2024-04-24T08:24:55.252 INFO:tasks.workunit.client.0.smithi090.stdout:8/281: write d2/f3 [5990684,104837] 0 2024-04-24T08:24:55.253 INFO:tasks.workunit.client.0.smithi090.stdout:4/258: rename d3/f16 to d3/d38/d49/f4c 0 2024-04-24T08:24:55.254 INFO:tasks.workunit.client.1.smithi149.stdout:1/778: dread d0/d1b/d4b/da1/ff5 [0,4194304] 0 2024-04-24T08:24:55.257 INFO:tasks.workunit.client.1.smithi149.stdout:0/605: sync 2024-04-24T08:24:55.257 INFO:tasks.workunit.client.1.smithi149.stdout:4/603: sync 2024-04-24T08:24:55.258 INFO:tasks.workunit.client.1.smithi149.stdout:8/583: write d1/d1f/d5e/f8c [822967,116959] 0 2024-04-24T08:24:55.259 INFO:tasks.workunit.client.1.smithi149.stdout:3/738: fsync d0/d5/f16 0 2024-04-24T08:24:55.262 INFO:tasks.workunit.client.1.smithi149.stdout:5/770: symlink db/d12/d2f/d95/l101 0 2024-04-24T08:24:55.262 INFO:tasks.workunit.client.0.smithi090.stdout:3/305: dwrite d2/df/d1a/f28 [4194304,4194304] 0 2024-04-24T08:24:55.264 INFO:tasks.workunit.client.1.smithi149.stdout:7/660: symlink d9/dc/d11/d84/ldd 0 2024-04-24T08:24:55.265 INFO:tasks.workunit.client.0.smithi090.stdout:6/345: mkdir d8/d1d/d2f/d67 0 2024-04-24T08:24:55.268 INFO:tasks.workunit.client.0.smithi090.stdout:0/384: write d1a/d4b/d53/d79/d35/d62/f6e [177186,20448] 0 2024-04-24T08:24:55.271 INFO:tasks.workunit.client.1.smithi149.stdout:9/703: link d1/da/l9a d1/d99/lfe 0 2024-04-24T08:24:55.273 INFO:tasks.workunit.client.0.smithi090.stdout:1/358: dwrite d3/d5/da/f64 [0,4194304] 0 2024-04-24T08:24:55.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/593: rmdir d0/d6/db/d39/d3d/d58/d98 39 2024-04-24T08:24:55.277 INFO:tasks.workunit.client.0.smithi090.stdout:9/317: dread d0/dc/f3e [0,4194304] 0 2024-04-24T08:24:55.277 INFO:tasks.workunit.client.0.smithi090.stdout:0/385: fsync d1a/d4b/d53/d79/f32 0 2024-04-24T08:24:55.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/403: dwrite d8/f9 [4194304,4194304] 0 2024-04-24T08:24:55.284 INFO:tasks.workunit.client.0.smithi090.stdout:9/318: write d0/f1b [3071425,97051] 0 2024-04-24T08:24:55.287 INFO:tasks.workunit.client.0.smithi090.stdout:4/259: dread d3/f3f [0,4194304] 0 2024-04-24T08:24:55.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/584: rmdir d1/d1f/d5e/d92 39 2024-04-24T08:24:55.289 INFO:tasks.workunit.client.0.smithi090.stdout:4/260: fsync d3/d18/d1d/f40 0 2024-04-24T08:24:55.290 INFO:tasks.workunit.client.0.smithi090.stdout:7/398: sync 2024-04-24T08:24:55.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/604: rename d2/d10/d1a/d34/f37 to d2/d10/d1a/fbe 0 2024-04-24T08:24:55.296 INFO:tasks.workunit.client.1.smithi149.stdout:4/605: chown d2/d10/d1a/f3e 33100224 1 2024-04-24T08:24:55.297 INFO:tasks.workunit.client.1.smithi149.stdout:5/771: unlink db/d12/d1e/d85/ff2 0 2024-04-24T08:24:55.299 INFO:tasks.workunit.client.1.smithi149.stdout:6/587: link d0/d1a/c1c d0/d1a/d28/d31/d38/d54/d87/cc3 0 2024-04-24T08:24:55.300 INFO:tasks.workunit.client.1.smithi149.stdout:4/606: dread - d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf zero size 2024-04-24T08:24:55.300 INFO:tasks.workunit.client.0.smithi090.stdout:3/306: creat d2/d15/d36/f5f x:0 0 0 2024-04-24T08:24:55.302 INFO:tasks.workunit.client.0.smithi090.stdout:3/307: write d2/df/d1a/f4c [1767628,48833] 0 2024-04-24T08:24:55.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/346: rename d8/d1d/l51 to d8/d1d/d29/l68 0 2024-04-24T08:24:55.303 INFO:tasks.workunit.client.1.smithi149.stdout:7/661: creat d9/dc/d2e/d3c/db8/fde x:0 0 0 2024-04-24T08:24:55.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/704: rmdir d1/da/dd/d55/d72 39 2024-04-24T08:24:55.306 INFO:tasks.workunit.client.0.smithi090.stdout:1/359: creat d3/d25/d78/f7d x:0 0 0 2024-04-24T08:24:55.310 INFO:tasks.workunit.client.0.smithi090.stdout:0/386: rmdir d1a/d4b/d53/d79/d35 39 2024-04-24T08:24:55.311 INFO:tasks.workunit.client.1.smithi149.stdout:0/606: mknod d2/d17/cc3 0 2024-04-24T08:24:55.314 INFO:tasks.workunit.client.0.smithi090.stdout:5/404: creat d8/d25/f8f x:0 0 0 2024-04-24T08:24:55.317 INFO:tasks.workunit.client.1.smithi149.stdout:8/585: creat d1/de/d18/d91/d9f/fc6 x:0 0 0 2024-04-24T08:24:55.318 INFO:tasks.workunit.client.1.smithi149.stdout:3/739: fsync d0/d5/d53/f47 0 2024-04-24T08:24:55.320 INFO:tasks.workunit.client.0.smithi090.stdout:4/261: creat d3/d38/f4d x:0 0 0 2024-04-24T08:24:55.321 INFO:tasks.workunit.client.1.smithi149.stdout:3/740: chown d0/d5/d1a/d70/c82 101604 1 2024-04-24T08:24:55.323 INFO:tasks.workunit.client.1.smithi149.stdout:5/772: rmdir db/d12/d22/d45 39 2024-04-24T08:24:55.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.323+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc78c000bf0 con 0x7fc79406eac0 2024-04-24T08:24:55.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.325+0000 7fc7aa000700 1 -- 172.21.15.90:0/842269599 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fc78c000bf0 con 0x7fc79406eac0 2024-04-24T08:24:55.326 INFO:tasks.workunit.client.0.smithi090.stdout:6/347: dwrite d8/d1d/d1f/d3e/f60 [0,4194304] 0 2024-04-24T08:24:55.326 INFO:tasks.workunit.client.1.smithi149.stdout:4/607: readlink d2/l45 0 2024-04-24T08:24:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.327+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 msgr2=0x7fc794070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.327+0000 7fc7b4000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 0x7fc794070f70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc798005fb0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.327+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 msgr2=0x7fc7ac071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.327+0000 7fc7b4000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d8d0 tx=0x7fc7a000dbe0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 shutdown_connections 2024-04-24T08:24:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc79406eac0 0x7fc794070f70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7ac0737b0 0x7fc7ac071d50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 --2- 172.21.15.90:0/842269599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7ac074d80 0x7fc7ac072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 >> 172.21.15.90:0/842269599 conn(0x7fc7ac0fb7f0 msgr2=0x7fc7ac101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 shutdown_connections 2024-04-24T08:24:55.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.328+0000 7fc7b4000700 1 -- 172.21.15.90:0/842269599 wait complete. 2024-04-24T08:24:55.331 INFO:tasks.workunit.client.1.smithi149.stdout:4/608: write d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 [176301,51356] 0 2024-04-24T08:24:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:9/705: dread d1/d2/f97 [0,4194304] 0 2024-04-24T08:24:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:7/662: rename d9/dc/d11/d38/c8a to d9/dc/d7b/cdf 0 2024-04-24T08:24:55.336 INFO:tasks.workunit.client.0.smithi090.stdout:1/360: creat d3/d25/d30/f7e x:0 0 0 2024-04-24T08:24:55.338 INFO:tasks.workunit.client.1.smithi149.stdout:7/663: truncate d9/dc/f6b 1209704 0 2024-04-24T08:24:55.338 INFO:tasks.workunit.client.1.smithi149.stdout:1/779: write d0/d5/db/d18/d2f/fa4 [2278984,66899] 0 2024-04-24T08:24:55.338 INFO:tasks.workunit.client.1.smithi149.stdout:2/594: dread d0/d6/db/f72 [0,4194304] 0 2024-04-24T08:24:55.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/399: dread db/d17/d29/f42 [0,4194304] 0 2024-04-24T08:24:55.341 INFO:tasks.workunit.client.0.smithi090.stdout:5/405: symlink d8/d25/l90 0 2024-04-24T08:24:55.342 INFO:tasks.workunit.client.1.smithi149.stdout:6/588: getdents d0/d1a/d28/d31/d4c/d4e/dba 0 2024-04-24T08:24:55.344 INFO:tasks.workunit.client.0.smithi090.stdout:7/400: readlink db/d17/d39/d59/l74 0 2024-04-24T08:24:55.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/406: dread - d8/d56/d6d/d7b/f80 zero size 2024-04-24T08:24:55.345 INFO:tasks.workunit.client.0.smithi090.stdout:4/262: creat d3/d18/d1d/f4e x:0 0 0 2024-04-24T08:24:55.346 INFO:tasks.workunit.client.0.smithi090.stdout:4/263: stat d3/d18/d1d/l3b 0 2024-04-24T08:24:55.349 INFO:tasks.workunit.client.1.smithi149.stdout:4/609: symlink d2/d18/d1e/d3b/lbf 0 2024-04-24T08:24:55.349 INFO:tasks.workunit.client.0.smithi090.stdout:4/264: write d3/d18/d1d/f40 [1782154,68788] 0 2024-04-24T08:24:55.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/407: truncate d8/da/d14/d2f/f66 259695 0 2024-04-24T08:24:55.349 INFO:tasks.workunit.client.0.smithi090.stdout:8/282: link d2/da/d13/d34/f4d d2/da/d13/f5f 0 2024-04-24T08:24:55.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/265: chown d3/d18 69 1 2024-04-24T08:24:55.351 INFO:tasks.workunit.client.0.smithi090.stdout:6/348: symlink d8/d1d/d29/l69 0 2024-04-24T08:24:55.353 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:24:55.354 INFO:tasks.workunit.client.0.smithi090.stdout:1/361: unlink d3/d5/da/dc/f50 0 2024-04-24T08:24:55.356 INFO:tasks.workunit.client.0.smithi090.stdout:0/387: creat d1a/d4b/d53/d79/d35/d68/f7b x:0 0 0 2024-04-24T08:24:55.357 INFO:tasks.workunit.client.1.smithi149.stdout:0/607: mkdir d2/d9/df/d48/dc4 0 2024-04-24T08:24:55.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/362: write d3/d5/f17 [4604130,13338] 0 2024-04-24T08:24:55.358 INFO:tasks.workunit.client.1.smithi149.stdout:3/741: dwrite d0/d5/d53/d6d/fea [0,4194304] 0 2024-04-24T08:24:55.362 INFO:tasks.workunit.client.1.smithi149.stdout:6/589: fdatasync d0/f12 0 2024-04-24T08:24:55.362 INFO:tasks.workunit.client.0.smithi090.stdout:7/401: fsync db/d17/d29/d32/d5a/d75/f81 0 2024-04-24T08:24:55.367 INFO:tasks.workunit.client.0.smithi090.stdout:8/283: dwrite d2/da/d13/d15/d50/f5d [0,4194304] 0 2024-04-24T08:24:55.370 INFO:tasks.workunit.client.0.smithi090.stdout:2/336: sync 2024-04-24T08:24:55.375 INFO:tasks.workunit.client.0.smithi090.stdout:9/319: getdents d0/d19/d33/d50 0 2024-04-24T08:24:55.375 INFO:tasks.workunit.client.0.smithi090.stdout:3/308: dread d2/fa [0,4194304] 0 2024-04-24T08:24:55.376 INFO:tasks.workunit.client.1.smithi149.stdout:0/608: dwrite d2/d3c/d57/f78 [0,4194304] 0 2024-04-24T08:24:55.384 INFO:tasks.workunit.client.0.smithi090.stdout:4/266: dwrite d3/d38/d49/f4c [4194304,4194304] 0 2024-04-24T08:24:55.388 INFO:tasks.workunit.client.0.smithi090.stdout:7/402: unlink db/d17/d29/d32/c5f 0 2024-04-24T08:24:55.394 INFO:tasks.workunit.client.0.smithi090.stdout:1/363: dread d3/f41 [0,4194304] 0 2024-04-24T08:24:55.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/388: truncate d1a/d4b/d53/d79/d35/f74 727330 0 2024-04-24T08:24:55.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/408: creat d8/d56/d6d/d7a/d50/f91 x:0 0 0 2024-04-24T08:24:55.399 INFO:tasks.workunit.client.0.smithi090.stdout:4/267: creat d3/d38/d49/f4f x:0 0 0 2024-04-24T08:24:55.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/403: creat db/d17/d29/d32/d5a/f8d x:0 0 0 2024-04-24T08:24:55.405 INFO:tasks.workunit.client.0.smithi090.stdout:9/320: dwrite d0/d19/f28 [4194304,4194304] 0 2024-04-24T08:24:55.406 INFO:tasks.workunit.client.0.smithi090.stdout:8/284: mkdir d2/da/d13/d60 0 2024-04-24T08:24:55.407 INFO:tasks.workunit.client.0.smithi090.stdout:7/404: chown db/f21 1 1 2024-04-24T08:24:55.408 INFO:tasks.workunit.client.0.smithi090.stdout:8/285: dread - d2/da/f38 zero size 2024-04-24T08:24:55.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/337: truncate d0/d14/f1d 3309799 0 2024-04-24T08:24:55.412 INFO:tasks.workunit.client.0.smithi090.stdout:0/389: mknod d1a/d4b/d53/d79/d35/d68/c7c 0 2024-04-24T08:24:55.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/409: unlink d8/d56/d6d/d7a/d50/f91 0 2024-04-24T08:24:55.415 INFO:tasks.workunit.client.0.smithi090.stdout:1/364: mkdir d3/d68/d7f 0 2024-04-24T08:24:55.416 INFO:tasks.workunit.client.1.smithi149.stdout:9/706: mkdir d1/d2/d33/d5d/d83/df4/dff 0 2024-04-24T08:24:55.418 INFO:tasks.workunit.client.0.smithi090.stdout:5/410: write d8/da/d3a/f43 [28171,59857] 0 2024-04-24T08:24:55.419 INFO:tasks.workunit.client.1.smithi149.stdout:9/707: write d1/d2/d15/f81 [1719672,71925] 0 2024-04-24T08:24:55.420 INFO:tasks.workunit.client.1.smithi149.stdout:9/708: readlink d1/d99/lf6 0 2024-04-24T08:24:55.422 INFO:tasks.workunit.client.1.smithi149.stdout:9/709: dread - d1/de/d12/ffc zero size 2024-04-24T08:24:55.423 INFO:tasks.workunit.client.0.smithi090.stdout:8/286: read - d2/da/d13/d15/d24/f46 zero size 2024-04-24T08:24:55.424 INFO:tasks.workunit.client.0.smithi090.stdout:2/338: mknod d0/d14/d36/c6b 0 2024-04-24T08:24:55.425 INFO:tasks.workunit.client.0.smithi090.stdout:4/268: dwrite d3/d18/d1d/f2a [0,4194304] 0 2024-04-24T08:24:55.426 INFO:tasks.workunit.client.0.smithi090.stdout:5/411: chown d8/da/d14/d2f/d44/f73 98298 1 2024-04-24T08:24:55.428 INFO:tasks.workunit.client.0.smithi090.stdout:3/309: getdents d2/d25/d2c 0 2024-04-24T08:24:55.429 INFO:tasks.workunit.client.1.smithi149.stdout:6/590: creat d0/d1a/d28/d31/d38/d54/d87/fc4 x:0 0 0 2024-04-24T08:24:55.429 INFO:tasks.workunit.client.0.smithi090.stdout:2/339: write d0/d14/f34 [823154,11247] 0 2024-04-24T08:24:55.433 INFO:tasks.workunit.client.1.smithi149.stdout:6/591: write d0/d1a/d28/d31/d33/f9f [1047765,13016] 0 2024-04-24T08:24:55.436 INFO:tasks.workunit.client.1.smithi149.stdout:8/586: rename d1/de/d18/d32/d63/d9e/fba to d1/de/d18/fc7 0 2024-04-24T08:24:55.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/365: rename d3/d25/d78 to d3/d5/da/dc/d80 0 2024-04-24T08:24:55.441 INFO:tasks.workunit.client.0.smithi090.stdout:6/349: write d8/dc/f10 [1798123,113209] 0 2024-04-24T08:24:55.447 INFO:tasks.workunit.client.0.smithi090.stdout:0/390: symlink d1a/l7d 0 2024-04-24T08:24:55.447 INFO:tasks.workunit.client.0.smithi090.stdout:9/321: creat d0/f59 x:0 0 0 2024-04-24T08:24:55.448 INFO:tasks.workunit.client.0.smithi090.stdout:6/350: dread - d8/f46 zero size 2024-04-24T08:24:55.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/340: dread d0/d14/f38 [4194304,4194304] 0 2024-04-24T08:24:55.453 INFO:tasks.workunit.client.0.smithi090.stdout:9/322: chown d0/d19/l31 115679798 1 2024-04-24T08:24:55.453 INFO:tasks.workunit.client.0.smithi090.stdout:3/310: unlink d2/d15/d36/c56 0 2024-04-24T08:24:55.454 INFO:tasks.workunit.client.0.smithi090.stdout:4/269: dwrite d3/f32 [0,4194304] 0 2024-04-24T08:24:55.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/391: readlink d1a/d4b/d53/d79/d3c/l46 0 2024-04-24T08:24:55.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/710: mkdir d1/de/d9d/dfd/d100 0 2024-04-24T08:24:55.461 INFO:tasks.workunit.client.1.smithi149.stdout:3/742: link d0/d5/da/d5a/l74 d0/d5/da/d19/d1b/d22/d38/lff 0 2024-04-24T08:24:55.463 INFO:tasks.workunit.client.1.smithi149.stdout:1/780: write d0/d5/db/d18/d4d/f90 [1022101,120925] 0 2024-04-24T08:24:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:1/781: dread d0/ddf/da0/f9e [0,4194304] 0 2024-04-24T08:24:55.468 INFO:tasks.workunit.client.1.smithi149.stdout:5/773: rename db/d12/d1e/c3b to db/d12/d1e/d65/c102 0 2024-04-24T08:24:55.471 INFO:tasks.workunit.client.1.smithi149.stdout:2/595: truncate d0/d6/fa0 2381219 0 2024-04-24T08:24:55.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/412: dread f6 [0,4194304] 0 2024-04-24T08:24:55.474 INFO:tasks.workunit.client.1.smithi149.stdout:2/596: chown d0/d6/db/d39/d78/d7c/f97 679021618 1 2024-04-24T08:24:55.475 INFO:tasks.workunit.client.0.smithi090.stdout:9/323: mknod d0/dc/d24/d2d/c5a 0 2024-04-24T08:24:55.475 INFO:tasks.workunit.client.0.smithi090.stdout:5/413: write d8/da/f35 [40704,75049] 0 2024-04-24T08:24:55.477 INFO:tasks.workunit.client.1.smithi149.stdout:6/592: creat d0/d1a/d28/d31/fc5 x:0 0 0 2024-04-24T08:24:55.479 INFO:tasks.workunit.client.1.smithi149.stdout:9/711: dwrite d1/da/dd/f32 [0,4194304] 0 2024-04-24T08:24:55.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/405: dwrite db/d17/d29/d38/f44 [0,4194304] 0 2024-04-24T08:24:55.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/270: dwrite d3/d38/d49/f4f [0,4194304] 0 2024-04-24T08:24:55.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/774: write db/d1f/d7f/fed [4795902,115139] 0 2024-04-24T08:24:55.487 INFO:tasks.workunit.client.1.smithi149.stdout:5/775: write db/d12/d2f/d74/dc6/fe9 [960384,110026] 0 2024-04-24T08:24:55.487 INFO:tasks.workunit.client.1.smithi149.stdout:2/597: mknod d0/d6/db/d28/d7f/cb2 0 2024-04-24T08:24:55.490 INFO:tasks.workunit.client.1.smithi149.stdout:5/776: write db/d12/d60/f67 [3901020,99565] 0 2024-04-24T08:24:55.493 INFO:tasks.workunit.client.1.smithi149.stdout:7/664: rename d9/d43/l88 to d9/dc/d2e/d3c/db8/le0 0 2024-04-24T08:24:55.497 INFO:tasks.workunit.client.1.smithi149.stdout:1/782: mkdir d0/d1b/d5d/de5/d101 0 2024-04-24T08:24:55.499 INFO:tasks.workunit.client.0.smithi090.stdout:0/392: fdatasync d1a/d4b/d53/f6a 0 2024-04-24T08:24:55.504 INFO:tasks.workunit.client.1.smithi149.stdout:9/712: fdatasync d1/da/f27 0 2024-04-24T08:24:55.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/271: dread d3/d18/d1d/f46 [0,4194304] 0 2024-04-24T08:24:55.508 INFO:tasks.workunit.client.0.smithi090.stdout:6/351: dwrite d8/d1d/d2f/d30/d38/f49 [4194304,4194304] 0 2024-04-24T08:24:55.510 INFO:tasks.workunit.client.1.smithi149.stdout:8/587: dread d1/f8e [0,4194304] 0 2024-04-24T08:24:55.513 INFO:tasks.workunit.client.0.smithi090.stdout:1/366: truncate d3/d5/da/dc/f3a 4117372 0 2024-04-24T08:24:55.515 INFO:tasks.workunit.client.0.smithi090.stdout:1/367: chown d3/f4 364 1 2024-04-24T08:24:55.516 INFO:tasks.workunit.client.0.smithi090.stdout:2/341: truncate d0/d7/f32 7949295 0 2024-04-24T08:24:55.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/610: rename d2/cd to d2/d18/d1e/d3c/d48/d69/cc0 0 2024-04-24T08:24:55.519 INFO:tasks.workunit.client.1.smithi149.stdout:0/609: rename d2/d9/df/d2f/d6c/dc1 to d2/d9/df/d2f/d6c/dc1/dc5 22 2024-04-24T08:24:55.522 INFO:tasks.workunit.client.0.smithi090.stdout:9/324: mkdir d0/dc/d24/d3d/d5b 0 2024-04-24T08:24:55.524 INFO:tasks.workunit.client.1.smithi149.stdout:1/783: creat d0/d1b/d4b/f102 x:0 0 0 2024-04-24T08:24:55.525 INFO:tasks.workunit.client.0.smithi090.stdout:2/342: read d0/d2c/f43 [358052,122733] 0 2024-04-24T08:24:55.526 INFO:tasks.workunit.client.0.smithi090.stdout:8/287: rename d2/l2a to d2/da/d13/l61 0 2024-04-24T08:24:55.527 INFO:tasks.workunit.client.1.smithi149.stdout:9/713: creat d1/de/d9d/dfd/f101 x:0 0 0 2024-04-24T08:24:55.528 INFO:tasks.workunit.client.1.smithi149.stdout:6/593: dwrite d0/f1 [4194304,4194304] 0 2024-04-24T08:24:55.530 INFO:tasks.workunit.client.1.smithi149.stdout:9/714: chown d1/da/dd/c1b 180 1 2024-04-24T08:24:55.531 INFO:tasks.workunit.client.1.smithi149.stdout:2/598: mkdir d0/d6/db/d27/d64/db3 0 2024-04-24T08:24:55.531 INFO:tasks.workunit.client.0.smithi090.stdout:0/393: truncate d1a/f64 755422 0 2024-04-24T08:24:55.532 INFO:tasks.workunit.client.0.smithi090.stdout:0/394: chown d1a/d4b 0 1 2024-04-24T08:24:55.532 INFO:tasks.workunit.client.0.smithi090.stdout:4/272: fdatasync d3/d18/d1d/f27 0 2024-04-24T08:24:55.534 INFO:tasks.workunit.client.0.smithi090.stdout:6/352: stat d8/dc/c18 0 2024-04-24T08:24:55.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/273: chown d3/df/l31 30577 1 2024-04-24T08:24:55.536 INFO:tasks.workunit.client.1.smithi149.stdout:2/599: chown d0/d6/db/d27/d64/l84 364348 1 2024-04-24T08:24:55.537 INFO:tasks.workunit.client.1.smithi149.stdout:0/610: creat d2/d3c/fc6 x:0 0 0 2024-04-24T08:24:55.539 INFO:tasks.workunit.client.1.smithi149.stdout:0/611: chown d2/fb 34 1 2024-04-24T08:24:55.539 INFO:tasks.workunit.client.1.smithi149.stdout:1/784: creat d0/ddf/da0/db8/dec/f103 x:0 0 0 2024-04-24T08:24:55.542 INFO:tasks.workunit.client.1.smithi149.stdout:9/715: mknod d1/da/d4a/c102 0 2024-04-24T08:24:55.545 INFO:tasks.workunit.client.0.smithi090.stdout:1/368: rename d3/d1d/f37 to d3/d5/da/dc/f81 0 2024-04-24T08:24:55.547 INFO:tasks.workunit.client.0.smithi090.stdout:3/311: truncate d2/df/d21/f32 2537801 0 2024-04-24T08:24:55.547 INFO:tasks.workunit.client.1.smithi149.stdout:6/594: creat d0/d1a/d28/d31/d38/fc6 x:0 0 0 2024-04-24T08:24:55.548 INFO:tasks.workunit.client.1.smithi149.stdout:6/595: readlink d0/d1a/l84 0 2024-04-24T08:24:55.550 INFO:tasks.workunit.client.1.smithi149.stdout:7/665: rename d9/dc/d11/d84/lce to d9/dc/d11/d84/da7/le1 0 2024-04-24T08:24:55.552 INFO:tasks.workunit.client.0.smithi090.stdout:5/414: dwrite d8/d56/d6d/d7a/d50/f5e [0,4194304] 0 2024-04-24T08:24:55.553 INFO:tasks.workunit.client.1.smithi149.stdout:2/600: fdatasync d0/d6/db/f9e 0 2024-04-24T08:24:55.557 INFO:tasks.workunit.client.1.smithi149.stdout:0/612: mkdir d2/d3c/dc7 0 2024-04-24T08:24:55.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/325: dread d0/f1b [0,4194304] 0 2024-04-24T08:24:55.559 INFO:tasks.workunit.client.0.smithi090.stdout:4/274: rmdir d3/df 39 2024-04-24T08:24:55.562 INFO:tasks.workunit.client.0.smithi090.stdout:9/326: write d0/d19/f1c [7843785,12251] 0 2024-04-24T08:24:55.565 INFO:tasks.workunit.client.0.smithi090.stdout:7/406: dwrite f4 [0,4194304] 0 2024-04-24T08:24:55.565 INFO:tasks.workunit.client.0.smithi090.stdout:6/353: creat d8/d1d/d1f/d3e/f6a x:0 0 0 2024-04-24T08:24:55.566 INFO:tasks.workunit.client.1.smithi149.stdout:6/596: read d0/d1a/d28/d31/f6c [625049,116652] 0 2024-04-24T08:24:55.570 INFO:tasks.workunit.client.1.smithi149.stdout:1/785: rename d0/d5/d4f/d60/lf0 to d0/d1b/d5d/de5/d101/l104 0 2024-04-24T08:24:55.572 INFO:tasks.workunit.client.1.smithi149.stdout:4/611: rename d2/c2b to d2/cc1 0 2024-04-24T08:24:55.572 INFO:tasks.workunit.client.0.smithi090.stdout:0/395: dwrite d1a/f30 [0,4194304] 0 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.572+0000 7fea4e000700 1 -- 172.21.15.90:0/855053759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 msgr2=0x7fea48101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.572+0000 7fea4e000700 1 --2- 172.21.15.90:0/855053759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48101890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fea34009910 tx=0x7fea34009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:55.574 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:55 smithi090 ceph-mon[25016]: pgmap v20: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 45 MiB/s rd, 187 MiB/s wr, 322 op/s 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.573+0000 7fea4e000700 1 -- 172.21.15.90:0/855053759 shutdown_connections 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.573+0000 7fea4e000700 1 --2- 172.21.15.90:0/855053759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48101890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.573+0000 7fea4e000700 1 --2- 172.21.15.90:0/855053759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.573+0000 7fea4e000700 1 -- 172.21.15.90:0/855053759 >> 172.21.15.90:0/855053759 conn(0x7fea480fb7f0 msgr2=0x7fea480fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:55.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/288: mkdir d2/d62 0 2024-04-24T08:24:55.576 INFO:tasks.workunit.client.0.smithi090.stdout:0/396: chown d1a/d4b/d53/d79/d35/d68 543855547 1 2024-04-24T08:24:55.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.574+0000 7fea4e000700 1 -- 172.21.15.90:0/855053759 shutdown_connections 2024-04-24T08:24:55.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.574+0000 7fea4e000700 1 -- 172.21.15.90:0/855053759 wait complete. 2024-04-24T08:24:55.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.577+0000 7fea4e000700 1 Processor -- start 2024-04-24T08:24:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:7/666: rmdir d9/dc/d11/d38 39 2024-04-24T08:24:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:4/612: write d2/d10/d91/f9d [848742,122396] 0 2024-04-24T08:24:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:8/588: truncate d1/d1f/d25/f8f 296886 0 2024-04-24T08:24:55.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.577+0000 7fea4e000700 1 -- start start 2024-04-24T08:24:55.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.578+0000 7fea4e000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.578+0000 7fea4e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.578+0000 7fea4e000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48199700 con 0x7fea48101440 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.578+0000 7fea4e000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48199840 con 0x7fea48100240 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.578+0000 7fea46a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.579+0000 7fea47400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.579+0000 7fea47400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:58292/0 (socket says 172.21.15.90:58292) 2024-04-24T08:24:55.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.579+0000 7fea47400700 1 -- 172.21.15.90:0/1662221151 learned_addr learned my addr 172.21.15.90:0/1662221151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:55.581 INFO:tasks.workunit.client.1.smithi149.stdout:7/667: dread - d9/d43/d95/fd0 zero size 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.580+0000 7fea47400700 1 -- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 msgr2=0x7fea48073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.580+0000 7fea47400700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.580+0000 7fea47400700 1 -- 172.21.15.90:0/1662221151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340095f0 con 0x7fea48100240 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.580+0000 7fea47400700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d900 tx=0x7fea3800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.581+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea380041f0 con 0x7fea48100240 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.581+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea38004350 con 0x7fea48100240 2024-04-24T08:24:55.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.581+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48199b20 con 0x7fea48100240 2024-04-24T08:24:55.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.581+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea38003cf0 con 0x7fea48100240 2024-04-24T08:24:55.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.581+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4819a020 con 0x7fea48100240 2024-04-24T08:24:55.583 INFO:tasks.workunit.client.1.smithi149.stdout:4/613: chown d2/d10/d91/d9f 6396865 1 2024-04-24T08:24:55.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.584+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48108d90 con 0x7fea48100240 2024-04-24T08:24:55.587 INFO:tasks.workunit.client.0.smithi090.stdout:6/354: dwrite d8/fa [0,4194304] 0 2024-04-24T08:24:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.584+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7fea38010460 con 0x7fea48100240 2024-04-24T08:24:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.586+0000 7fea3fe00700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 0x7fea2c070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.587+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fea38003680 con 0x7fea48100240 2024-04-24T08:24:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.587+0000 7fea46a00700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 0x7fea2c070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.588+0000 7fea46a00700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 0x7fea2c070f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea34006010 tx=0x7fea3400b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:0/613: rename d2/d9/df/d40/l8c to d2/d9/df/d40/lc8 0 2024-04-24T08:24:55.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.597+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fea3805c650 con 0x7fea48100240 2024-04-24T08:24:55.599 INFO:tasks.workunit.client.1.smithi149.stdout:6/597: creat d0/d1a/d28/d31/d4c/d80/d6f/fc7 x:0 0 0 2024-04-24T08:24:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:8/589: chown d1/de/d18/d91/la2 244255 1 2024-04-24T08:24:55.608 INFO:tasks.workunit.client.1.smithi149.stdout:7/668: creat d9/d43/d4f/fe2 x:0 0 0 2024-04-24T08:24:55.609 INFO:tasks.workunit.client.0.smithi090.stdout:5/415: chown d8/da/d18/d21/d4c/l78 269 1 2024-04-24T08:24:55.612 INFO:tasks.workunit.client.0.smithi090.stdout:6/355: dwrite d8/d1d/d2f/d30/f4b [0,4194304] 0 2024-04-24T08:24:55.613 INFO:tasks.workunit.client.0.smithi090.stdout:4/275: rmdir d3/d25 39 2024-04-24T08:24:55.618 INFO:tasks.workunit.client.0.smithi090.stdout:6/356: write d8/d1d/f24 [6389951,110130] 0 2024-04-24T08:24:55.618 INFO:tasks.workunit.client.1.smithi149.stdout:2/601: dwrite d0/d6/db/d27/f47 [0,4194304] 0 2024-04-24T08:24:55.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/614: rename d2/d9/df/d92 to d2/d9/df/d2f/dc9 0 2024-04-24T08:24:55.623 INFO:tasks.workunit.client.0.smithi090.stdout:9/327: dread d0/f2 [0,4194304] 0 2024-04-24T08:24:55.627 INFO:tasks.workunit.client.1.smithi149.stdout:6/598: rmdir d0/d1a/d28/d31/d4c/d76 39 2024-04-24T08:24:55.631 INFO:tasks.workunit.client.1.smithi149.stdout:8/590: chown d1/l10 0 1 2024-04-24T08:24:55.631 INFO:tasks.workunit.client.1.smithi149.stdout:6/599: chown d0/d1a/d28/d31/f6c 1 1 2024-04-24T08:24:55.634 INFO:tasks.workunit.client.1.smithi149.stdout:1/786: dread d0/d5/db/d63/f6b [0,4194304] 0 2024-04-24T08:24:55.634 INFO:tasks.workunit.client.1.smithi149.stdout:8/591: chown d1/d1f/d5e/l62 65434 1 2024-04-24T08:24:55.635 INFO:tasks.workunit.client.1.smithi149.stdout:7/669: unlink d9/dc/d2e/d51/f7c 0 2024-04-24T08:24:55.638 INFO:tasks.workunit.client.0.smithi090.stdout:2/343: rename d0/d14/d17/l48 to d0/d14/d17/d2a/d31/d46/d60/d65/l6c 0 2024-04-24T08:24:55.640 INFO:tasks.workunit.client.0.smithi090.stdout:2/344: dread - d0/d14/d17/d2a/d31/f52 zero size 2024-04-24T08:24:55.641 INFO:tasks.workunit.client.1.smithi149.stdout:2/602: dwrite d0/d6/db/d39/fa6 [0,4194304] 0 2024-04-24T08:24:55.644 INFO:tasks.workunit.client.0.smithi090.stdout:3/312: mkdir d2/df/d1a/d37/d53/d60 0 2024-04-24T08:24:55.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/615: mkdir d2/d9/df/d56/dca 0 2024-04-24T08:24:55.648 INFO:tasks.workunit.client.0.smithi090.stdout:4/276: symlink d3/d38/d49/l50 0 2024-04-24T08:24:55.651 INFO:tasks.workunit.client.1.smithi149.stdout:7/670: unlink d9/dc/d11/d84/ldd 0 2024-04-24T08:24:55.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/614: creat d2/fc2 x:0 0 0 2024-04-24T08:24:55.655 INFO:tasks.workunit.client.1.smithi149.stdout:9/716: link d1/f88 d1/da/f103 0 2024-04-24T08:24:55.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/289: mkdir d2/da/d13/d63 0 2024-04-24T08:24:55.658 INFO:tasks.workunit.client.0.smithi090.stdout:0/397: mknod d1a/d4b/d53/d79/d69/d78/c7e 0 2024-04-24T08:24:55.658 INFO:tasks.workunit.client.1.smithi149.stdout:4/615: write d2/d18/d1e/f2c [3133388,118615] 0 2024-04-24T08:24:55.659 INFO:tasks.workunit.client.1.smithi149.stdout:2/603: dwrite d0/d6/db/d39/f8e [0,4194304] 0 2024-04-24T08:24:55.665 INFO:tasks.workunit.client.0.smithi090.stdout:1/369: rename d3/d25/f4a to d3/d5/da/f82 0 2024-04-24T08:24:55.665 INFO:tasks.workunit.client.1.smithi149.stdout:7/671: unlink d9/dc/d11/d84/da7/ld4 0 2024-04-24T08:24:55.668 INFO:tasks.workunit.client.1.smithi149.stdout:9/717: rmdir d1/d2/d33 39 2024-04-24T08:24:55.669 INFO:tasks.workunit.client.0.smithi090.stdout:5/416: mknod d8/d7c/c92 0 2024-04-24T08:24:55.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/313: dwrite d2/df/f3c [0,4194304] 0 2024-04-24T08:24:55.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/290: dread d2/f12 [0,4194304] 0 2024-04-24T08:24:55.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/417: chown d8/d56/d6d/d7a/d50/f8e 5 1 2024-04-24T08:24:55.675 INFO:tasks.workunit.client.1.smithi149.stdout:1/787: link d0/d5/db/d82/f9b d0/d5/d4f/d60/f105 0 2024-04-24T08:24:55.676 INFO:tasks.workunit.client.1.smithi149.stdout:1/788: chown d0/d5/df/fbf 0 1 2024-04-24T08:24:55.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/616: symlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/lc3 0 2024-04-24T08:24:55.679 INFO:tasks.workunit.client.1.smithi149.stdout:2/604: dread d0/d6/db/f9e [0,4194304] 0 2024-04-24T08:24:55.680 INFO:tasks.workunit.client.1.smithi149.stdout:4/617: stat d2/d18/d1e/d3c/d48/l83 0 2024-04-24T08:24:55.683 INFO:tasks.workunit.client.1.smithi149.stdout:9/718: symlink d1/da/d13/d74/dbb/l104 0 2024-04-24T08:24:55.686 INFO:tasks.workunit.client.1.smithi149.stdout:9/719: write d1/da/d4a/fdd [315015,113243] 0 2024-04-24T08:24:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/672: mknod d9/dc/d11/ce3 0 2024-04-24T08:24:55.693 INFO:tasks.workunit.client.1.smithi149.stdout:0/616: write d2/d9/df/d56/f5f [294732,45710] 0 2024-04-24T08:24:55.696 INFO:tasks.workunit.client.0.smithi090.stdout:4/277: rename d3/d38/f43 to d3/d18/d1d/d22/f51 0 2024-04-24T08:24:55.699 INFO:tasks.workunit.client.0.smithi090.stdout:6/357: dread d8/dc/d19/f34 [0,4194304] 0 2024-04-24T08:24:55.701 INFO:tasks.workunit.client.0.smithi090.stdout:2/345: truncate d0/d7/f11 3222256 0 2024-04-24T08:24:55.703 INFO:tasks.workunit.client.0.smithi090.stdout:7/407: getdents db/d17 0 2024-04-24T08:24:55.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/328: write d0/fb [1807384,107725] 0 2024-04-24T08:24:55.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/618: dread d2/d18/d1e/d3c/f84 [0,4194304] 0 2024-04-24T08:24:55.705 INFO:tasks.workunit.client.0.smithi090.stdout:3/314: creat d2/df/d1a/d37/d59/f61 x:0 0 0 2024-04-24T08:24:55.705 INFO:tasks.workunit.client.0.smithi090.stdout:7/408: read db/f10 [4100991,15989] 0 2024-04-24T08:24:55.706 INFO:tasks.workunit.client.1.smithi149.stdout:2/605: truncate d0/d6/fa0 1477121 0 2024-04-24T08:24:55.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/315: write d2/d15/d36/f5f [186062,46597] 0 2024-04-24T08:24:55.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/418: rmdir d8/d56/d6d/d7b 39 2024-04-24T08:24:55.708 INFO:tasks.workunit.client.1.smithi149.stdout:2/606: write d0/d6/db/d27/d64/f74 [2230740,118608] 0 2024-04-24T08:24:55.710 INFO:tasks.workunit.client.1.smithi149.stdout:2/607: chown d0/d6/l19 315564534 1 2024-04-24T08:24:55.711 INFO:tasks.workunit.client.1.smithi149.stdout:0/617: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fcb x:0 0 0 2024-04-24T08:24:55.720 INFO:tasks.workunit.client.1.smithi149.stdout:7/673: rename d9/dc/d11/l6a to d9/dc/d11/d38/le4 0 2024-04-24T08:24:55.721 INFO:tasks.workunit.client.0.smithi090.stdout:0/398: unlink d1a/d4b/d53/d79/d35/f74 0 2024-04-24T08:24:55.725 INFO:tasks.workunit.client.1.smithi149.stdout:7/674: truncate d9/dc/d11/d83/da6/d67/d69/fcf 14203 0 2024-04-24T08:24:55.727 INFO:tasks.workunit.client.0.smithi090.stdout:5/419: dwrite d8/da/d18/d21/f41 [0,4194304] 0 2024-04-24T08:24:55.727 INFO:tasks.workunit.client.1.smithi149.stdout:1/789: getdents d0/d1b/d4b/da1 0 2024-04-24T08:24:55.728 INFO:tasks.workunit.client.0.smithi090.stdout:4/278: write d3/df/f30 [1683983,81417] 0 2024-04-24T08:24:55.729 INFO:tasks.workunit.client.1.smithi149.stdout:0/618: readlink d2/d9/l14 0 2024-04-24T08:24:55.730 INFO:tasks.workunit.client.1.smithi149.stdout:2/608: rmdir d0/d6/db/d39/d3d/d44/d57 39 2024-04-24T08:24:55.730 INFO:tasks.workunit.client.0.smithi090.stdout:2/346: fdatasync d0/d14/f38 0 2024-04-24T08:24:55.731 INFO:tasks.workunit.client.1.smithi149.stdout:2/609: fsync d0/d6/f16 0 2024-04-24T08:24:55.732 INFO:tasks.workunit.client.1.smithi149.stdout:4/619: dread d2/d18/f64 [0,4194304] 0 2024-04-24T08:24:55.733 INFO:tasks.workunit.client.0.smithi090.stdout:9/329: creat d0/dc/d24/d3d/f5c x:0 0 0 2024-04-24T08:24:55.735 INFO:tasks.workunit.client.0.smithi090.stdout:2/347: write d0/d14/d17/d2a/f5d [220783,4623] 0 2024-04-24T08:24:55.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/316: mknod d2/df/d1a/d37/d53/c62 0 2024-04-24T08:24:55.738 INFO:tasks.workunit.client.1.smithi149.stdout:0/619: chown d2/d9/df/f16 7842176 1 2024-04-24T08:24:55.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/620: stat d2/d9/df/f16 0 2024-04-24T08:24:55.740 INFO:tasks.workunit.client.1.smithi149.stdout:1/790: rename d0/d5/d4f/cab to d0/d1b/d4b/da1/c106 0 2024-04-24T08:24:55.741 INFO:tasks.workunit.client.1.smithi149.stdout:9/720: getdents d1/da/d4a/dad 0 2024-04-24T08:24:55.742 INFO:tasks.workunit.client.0.smithi090.stdout:1/370: mknod d3/d1d/d34/d5a/c83 0 2024-04-24T08:24:55.743 INFO:tasks.workunit.client.1.smithi149.stdout:9/721: write d1/da/f27 [6530655,6805] 0 2024-04-24T08:24:55.746 INFO:tasks.workunit.client.1.smithi149.stdout:9/722: write d1/d2/d15/f81 [191811,118012] 0 2024-04-24T08:24:55.746 INFO:tasks.workunit.client.1.smithi149.stdout:0/621: mkdir d2/d9/d1a/d36/dcc 0 2024-04-24T08:24:55.751 INFO:tasks.workunit.client.1.smithi149.stdout:0/622: chown d2/d9/d1a/d36/d85/lbe 12 1 2024-04-24T08:24:55.752 INFO:tasks.workunit.client.1.smithi149.stdout:9/723: dread d1/da/dd/d55/fb1 [0,4194304] 0 2024-04-24T08:24:55.755 INFO:tasks.workunit.client.1.smithi149.stdout:7/675: creat d9/fe5 x:0 0 0 2024-04-24T08:24:55.756 INFO:tasks.workunit.client.1.smithi149.stdout:7/676: readlink d9/l1d 0 2024-04-24T08:24:55.757 INFO:tasks.workunit.client.0.smithi090.stdout:2/348: dwrite d0/d14/d17/d2a/d31/f52 [0,4194304] 0 2024-04-24T08:24:55.758 INFO:tasks.workunit.client.0.smithi090.stdout:9/330: fdatasync d0/dc/f3e 0 2024-04-24T08:24:55.758 INFO:tasks.workunit.client.1.smithi149.stdout:2/610: rmdir d0/d6/db/d39/d3d/d58/d98 39 2024-04-24T08:24:55.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/331: stat d0 0 2024-04-24T08:24:55.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/332: chown d0/dc/f3e 2051177 1 2024-04-24T08:24:55.761 INFO:tasks.workunit.client.1.smithi149.stdout:1/791: truncate d0/d1b/d2a/ffa 814518 0 2024-04-24T08:24:55.766 INFO:tasks.workunit.client.0.smithi090.stdout:5/420: dread d8/d25/f3d [0,4194304] 0 2024-04-24T08:24:55.767 INFO:tasks.workunit.client.0.smithi090.stdout:9/333: truncate d0/dc/d24/d3d/f5c 76947 0 2024-04-24T08:24:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:55 smithi149 ceph-mon[31282]: pgmap v20: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 45 MiB/s rd, 187 MiB/s wr, 322 op/s 2024-04-24T08:24:55.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/349: dwrite d0/d7/d57/f66 [0,4194304] 0 2024-04-24T08:24:55.783 INFO:tasks.workunit.client.0.smithi090.stdout:2/350: stat d0/d14/d36 0 2024-04-24T08:24:55.808 INFO:tasks.workunit.client.1.smithi149.stdout:5/777: sync 2024-04-24T08:24:55.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/778: readlink db/d12/d1e/d85/d92/lec 0 2024-04-24T08:24:55.831 INFO:tasks.workunit.client.1.smithi149.stdout:0/623: mkdir d2/d9/d1a/d36/d85/dcd 0 2024-04-24T08:24:55.835 INFO:tasks.workunit.client.1.smithi149.stdout:0/624: truncate d2/d3c/fc6 984826 0 2024-04-24T08:24:55.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/371: mkdir d3/d5/da/d84 0 2024-04-24T08:24:55.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/279: dwrite d3/d18/d1d/f27 [0,4194304] 0 2024-04-24T08:24:55.850 INFO:tasks.workunit.client.1.smithi149.stdout:5/779: creat db/d12/d2f/d74/dc6/f103 x:0 0 0 2024-04-24T08:24:55.852 INFO:tasks.workunit.client.1.smithi149.stdout:4/620: getdents d2/d24/da7 0 2024-04-24T08:24:55.856 INFO:tasks.workunit.client.0.smithi090.stdout:3/317: truncate d2/f2f 769520 0 2024-04-24T08:24:55.865 INFO:tasks.workunit.client.0.smithi090.stdout:2/351: mknod d0/d14/d4d/c6d 0 2024-04-24T08:24:55.866 INFO:tasks.workunit.client.1.smithi149.stdout:1/792: mknod d0/d1b/db5/c107 0 2024-04-24T08:24:55.870 INFO:tasks.workunit.client.1.smithi149.stdout:1/793: truncate d0/d5/d4f/d60/fd5 656234 0 2024-04-24T08:24:55.872 INFO:tasks.workunit.client.0.smithi090.stdout:0/399: creat d1a/d4b/d53/f7f x:0 0 0 2024-04-24T08:24:55.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/358: rename d8/dc/l20 to d8/dc/d19/l6b 0 2024-04-24T08:24:55.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/724: creat d1/da/dd/d55/f105 x:0 0 0 2024-04-24T08:24:55.884 INFO:tasks.workunit.client.1.smithi149.stdout:5/780: dwrite db/d23/f43 [0,4194304] 0 2024-04-24T08:24:55.886 INFO:tasks.workunit.client.1.smithi149.stdout:5/781: readlink db/d12/d22/l70 0 2024-04-24T08:24:55.889 INFO:tasks.workunit.client.1.smithi149.stdout:5/782: readlink db/d12/d2f/d78/ld1 0 2024-04-24T08:24:55.890 INFO:tasks.workunit.client.0.smithi090.stdout:9/334: truncate d0/d19/f46 1552507 0 2024-04-24T08:24:55.891 INFO:tasks.workunit.client.0.smithi090.stdout:9/335: readlink d0/dc/d24/d3d/l4c 0 2024-04-24T08:24:55.892 INFO:tasks.workunit.client.1.smithi149.stdout:5/783: stat db/d12/d2f/d95/fe5 0 2024-04-24T08:24:55.894 INFO:tasks.workunit.client.0.smithi090.stdout:2/352: creat d0/d14/d17/d2a/f6e x:0 0 0 2024-04-24T08:24:55.896 INFO:tasks.workunit.client.1.smithi149.stdout:9/725: chown d1/d2/d33/d5d/d83 10 1 2024-04-24T08:24:55.897 INFO:tasks.workunit.client.0.smithi090.stdout:1/372: symlink d3/d1d/d34/d73/l85 0 2024-04-24T08:24:55.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/400: write d1a/f1e [743473,63829] 0 2024-04-24T08:24:55.904 INFO:tasks.workunit.client.0.smithi090.stdout:5/421: rename d8/d56/d6d to d8/da/d3a/d2d/d37/d93 0 2024-04-24T08:24:55.905 INFO:tasks.workunit.client.0.smithi090.stdout:0/401: readlink d1a/d4b/d53/d79/d3c/l46 0 2024-04-24T08:24:55.907 INFO:tasks.workunit.client.1.smithi149.stdout:9/726: symlink d1/da/de8/l106 0 2024-04-24T08:24:55.909 INFO:tasks.workunit.client.1.smithi149.stdout:9/727: stat d1/da/dd/d55/f105 0 2024-04-24T08:24:55.910 INFO:tasks.workunit.client.0.smithi090.stdout:6/359: creat d8/d1d/d29/f6c x:0 0 0 2024-04-24T08:24:55.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/336: dread d0/f1b [0,4194304] 0 2024-04-24T08:24:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:9/728: mknod d1/d2/d15/d1c/dca/c107 0 2024-04-24T08:24:55.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/337: write d0/d19/d33/d4d/f52 [2316861,98720] 0 2024-04-24T08:24:55.915 INFO:tasks.workunit.client.0.smithi090.stdout:1/373: write d3/d5/da/dc/f57 [3364500,47336] 0 2024-04-24T08:24:55.922 INFO:tasks.workunit.client.1.smithi149.stdout:9/729: truncate d1/d2/d15/fd9 392397 0 2024-04-24T08:24:55.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/730: chown d1/da/d4a/c102 12 1 2024-04-24T08:24:55.925 INFO:tasks.workunit.client.0.smithi090.stdout:0/402: symlink d1a/d4b/d53/l80 0 2024-04-24T08:24:55.927 INFO:tasks.workunit.client.0.smithi090.stdout:3/318: getdents d2/df/d57 0 2024-04-24T08:24:55.931 INFO:tasks.workunit.client.1.smithi149.stdout:3/743: sync 2024-04-24T08:24:55.931 INFO:tasks.workunit.client.0.smithi090.stdout:6/360: mknod d8/d1d/d1f/d3e/d4d/c6d 0 2024-04-24T08:24:55.932 INFO:tasks.workunit.client.1.smithi149.stdout:3/744: chown d0/f11 0 1 2024-04-24T08:24:55.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/338: dwrite d0/f5 [4194304,4194304] 0 2024-04-24T08:24:55.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/374: dwrite d3/d25/f6d [0,4194304] 0 2024-04-24T08:24:55.941 INFO:tasks.workunit.client.0.smithi090.stdout:1/375: fsync d3/d5/d58/f60 0 2024-04-24T08:24:55.946 INFO:tasks.workunit.client.0.smithi090.stdout:3/319: creat d2/df/d5b/f63 x:0 0 0 2024-04-24T08:24:55.956 INFO:tasks.workunit.client.0.smithi090.stdout:1/376: creat d3/d1d/f86 x:0 0 0 2024-04-24T08:24:55.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/339: dwrite d0/d19/f1c [4194304,4194304] 0 2024-04-24T08:24:55.957 INFO:tasks.workunit.client.0.smithi090.stdout:3/320: rename d2/d25/d33/l3f to d2/d15/d36/l64 0 2024-04-24T08:24:55.963 INFO:tasks.workunit.client.0.smithi090.stdout:9/340: dread d0/d19/d33/d50/f51 [0,4194304] 0 2024-04-24T08:24:55.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/321: dread d2/d15/d36/f5f [0,4194304] 0 2024-04-24T08:24:55.967 INFO:tasks.workunit.client.0.smithi090.stdout:3/322: chown d2 0 1 2024-04-24T08:24:55.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/377: creat d3/d5/da/dc/f87 x:0 0 0 2024-04-24T08:24:55.973 INFO:tasks.workunit.client.0.smithi090.stdout:6/361: dread f6 [0,4194304] 0 2024-04-24T08:24:55.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/323: creat d2/df/d1a/d37/d53/f65 x:0 0 0 2024-04-24T08:24:55.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/362: mknod d8/d1d/d1f/d3e/d4f/c6e 0 2024-04-24T08:24:55.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/324: write d2/df/f10 [1337153,107206] 0 2024-04-24T08:24:55.979 INFO:tasks.workunit.client.0.smithi090.stdout:6/363: readlink d8/l47 0 2024-04-24T08:24:55.983 INFO:tasks.workunit.client.0.smithi090.stdout:3/325: symlink d2/df/d57/d4f/l66 0 2024-04-24T08:24:55.985 INFO:tasks.workunit.client.0.smithi090.stdout:3/326: stat d2/df/d1a/l34 0 2024-04-24T08:24:55.986 INFO:tasks.workunit.client.0.smithi090.stdout:6/364: getdents d8/d1d/d2f/d67 0 2024-04-24T08:24:55.989 INFO:tasks.workunit.client.1.smithi149.stdout:7/677: dwrite d9/dc/fb7 [0,4194304] 0 2024-04-24T08:24:55.990 INFO:tasks.workunit.client.1.smithi149.stdout:6/600: sync 2024-04-24T08:24:55.990 INFO:tasks.workunit.client.1.smithi149.stdout:8/592: sync 2024-04-24T08:24:55.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/341: dwrite d0/dc/d24/d2d/f47 [4194304,4194304] 0 2024-04-24T08:24:55.995 INFO:tasks.workunit.client.1.smithi149.stdout:4/621: rename d2/d18/f64 to d2/d10/d91/fc4 0 2024-04-24T08:24:55.997 INFO:tasks.workunit.client.1.smithi149.stdout:6/601: symlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/lc8 0 2024-04-24T08:24:56.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:55.999+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea48073fe0 con 0x7fea2c06ea70 2024-04-24T08:24:56.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.001+0000 7fea3fe00700 1 -- 172.21.15.90:0/1662221151 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fea48073fe0 con 0x7fea2c06ea70 2024-04-24T08:24:56.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 msgr2=0x7fea2c070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 0x7fea2c070f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea34006010 tx=0x7fea3400b560 comp rx=0 tx=0).stop 2024-04-24T08:24:56.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 msgr2=0x7fea48074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d900 tx=0x7fea3800dc10 comp rx=0 tx=0).stop 2024-04-24T08:24:56.007 INFO:tasks.workunit.client.0.smithi090.stdout:9/342: rename d0/dc/d24/f4e to d0/dc/d24/d2d/f5d 0 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 shutdown_connections 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fea2c06ea70 0x7fea2c070f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea48100240 0x7fea48074af0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d900 tx=0x7fea3800dc10 comp rx=0 tx=0).stop 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 --2- 172.21.15.90:0/1662221151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea48101440 0x7fea48073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.005+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 >> 172.21.15.90:0/1662221151 conn(0x7fea480fb7f0 msgr2=0x7fea48104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.006+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 shutdown_connections 2024-04-24T08:24:56.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.006+0000 7fea4e000700 1 -- 172.21.15.90:0/1662221151 wait complete. 2024-04-24T08:24:56.010 INFO:tasks.workunit.client.1.smithi149.stdout:5/784: rename db/d12/d1e/ld8 to db/d12/d60/db2/d83/dfb/l104 0 2024-04-24T08:24:56.011 INFO:tasks.workunit.client.0.smithi090.stdout:6/365: dwrite d8/dc/f5f [0,4194304] 0 2024-04-24T08:24:56.013 INFO:tasks.workunit.client.1.smithi149.stdout:6/602: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d66/dc9 0 2024-04-24T08:24:56.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/366: write d8/d1d/d29/f6c [420023,59046] 0 2024-04-24T08:24:56.014 INFO:tasks.workunit.client.1.smithi149.stdout:2/611: dwrite d0/d6/db/f1f [0,4194304] 0 2024-04-24T08:24:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:4/622: dwrite d2/d18/d1e/f33 [0,4194304] 0 2024-04-24T08:24:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:2/612: chown d0/c14 864303501 1 2024-04-24T08:24:56.025 INFO:tasks.workunit.client.1.smithi149.stdout:2/613: stat d0/d6/db/d39/d78/d7c/f82 0 2024-04-24T08:24:56.028 INFO:tasks.workunit.client.0.smithi090.stdout:6/367: creat d8/d1d/f6f x:0 0 0 2024-04-24T08:24:56.033 INFO:tasks.workunit.client.0.smithi090.stdout:6/368: truncate d8/d1d/f6f 620126 0 2024-04-24T08:24:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:5/785: dread db/d12/d22/f31 [0,4194304] 0 2024-04-24T08:24:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/614: mkdir d0/d6/db/d27/d81/db4 0 2024-04-24T08:24:56.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/422: creat d8/da/d14/f94 x:0 0 0 2024-04-24T08:24:56.037 INFO:tasks.workunit.client.1.smithi149.stdout:6/603: read d0/f10 [2164587,28780] 0 2024-04-24T08:24:56.039 INFO:tasks.workunit.client.0.smithi090.stdout:5/423: fdatasync d8/d59/f86 0 2024-04-24T08:24:56.039 INFO:tasks.workunit.client.0.smithi090.stdout:6/369: mknod d8/d1d/d1f/d3e/d4f/c70 0 2024-04-24T08:24:56.042 INFO:tasks.workunit.client.1.smithi149.stdout:2/615: rmdir d0/d6/db/d28/d7f/d9b 39 2024-04-24T08:24:56.042 INFO:tasks.workunit.client.1.smithi149.stdout:5/786: write db/d12/d1e/d85/d92/da6/fcf [2236997,14186] 0 2024-04-24T08:24:56.044 INFO:tasks.workunit.client.1.smithi149.stdout:2/616: chown d0/d6/db/d39/d78/d7c 3849 1 2024-04-24T08:24:56.047 INFO:tasks.workunit.client.1.smithi149.stdout:2/617: chown d0/d6/db/d39/d3d/d44/f53 10340 1 2024-04-24T08:24:56.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/424: write d8/da/d3a/d2d/d37/d93/d7a/f6f [1411178,44943] 0 2024-04-24T08:24:56.047 INFO:tasks.workunit.client.0.smithi090.stdout:6/370: mknod d8/d1d/d2f/c71 0 2024-04-24T08:24:56.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/371: chown d8/dc/d19/l2d 15694060 1 2024-04-24T08:24:56.050 INFO:tasks.workunit.client.1.smithi149.stdout:6/604: symlink d0/d1a/d28/d31/d4c/d76/lca 0 2024-04-24T08:24:56.050 INFO:tasks.workunit.client.1.smithi149.stdout:5/787: mknod db/c105 0 2024-04-24T08:24:56.051 INFO:tasks.workunit.client.0.smithi090.stdout:5/425: symlink d8/da/d14/l95 0 2024-04-24T08:24:56.054 INFO:tasks.workunit.client.0.smithi090.stdout:6/372: truncate d8/d1d/d1f/d3e/f6a 559059 0 2024-04-24T08:24:56.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/618: creat d0/d6/db/d39/d3d/fb5 x:0 0 0 2024-04-24T08:24:56.063 INFO:tasks.workunit.client.0.smithi090.stdout:6/373: dread f6 [0,4194304] 0 2024-04-24T08:24:56.064 INFO:tasks.workunit.client.1.smithi149.stdout:2/619: creat d0/d6/db/d39/d3d/fb6 x:0 0 0 2024-04-24T08:24:56.068 INFO:tasks.workunit.client.1.smithi149.stdout:1/794: dwrite d0/d5/db/d63/f95 [0,4194304] 0 2024-04-24T08:24:56.070 INFO:tasks.workunit.client.0.smithi090.stdout:0/403: rmdir d1a 39 2024-04-24T08:24:56.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/374: creat d8/dc/f72 x:0 0 0 2024-04-24T08:24:56.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/375: chown d8/d1d/d2f/d30/f4b 59602822 1 2024-04-24T08:24:56.076 INFO:tasks.workunit.client.1.smithi149.stdout:8/593: rename d1/de/d18/f6f to d1/de/fc8 0 2024-04-24T08:24:56.077 INFO:tasks.workunit.client.0.smithi090.stdout:6/376: write d8/d1d/d2f/d30/f52 [5151714,122309] 0 2024-04-24T08:24:56.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/404: creat d1a/d4b/d53/d57/f81 x:0 0 0 2024-04-24T08:24:56.081 INFO:tasks.workunit.client.1.smithi149.stdout:9/731: write d1/da/f103 [1642991,83126] 0 2024-04-24T08:24:56.082 INFO:tasks.workunit.client.1.smithi149.stdout:8/594: mkdir d1/de/d18/d32/d65/d8b/dc9 0 2024-04-24T08:24:56.084 INFO:tasks.workunit.client.0.smithi090.stdout:5/426: mkdir d8/da/d3a/d96 0 2024-04-24T08:24:56.088 INFO:tasks.workunit.client.0.smithi090.stdout:7/409: creat db/d17/d29/f8e x:0 0 0 2024-04-24T08:24:56.089 INFO:tasks.workunit.client.1.smithi149.stdout:3/745: write d0/d5/da/f6e [988533,25252] 0 2024-04-24T08:24:56.090 INFO:tasks.workunit.client.1.smithi149.stdout:1/795: rename d0/d1b/d5d/de1 to d0/d1b/db5/d108 0 2024-04-24T08:24:56.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/732: symlink d1/d2/d33/d5d/d83/dbf/l108 0 2024-04-24T08:24:56.093 INFO:tasks.workunit.client.0.smithi090.stdout:0/405: mkdir d1a/d4b/d53/d79/d82 0 2024-04-24T08:24:56.095 INFO:tasks.workunit.client.0.smithi090.stdout:8/291: rename d2/da/d13/d15 to d2/da/d13/d64 0 2024-04-24T08:24:56.099 INFO:tasks.workunit.client.0.smithi090.stdout:6/377: dwrite d8/d1d/d29/f2c [4194304,4194304] 0 2024-04-24T08:24:56.100 INFO:tasks.workunit.client.0.smithi090.stdout:0/406: unlink d1a/d4b/c6b 0 2024-04-24T08:24:56.108 INFO:tasks.workunit.client.1.smithi149.stdout:3/746: dwrite d0/d5/d53/d34/f3b [4194304,4194304] 0 2024-04-24T08:24:56.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/595: creat d1/de/d18/d32/fca x:0 0 0 2024-04-24T08:24:56.113 INFO:tasks.workunit.client.1.smithi149.stdout:8/596: chown d1/f8e 74893 1 2024-04-24T08:24:56.117 INFO:tasks.workunit.client.0.smithi090.stdout:8/292: symlink d2/da/d13/d60/l65 0 2024-04-24T08:24:56.117 INFO:tasks.workunit.client.1.smithi149.stdout:8/597: fdatasync d1/d1f/d25/f80 0 2024-04-24T08:24:56.118 INFO:tasks.workunit.client.0.smithi090.stdout:0/407: mkdir d1a/d4b/d53/d79/d83 0 2024-04-24T08:24:56.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/378: mknod d8/d1d/d1f/d3e/d44/c73 0 2024-04-24T08:24:56.123 INFO:tasks.workunit.client.0.smithi090.stdout:2/353: rename d0/d7/d24 to d0/d7/d6f 0 2024-04-24T08:24:56.124 INFO:tasks.workunit.client.0.smithi090.stdout:6/379: chown d8/d1d/d29 19 1 2024-04-24T08:24:56.124 INFO:tasks.workunit.client.0.smithi090.stdout:2/354: fsync d0/d2e/f37 0 2024-04-24T08:24:56.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/408: creat d1a/d4b/d53/d79/d3c/f84 x:0 0 0 2024-04-24T08:24:56.125 INFO:tasks.workunit.client.1.smithi149.stdout:8/598: symlink d1/de/d18/d32/d65/d8b/dc9/lcb 0 2024-04-24T08:24:56.128 INFO:tasks.workunit.client.0.smithi090.stdout:9/343: mkdir d0/d5e 0 2024-04-24T08:24:56.128 INFO:tasks.workunit.client.0.smithi090.stdout:6/380: read d8/d1d/d1f/d3e/f5c [7642885,73377] 0 2024-04-24T08:24:56.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/293: getdents d2/da 0 2024-04-24T08:24:56.129 INFO:tasks.workunit.client.1.smithi149.stdout:3/747: dwrite d0/d5/d53/d4c/fc2 [0,4194304] 0 2024-04-24T08:24:56.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/381: write d8/dc/f72 [75244,35901] 0 2024-04-24T08:24:56.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/327: write d2/f9 [2743914,7584] 0 2024-04-24T08:24:56.137 INFO:tasks.workunit.client.0.smithi090.stdout:0/409: creat d1a/d4b/f85 x:0 0 0 2024-04-24T08:24:56.138 INFO:tasks.workunit.client.1.smithi149.stdout:7/678: truncate d9/d43/f7e 3390397 0 2024-04-24T08:24:56.139 INFO:tasks.workunit.client.1.smithi149.stdout:3/748: rmdir d0/d5/d53/d4c/dca 39 2024-04-24T08:24:56.144 INFO:tasks.workunit.client.0.smithi090.stdout:2/355: rename d0/d14/d17/d2a to d0/d7/d70 0 2024-04-24T08:24:56.149 INFO:tasks.workunit.client.1.smithi149.stdout:3/749: creat d0/d5/d53/dd6/f100 x:0 0 0 2024-04-24T08:24:56.151 INFO:tasks.workunit.client.1.smithi149.stdout:7/679: symlink d9/dc/d2e/d3c/db8/le6 0 2024-04-24T08:24:56.152 INFO:tasks.workunit.client.1.smithi149.stdout:7/680: readlink d9/d43/l5f 0 2024-04-24T08:24:56.154 INFO:tasks.workunit.client.1.smithi149.stdout:3/750: creat d0/d5/da/d5e/f101 x:0 0 0 2024-04-24T08:24:56.154 INFO:tasks.workunit.client.0.smithi090.stdout:8/294: dwrite d2/da/d13/d64/d50/f5d [4194304,4194304] 0 2024-04-24T08:24:56.155 INFO:tasks.workunit.client.1.smithi149.stdout:4/623: write d2/d24/d2e/d38/f5e [4718066,95373] 0 2024-04-24T08:24:56.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/295: write d2/f2c [1199076,76870] 0 2024-04-24T08:24:56.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/296: stat d2/f12 0 2024-04-24T08:24:56.159 INFO:tasks.workunit.client.0.smithi090.stdout:2/356: symlink d0/d7/d70/l71 0 2024-04-24T08:24:56.160 INFO:tasks.workunit.client.0.smithi090.stdout:3/328: mknod d2/df/d1a/d37/d4b/c67 0 2024-04-24T08:24:56.160 INFO:tasks.workunit.client.1.smithi149.stdout:4/624: fsync d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf 0 2024-04-24T08:24:56.161 INFO:tasks.workunit.client.0.smithi090.stdout:2/357: truncate d0/f22 9352284 0 2024-04-24T08:24:56.162 INFO:tasks.workunit.client.1.smithi149.stdout:7/681: unlink d9/dc/f35 0 2024-04-24T08:24:56.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/358: write d0/d7/fd [8543529,120181] 0 2024-04-24T08:24:56.166 INFO:tasks.workunit.client.0.smithi090.stdout:6/382: rmdir d8/d1d/d2f/d67 0 2024-04-24T08:24:56.167 INFO:tasks.workunit.client.1.smithi149.stdout:4/625: fdatasync d2/d24/d2e/f51 0 2024-04-24T08:24:56.174 INFO:tasks.workunit.client.1.smithi149.stdout:0/625: mkdir d2/d9/df/d2f/d6c/dce 0 2024-04-24T08:24:56.175 INFO:tasks.workunit.client.1.smithi149.stdout:3/751: rename d0/d5/d53/d6d/ce3 to d0/d5/d53/d4c/dca/dfb/c102 0 2024-04-24T08:24:56.176 INFO:tasks.workunit.client.0.smithi090.stdout:0/410: dread d1a/d4b/d53/d79/d35/f42 [0,4194304] 0 2024-04-24T08:24:56.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/329: mknod d2/df/d21/c68 0 2024-04-24T08:24:56.180 INFO:tasks.workunit.client.1.smithi149.stdout:6/605: dwrite d0/fa [0,4194304] 0 2024-04-24T08:24:56.181 INFO:tasks.workunit.client.1.smithi149.stdout:2/620: write d0/d6/db/d28/d7f/f87 [863022,50628] 0 2024-04-24T08:24:56.182 INFO:tasks.workunit.client.1.smithi149.stdout:2/621: readlink d0/d6/db/d28/l5b 0 2024-04-24T08:24:56.185 INFO:tasks.workunit.client.0.smithi090.stdout:5/427: rmdir d8/da 39 2024-04-24T08:24:56.188 INFO:tasks.workunit.client.1.smithi149.stdout:0/626: rmdir d2/d9/d1a 39 2024-04-24T08:24:56.189 INFO:tasks.workunit.client.1.smithi149.stdout:2/622: truncate d0/d6/db/d39/d3d/d44/f96 157961 0 2024-04-24T08:24:56.193 INFO:tasks.workunit.client.1.smithi149.stdout:3/752: creat d0/d5/da/d5a/f103 x:0 0 0 2024-04-24T08:24:56.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/411: mknod d1a/d4b/d53/d79/d58/c86 0 2024-04-24T08:24:56.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/412: read d1a/f64 [657668,75158] 0 2024-04-24T08:24:56.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/330: dwrite d2/df/d5b/f63 [0,4194304] 0 2024-04-24T08:24:56.203 INFO:tasks.workunit.client.0.smithi090.stdout:7/410: truncate db/d17/d29/d38/f4e 1908498 0 2024-04-24T08:24:56.204 INFO:tasks.workunit.client.1.smithi149.stdout:6/606: creat d0/d1a/d28/d31/d4c/d80/d6f/fcb x:0 0 0 2024-04-24T08:24:56.205 INFO:tasks.workunit.client.0.smithi090.stdout:6/383: mkdir d8/d74 0 2024-04-24T08:24:56.207 INFO:tasks.workunit.client.1.smithi149.stdout:2/623: mknod d0/d6/db/d39/d78/cb7 0 2024-04-24T08:24:56.214 INFO:tasks.workunit.client.1.smithi149.stdout:1/796: dwrite d0/d5/df/fbf [0,4194304] 0 2024-04-24T08:24:56.216 INFO:tasks.workunit.client.1.smithi149.stdout:0/627: dwrite d2/d9/f2d [0,4194304] 0 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.215+0000 7f2ab9200700 1 -- 172.21.15.90:0/3391980718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4103640 msgr2=0x7f2ab4103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.215+0000 7f2ab9200700 1 --2- 172.21.15.90:0/3391980718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4103640 0x7f2ab4103a90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009910 tx=0x7f2aa0009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.216+0000 7f2ab9200700 1 -- 172.21.15.90:0/3391980718 shutdown_connections 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.216+0000 7f2ab9200700 1 --2- 172.21.15.90:0/3391980718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4103640 0x7f2ab4103a90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.216+0000 7f2ab9200700 1 --2- 172.21.15.90:0/3391980718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4102490 0x7f2ab41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.216+0000 7f2ab9200700 1 -- 172.21.15.90:0/3391980718 >> 172.21.15.90:0/3391980718 conn(0x7f2ab40fda60 msgr2=0x7f2ab40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:56.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.218+0000 7f2ab9200700 1 -- 172.21.15.90:0/3391980718 shutdown_connections 2024-04-24T08:24:56.219 INFO:tasks.workunit.client.1.smithi149.stdout:6/607: stat d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/f94 0 2024-04-24T08:24:56.220 INFO:tasks.workunit.client.1.smithi149.stdout:9/733: dwrite d1/d2/d15/d1c/f3a [0,4194304] 0 2024-04-24T08:24:56.220 INFO:tasks.workunit.client.1.smithi149.stdout:1/797: readlink d0/d1b/db5/d108/l9f 0 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.218+0000 7f2ab9200700 1 -- 172.21.15.90:0/3391980718 wait complete. 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.219+0000 7f2ab9200700 1 Processor -- start 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.219+0000 7f2ab9200700 1 -- start start 2024-04-24T08:24:56.221 INFO:tasks.workunit.client.0.smithi090.stdout:5/428: creat d8/da/d14/d2f/d44/f97 x:0 0 0 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.220+0000 7f2ab9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.220+0000 7f2ab9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 0x7f2ab410c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.220+0000 7f2ab9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab410c9f0 con 0x7f2ab4102490 2024-04-24T08:24:56.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.220+0000 7f2ab9200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41a9eb0 con 0x7f2ab4103640 2024-04-24T08:24:56.222 INFO:tasks.workunit.client.0.smithi090.stdout:3/331: dwrite d2/d25/d33/f44 [0,4194304] 0 2024-04-24T08:24:56.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.220+0000 7f2ab1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.221+0000 7f2ab1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44502/0 (socket says 172.21.15.90:44502) 2024-04-24T08:24:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.221+0000 7f2ab1e00700 1 -- 172.21.15.90:0/4277941183 learned_addr learned my addr 172.21.15.90:0/4277941183 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.221+0000 7f2ab1400700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 0x7f2ab410c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.221+0000 7f2ab1e00700 1 -- 172.21.15.90:0/4277941183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 msgr2=0x7f2ab410c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.222+0000 7f2ab1e00700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 0x7f2ab410c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/429: truncate d8/da/d18/f60 625673 0 2024-04-24T08:24:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.222+0000 7f2ab1e00700 1 -- 172.21.15.90:0/4277941183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00095f0 con 0x7f2ab4102490 2024-04-24T08:24:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.223+0000 7f2ab1400700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 0x7f2ab410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:24:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.223+0000 7f2ab1e00700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400b480 tx=0x7f2aa400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:56.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.224+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2aa40118c0 con 0x7f2ab4102490 2024-04-24T08:24:56.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/430: fdatasync d8/da/d14/d2f/f66 0 2024-04-24T08:24:56.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.224+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2aa4010380 con 0x7f2ab4102490 2024-04-24T08:24:56.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.224+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2aa400f4f0 con 0x7f2ab4102490 2024-04-24T08:24:56.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.224+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab41aa0b0 con 0x7f2ab4102490 2024-04-24T08:24:56.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.224+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab41aa5b0 con 0x7f2ab4102490 2024-04-24T08:24:56.226 INFO:tasks.workunit.client.0.smithi090.stdout:5/431: chown d8/da/d18/d21/l29 368115 1 2024-04-24T08:24:56.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.226+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab410f3c0 con 0x7f2ab4102490 2024-04-24T08:24:56.228 INFO:tasks.workunit.client.0.smithi090.stdout:5/432: chown d8/d7c/c92 789 1 2024-04-24T08:24:56.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.232+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f2aa4011a20 con 0x7f2ab4102490 2024-04-24T08:24:56.235 INFO:tasks.workunit.client.1.smithi149.stdout:7/682: link d9/dc/c1f d9/dc/d7b/ce7 0 2024-04-24T08:24:56.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.234+0000 7f2aaaa00700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 0x7f2a98070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.235+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2aa408e9e0 con 0x7f2ab4102490 2024-04-24T08:24:56.237 INFO:tasks.workunit.client.0.smithi090.stdout:2/359: getdents d0/d7/d57 0 2024-04-24T08:24:56.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.237+0000 7f2ab1400700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 0x7f2a98070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.238 INFO:tasks.workunit.client.1.smithi149.stdout:4/626: getdents d2/d18/da4 0 2024-04-24T08:24:56.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.237+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f2aa4057bc0 con 0x7f2ab4102490 2024-04-24T08:24:56.239 INFO:tasks.workunit.client.0.smithi090.stdout:2/360: readlink d0/d7/l47 0 2024-04-24T08:24:56.240 INFO:tasks.workunit.client.0.smithi090.stdout:6/384: write d8/d1d/d1f/d3e/f5c [98970,63808] 0 2024-04-24T08:24:56.242 INFO:tasks.workunit.client.1.smithi149.stdout:7/683: truncate d9/dc/f6b 2059650 0 2024-04-24T08:24:56.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.241+0000 7f2ab1400700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 0x7f2a98070fc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:56.244 INFO:tasks.workunit.client.1.smithi149.stdout:2/624: dread - d0/d6/db/d39/d3d/d44/d75/f79 zero size 2024-04-24T08:24:56.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/332: write d2/d15/f31 [2887210,73512] 0 2024-04-24T08:24:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:0/628: dwrite d2/d9/df/d2f/d6c/dc1/d4e/d6f/f76 [0,4194304] 0 2024-04-24T08:24:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:1/798: dread d0/d5/db/d18/f8a [0,4194304] 0 2024-04-24T08:24:56.259 INFO:tasks.workunit.client.1.smithi149.stdout:0/629: fsync d2/d9/df/d2f/d6c/f9a 0 2024-04-24T08:24:56.259 INFO:tasks.workunit.client.1.smithi149.stdout:9/734: creat d1/de/d12/f109 x:0 0 0 2024-04-24T08:24:56.261 INFO:tasks.workunit.client.1.smithi149.stdout:9/735: fdatasync d1/d2/d34/f40 0 2024-04-24T08:24:56.264 INFO:tasks.workunit.client.1.smithi149.stdout:0/630: read d2/d9/df/d2f/d6c/dc1/f79 [444317,63344] 0 2024-04-24T08:24:56.264 INFO:tasks.workunit.client.1.smithi149.stdout:9/736: chown d1/da/dd/c4d 6 1 2024-04-24T08:24:56.268 INFO:tasks.workunit.client.1.smithi149.stdout:2/625: chown d0/d6/db/l32 924198 1 2024-04-24T08:24:56.272 INFO:tasks.workunit.client.1.smithi149.stdout:6/608: mknod d0/d1a/d28/d31/d4c/d80/ccc 0 2024-04-24T08:24:56.272 INFO:tasks.workunit.client.0.smithi090.stdout:8/297: write d2/da/d13/d64/d24/f2f [2693782,101123] 0 2024-04-24T08:24:56.274 INFO:tasks.workunit.client.0.smithi090.stdout:3/333: symlink d2/df/d1a/l69 0 2024-04-24T08:24:56.276 INFO:tasks.workunit.client.0.smithi090.stdout:3/334: fdatasync d2/df/d21/f27 0 2024-04-24T08:24:56.277 INFO:tasks.workunit.client.1.smithi149.stdout:4/627: unlink d2/d18/d1e/c2d 0 2024-04-24T08:24:56.280 INFO:tasks.workunit.client.1.smithi149.stdout:7/684: read - d9/d43/d4f/f92 zero size 2024-04-24T08:24:56.287 INFO:tasks.workunit.client.0.smithi090.stdout:0/413: dread d1a/f24 [4194304,4194304] 0 2024-04-24T08:24:56.287 INFO:tasks.workunit.client.1.smithi149.stdout:7/685: chown d9/dc/d11/d84/da7/fd3 134052100 1 2024-04-24T08:24:56.290 INFO:tasks.workunit.client.0.smithi090.stdout:0/414: chown d1a/d4b/d53/d79/d83 14333377 1 2024-04-24T08:24:56.293 INFO:tasks.workunit.client.1.smithi149.stdout:9/737: creat d1/d2/d34/f10a x:0 0 0 2024-04-24T08:24:56.293 INFO:tasks.workunit.client.1.smithi149.stdout:3/753: truncate d0/d5/d53/d4c/fc2 1300749 0 2024-04-24T08:24:56.295 INFO:tasks.workunit.client.1.smithi149.stdout:3/754: stat d0/d5/da/d59 0 2024-04-24T08:24:56.298 INFO:tasks.workunit.client.1.smithi149.stdout:6/609: dread d0/fb [0,4194304] 0 2024-04-24T08:24:56.300 INFO:tasks.workunit.client.1.smithi149.stdout:3/755: truncate d0/d5/d53/d34/d3a/d93/ff9 209344 0 2024-04-24T08:24:56.300 INFO:tasks.workunit.client.0.smithi090.stdout:5/433: creat d8/da/d14/d2f/d81/f98 x:0 0 0 2024-04-24T08:24:56.301 INFO:tasks.workunit.client.0.smithi090.stdout:5/434: chown d8/da/d14/c7d 0 1 2024-04-24T08:24:56.302 INFO:tasks.workunit.client.0.smithi090.stdout:5/435: stat d8/da/d14/l16 0 2024-04-24T08:24:56.303 INFO:tasks.workunit.client.0.smithi090.stdout:7/411: link db/d17/d29/f2f db/d54/d83/f8f 0 2024-04-24T08:24:56.304 INFO:tasks.workunit.client.0.smithi090.stdout:2/361: creat d0/d7/d67/f72 x:0 0 0 2024-04-24T08:24:56.309 INFO:tasks.workunit.client.1.smithi149.stdout:1/799: dread d0/ddf/fc0 [0,4194304] 0 2024-04-24T08:24:56.312 INFO:tasks.workunit.client.0.smithi090.stdout:8/298: creat d2/da/d13/d34/f66 x:0 0 0 2024-04-24T08:24:56.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/299: write d2/da/f38 [878757,11636] 0 2024-04-24T08:24:56.318 INFO:tasks.workunit.client.0.smithi090.stdout:1/378: sync 2024-04-24T08:24:56.318 INFO:tasks.workunit.client.0.smithi090.stdout:4/280: sync 2024-04-24T08:24:56.319 INFO:tasks.workunit.client.0.smithi090.stdout:7/412: unlink db/d17/d29/d32/d5a/d75/l35 0 2024-04-24T08:24:56.320 INFO:tasks.workunit.client.0.smithi090.stdout:2/362: rmdir d0/d7/d70/d31/d46/d60 39 2024-04-24T08:24:56.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/300: rmdir d2/da 39 2024-04-24T08:24:56.322 INFO:tasks.workunit.client.0.smithi090.stdout:7/413: chown db/d17/d29/d38/c52 3911240 1 2024-04-24T08:24:56.323 INFO:tasks.workunit.client.0.smithi090.stdout:0/415: symlink d1a/d4b/d53/d79/l87 0 2024-04-24T08:24:56.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/281: rmdir d3 39 2024-04-24T08:24:56.326 INFO:tasks.workunit.client.0.smithi090.stdout:6/385: link d8/dc/l26 d8/d1d/d1f/d3e/l75 0 2024-04-24T08:24:56.328 INFO:tasks.workunit.client.0.smithi090.stdout:1/379: write d3/d1d/f55 [1729432,2926] 0 2024-04-24T08:24:56.328 INFO:tasks.workunit.client.0.smithi090.stdout:2/363: symlink d0/d7/d70/d31/l73 0 2024-04-24T08:24:56.330 INFO:tasks.workunit.client.0.smithi090.stdout:0/416: read d1a/d4b/f6f [716137,106223] 0 2024-04-24T08:24:56.333 INFO:tasks.workunit.client.0.smithi090.stdout:8/301: chown d2/da/d13/d64/f4e 39 1 2024-04-24T08:24:56.336 INFO:tasks.workunit.client.0.smithi090.stdout:0/417: mknod d1a/d4b/d53/d79/d58/c88 0 2024-04-24T08:24:56.338 INFO:tasks.workunit.client.0.smithi090.stdout:5/436: getdents d8/da/d3a/d2d/d37/d93/d7b 0 2024-04-24T08:24:56.339 INFO:tasks.workunit.client.1.smithi149.stdout:3/756: symlink d0/d5/da/d19/l104 0 2024-04-24T08:24:56.340 INFO:tasks.workunit.client.0.smithi090.stdout:2/364: creat d0/d7/d70/d31/d46/d60/f74 x:0 0 0 2024-04-24T08:24:56.340 INFO:tasks.workunit.client.1.smithi149.stdout:7/686: unlink d9/d43/d4f/c60 0 2024-04-24T08:24:56.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/380: getdents d3/d5/da/d84 0 2024-04-24T08:24:56.341 INFO:tasks.workunit.client.1.smithi149.stdout:1/800: chown d0/ddf/d53/fff 13131745 1 2024-04-24T08:24:56.343 INFO:tasks.workunit.client.1.smithi149.stdout:4/628: write d2/d10/d1a/f36 [8507663,51178] 0 2024-04-24T08:24:56.345 INFO:tasks.workunit.client.0.smithi090.stdout:2/365: dread - d0/d5e/f63 zero size 2024-04-24T08:24:56.348 INFO:tasks.workunit.client.0.smithi090.stdout:0/418: rename d1a/d4b/d53/d79/d35/d62/l6c to d1a/d4b/d53/d79/d69/l89 0 2024-04-24T08:24:56.349 INFO:tasks.workunit.client.0.smithi090.stdout:6/386: link d8/d1d/d2f/d30/d38/c65 d8/d1d/d1f/d3e/d4f/c76 0 2024-04-24T08:24:56.352 INFO:tasks.workunit.client.0.smithi090.stdout:6/387: stat d8/d1d/d1f 0 2024-04-24T08:24:56.355 INFO:tasks.workunit.client.1.smithi149.stdout:7/687: creat d9/dc/d11/d38/fe8 x:0 0 0 2024-04-24T08:24:56.356 INFO:tasks.workunit.client.0.smithi090.stdout:5/437: mkdir d8/d99 0 2024-04-24T08:24:56.359 INFO:tasks.workunit.client.0.smithi090.stdout:7/414: truncate db/d17/d29/d38/f44 791021 0 2024-04-24T08:24:56.360 INFO:tasks.workunit.client.1.smithi149.stdout:0/631: dwrite d2/d9/df/d2f/f34 [0,4194304] 0 2024-04-24T08:24:56.362 INFO:tasks.workunit.client.0.smithi090.stdout:7/415: write db/d17/f85 [851717,3382] 0 2024-04-24T08:24:56.363 INFO:tasks.workunit.client.0.smithi090.stdout:0/419: mkdir d1a/d4b/d53/d79/d69/d8a 0 2024-04-24T08:24:56.364 INFO:tasks.workunit.client.1.smithi149.stdout:6/610: dwrite d0/d1a/d28/d31/d33/f9f [0,4194304] 0 2024-04-24T08:24:56.364 INFO:tasks.workunit.client.0.smithi090.stdout:7/416: fdatasync db/d17/f2d 0 2024-04-24T08:24:56.365 INFO:tasks.workunit.client.0.smithi090.stdout:8/302: write d2/f12 [959902,56325] 0 2024-04-24T08:24:56.368 INFO:tasks.workunit.client.1.smithi149.stdout:6/611: fdatasync d0/d1a/d28/d4a/f77 0 2024-04-24T08:24:56.368 INFO:tasks.workunit.client.1.smithi149.stdout:2/626: dwrite d0/d6/db/d27/d64/f6e [0,4194304] 0 2024-04-24T08:24:56.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/388: rmdir d8/d1d/d1f/d3e/d4d 39 2024-04-24T08:24:56.370 INFO:tasks.workunit.client.0.smithi090.stdout:1/381: dwrite d3/d5/f28 [0,4194304] 0 2024-04-24T08:24:56.371 INFO:tasks.workunit.client.0.smithi090.stdout:8/303: write d2/da/f35 [1261420,81523] 0 2024-04-24T08:24:56.375 INFO:tasks.workunit.client.1.smithi149.stdout:2/627: write d0/d6/db/d27/fab [730630,119763] 0 2024-04-24T08:24:56.375 INFO:tasks.workunit.client.1.smithi149.stdout:3/757: write d0/d26/d3c/db1/ffc [3312397,55865] 0 2024-04-24T08:24:56.376 INFO:tasks.workunit.client.0.smithi090.stdout:2/366: fsync d0/d7/f11 0 2024-04-24T08:24:56.379 INFO:tasks.workunit.client.0.smithi090.stdout:0/420: readlink l10 0 2024-04-24T08:24:56.380 INFO:tasks.workunit.client.0.smithi090.stdout:2/367: stat d0/d7/d70/c3d 0 2024-04-24T08:24:56.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/417: mkdir db/d90 0 2024-04-24T08:24:56.382 INFO:tasks.workunit.client.1.smithi149.stdout:2/628: chown d0/d6/db/d27/c50 202 1 2024-04-24T08:24:56.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/282: dwrite d3/f3f [0,4194304] 0 2024-04-24T08:24:56.388 INFO:tasks.workunit.client.1.smithi149.stdout:4/629: dwrite d2/d24/d2e/d38/f5c [0,4194304] 0 2024-04-24T08:24:56.395 INFO:tasks.workunit.client.0.smithi090.stdout:8/304: stat d2/l8 0 2024-04-24T08:24:56.397 INFO:tasks.workunit.client.0.smithi090.stdout:5/438: dwrite d8/d25/f8f [0,4194304] 0 2024-04-24T08:24:56.401 INFO:tasks.workunit.client.0.smithi090.stdout:1/382: symlink d3/d5/d58/l88 0 2024-04-24T08:24:56.406 INFO:tasks.workunit.client.0.smithi090.stdout:5/439: write d8/da/d3a/d2d/d37/d93/d7a/f87 [859755,29263] 0 2024-04-24T08:24:56.411 INFO:tasks.workunit.client.0.smithi090.stdout:5/440: fdatasync d8/da/d3a/d2d/d37/d93/d7a/d50/f68 0 2024-04-24T08:24:56.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/283: rename d3/lb to d3/d18/d34/l52 0 2024-04-24T08:24:56.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/368: dwrite d0/d14/d17/f1b [0,4194304] 0 2024-04-24T08:24:56.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/383: unlink d3/d1d/d34/f6b 0 2024-04-24T08:24:56.427 INFO:tasks.workunit.client.0.smithi090.stdout:0/421: creat d1a/d4b/d53/f8b x:0 0 0 2024-04-24T08:24:56.429 INFO:tasks.workunit.client.0.smithi090.stdout:5/441: readlink d8/d59/l8d 0 2024-04-24T08:24:56.435 INFO:tasks.workunit.client.1.smithi149.stdout:8/599: sync 2024-04-24T08:24:56.445 INFO:tasks.workunit.client.0.smithi090.stdout:1/384: symlink d3/d1d/d34/l89 0 2024-04-24T08:24:56.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/418: dwrite db/d17/d29/d32/d5a/d75/f70 [0,4194304] 0 2024-04-24T08:24:56.446 INFO:tasks.workunit.client.1.smithi149.stdout:2/629: unlink d0/d6/db/f15 0 2024-04-24T08:24:56.447 INFO:tasks.workunit.client.0.smithi090.stdout:5/442: dread - d8/da/d14/d2f/f47 zero size 2024-04-24T08:24:56.448 INFO:tasks.workunit.client.1.smithi149.stdout:3/758: rename d0/d5/da/f28 to d0/d5/d53/d34/d3a/d93/f105 0 2024-04-24T08:24:56.453 INFO:tasks.workunit.client.1.smithi149.stdout:1/801: truncate d0/d5/db/d18/d4d/d50/fd8 225894 0 2024-04-24T08:24:56.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/443: write d8/da/f35 [513780,104255] 0 2024-04-24T08:24:56.454 INFO:tasks.workunit.client.0.smithi090.stdout:6/389: dwrite d8/dc/f3a [0,4194304] 0 2024-04-24T08:24:56.455 INFO:tasks.workunit.client.1.smithi149.stdout:8/600: dwrite d1/de/f38 [0,4194304] 0 2024-04-24T08:24:56.455 INFO:tasks.workunit.client.0.smithi090.stdout:4/284: truncate d3/f9 1957620 0 2024-04-24T08:24:56.456 INFO:tasks.workunit.client.0.smithi090.stdout:8/305: dread d2/da/d13/d64/f49 [0,4194304] 0 2024-04-24T08:24:56.460 INFO:tasks.workunit.client.0.smithi090.stdout:8/306: dread - d2/da/d13/d64/d24/f5b zero size 2024-04-24T08:24:56.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/369: rename d0/d7/d70/c4b to d0/c75 0 2024-04-24T08:24:56.466 INFO:tasks.workunit.client.1.smithi149.stdout:4/630: dread d2/d10/f81 [4194304,4194304] 0 2024-04-24T08:24:56.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/307: write d2/f3 [4677750,127910] 0 2024-04-24T08:24:56.473 INFO:tasks.workunit.client.1.smithi149.stdout:1/802: unlink d0/ddf/d53/fff 0 2024-04-24T08:24:56.475 INFO:tasks.workunit.client.1.smithi149.stdout:5/788: sync 2024-04-24T08:24:56.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/385: dwrite d3/d1d/d34/d5a/f5b [0,4194304] 0 2024-04-24T08:24:56.480 INFO:tasks.workunit.client.0.smithi090.stdout:0/422: dwrite d1a/d4b/d53/d79/d35/f55 [0,4194304] 0 2024-04-24T08:24:56.480 INFO:tasks.workunit.client.1.smithi149.stdout:8/601: dread - d1/d1f/d22/f71 zero size 2024-04-24T08:24:56.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/612: write d0/f11 [5468884,54149] 0 2024-04-24T08:24:56.481 INFO:tasks.workunit.client.0.smithi090.stdout:1/386: dread - d3/d5/d3e/f63 zero size 2024-04-24T08:24:56.487 INFO:tasks.workunit.client.0.smithi090.stdout:4/285: dread d3/df/f36 [0,4194304] 0 2024-04-24T08:24:56.489 INFO:tasks.workunit.client.0.smithi090.stdout:7/419: creat db/d90/f91 x:0 0 0 2024-04-24T08:24:56.491 INFO:tasks.workunit.client.1.smithi149.stdout:0/632: dread d2/d9/d1a/d36/f74 [0,4194304] 0 2024-04-24T08:24:56.492 INFO:tasks.workunit.client.0.smithi090.stdout:6/390: mknod d8/c77 0 2024-04-24T08:24:56.496 INFO:tasks.workunit.client.0.smithi090.stdout:2/370: creat d0/d2c/f76 x:0 0 0 2024-04-24T08:24:56.496 INFO:tasks.workunit.client.0.smithi090.stdout:5/444: dread d8/da/d18/f1f [0,4194304] 0 2024-04-24T08:24:56.503 INFO:tasks.workunit.client.0.smithi090.stdout:2/371: truncate d0/d7/d70/d31/d46/d60/f74 618743 0 2024-04-24T08:24:56.508 INFO:tasks.workunit.client.0.smithi090.stdout:8/308: symlink d2/da/d13/d64/d24/l67 0 2024-04-24T08:24:56.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/387: mknod d3/d25/c8a 0 2024-04-24T08:24:56.514 INFO:tasks.workunit.client.1.smithi149.stdout:1/803: read d0/d5/d4f/d60/f105 [72915,116355] 0 2024-04-24T08:24:56.515 INFO:tasks.workunit.client.1.smithi149.stdout:2/630: creat d0/d6/db/d39/d3d/fb8 x:0 0 0 2024-04-24T08:24:56.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/423: readlink d1a/d4b/d53/l65 0 2024-04-24T08:24:56.516 INFO:tasks.workunit.client.0.smithi090.stdout:5/445: dwrite d8/da/d3a/d2d/d37/f6a [0,4194304] 0 2024-04-24T08:24:56.517 INFO:tasks.workunit.client.1.smithi149.stdout:3/759: dwrite d0/d5/f16 [0,4194304] 0 2024-04-24T08:24:56.517 INFO:tasks.workunit.client.0.smithi090.stdout:4/286: write d3/d18/d34/f4a [451588,49921] 0 2024-04-24T08:24:56.518 INFO:tasks.workunit.client.0.smithi090.stdout:2/372: dread d0/d14/d17/f1b [0,4194304] 0 2024-04-24T08:24:56.519 INFO:tasks.workunit.client.0.smithi090.stdout:5/446: readlink d8/da/d18/d21/d4c/l78 0 2024-04-24T08:24:56.520 INFO:tasks.workunit.client.0.smithi090.stdout:2/373: readlink d0/d7/l47 0 2024-04-24T08:24:56.521 INFO:tasks.workunit.client.1.smithi149.stdout:8/602: symlink d1/de/d18/daf/lcc 0 2024-04-24T08:24:56.525 INFO:tasks.workunit.client.0.smithi090.stdout:9/344: sync 2024-04-24T08:24:56.528 INFO:tasks.workunit.client.0.smithi090.stdout:8/309: symlink d2/da/l68 0 2024-04-24T08:24:56.531 INFO:tasks.workunit.client.1.smithi149.stdout:7/688: rename d9/dc/d2e/d3c/db8/fde to d9/dc/d11/d83/da6/d67/d69/fe9 0 2024-04-24T08:24:56.532 INFO:tasks.workunit.client.0.smithi090.stdout:8/310: write d2/da/f38 [1162220,24790] 0 2024-04-24T08:24:56.534 INFO:tasks.workunit.client.1.smithi149.stdout:6/613: dread d0/d1a/f2e [0,4194304] 0 2024-04-24T08:24:56.534 INFO:tasks.workunit.client.0.smithi090.stdout:1/388: creat d3/d1d/d34/d5a/f8b x:0 0 0 2024-04-24T08:24:56.535 INFO:tasks.workunit.client.1.smithi149.stdout:5/789: rmdir db/d12/d22/d45 39 2024-04-24T08:24:56.536 INFO:tasks.workunit.client.1.smithi149.stdout:7/689: dread - d9/dc/d11/d84/da7/fd3 zero size 2024-04-24T08:24:56.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/424: creat d1a/d4b/d53/d79/d69/d78/f8c x:0 0 0 2024-04-24T08:24:56.541 INFO:tasks.workunit.client.1.smithi149.stdout:8/603: fdatasync d1/d1f/f69 0 2024-04-24T08:24:56.545 INFO:tasks.workunit.client.1.smithi149.stdout:8/604: write d1/de/d18/fb7 [439037,3389] 0 2024-04-24T08:24:56.546 INFO:tasks.workunit.client.1.smithi149.stdout:6/614: symlink d0/d1a/d7e/lcd 0 2024-04-24T08:24:56.548 INFO:tasks.workunit.client.1.smithi149.stdout:9/738: sync 2024-04-24T08:24:56.551 INFO:tasks.workunit.client.1.smithi149.stdout:5/790: creat db/d12/d2f/d74/f106 x:0 0 0 2024-04-24T08:24:56.553 INFO:tasks.workunit.client.1.smithi149.stdout:5/791: write db/f94 [513570,80777] 0 2024-04-24T08:24:56.555 INFO:tasks.workunit.client.1.smithi149.stdout:5/792: fdatasync db/d12/d60/f67 0 2024-04-24T08:24:56.556 INFO:tasks.workunit.client.0.smithi090.stdout:8/311: fdatasync d2/da/d13/d64/f16 0 2024-04-24T08:24:56.557 INFO:tasks.workunit.client.1.smithi149.stdout:2/631: mkdir d0/d6/db/d28/d7f/d8f/db9 0 2024-04-24T08:24:56.559 INFO:tasks.workunit.client.0.smithi090.stdout:2/374: dwrite d0/d14/f38 [4194304,4194304] 0 2024-04-24T08:24:56.560 INFO:tasks.workunit.client.0.smithi090.stdout:6/391: rmdir d8/d74 0 2024-04-24T08:24:56.562 INFO:tasks.workunit.client.0.smithi090.stdout:4/287: dwrite d3/d38/f42 [0,4194304] 0 2024-04-24T08:24:56.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/312: write d2/f59 [198870,127808] 0 2024-04-24T08:24:56.566 INFO:tasks.workunit.client.1.smithi149.stdout:4/631: getdents d2 0 2024-04-24T08:24:56.568 INFO:tasks.workunit.client.1.smithi149.stdout:3/760: dread d0/d5/d53/d34/d3a/d93/f105 [0,4194304] 0 2024-04-24T08:24:56.571 INFO:tasks.workunit.client.0.smithi090.stdout:5/447: write d8/da/d18/f2b [221197,114720] 0 2024-04-24T08:24:56.573 INFO:tasks.workunit.client.1.smithi149.stdout:1/804: write d0/d5/d4f/d60/f105 [959732,54211] 0 2024-04-24T08:24:56.576 INFO:tasks.workunit.client.0.smithi090.stdout:9/345: dread d0/dc/d24/f4b [0,4194304] 0 2024-04-24T08:24:56.577 INFO:tasks.workunit.client.0.smithi090.stdout:6/392: chown d8/dc/f11 1 1 2024-04-24T08:24:56.578 INFO:tasks.workunit.client.0.smithi090.stdout:9/346: write d0/f18 [1162842,111952] 0 2024-04-24T08:24:56.580 INFO:tasks.workunit.client.1.smithi149.stdout:1/805: chown d0/d5/f8 1559685262 1 2024-04-24T08:24:56.582 INFO:tasks.workunit.client.1.smithi149.stdout:9/739: dwrite d1/d2/d34/ff2 [0,4194304] 0 2024-04-24T08:24:56.586 INFO:tasks.workunit.client.1.smithi149.stdout:2/632: dread d0/d6/f16 [0,4194304] 0 2024-04-24T08:24:56.586 INFO:tasks.workunit.client.1.smithi149.stdout:4/632: fsync d2/d10/fa6 0 2024-04-24T08:24:56.589 INFO:tasks.workunit.client.1.smithi149.stdout:7/690: write d9/dc/d72/f76 [1973323,113416] 0 2024-04-24T08:24:56.591 INFO:tasks.workunit.client.1.smithi149.stdout:6/615: mknod d0/d1a/d28/d31/d4c/cce 0 2024-04-24T08:24:56.593 INFO:tasks.workunit.client.0.smithi090.stdout:1/389: creat d3/f8c x:0 0 0 2024-04-24T08:24:56.594 INFO:tasks.workunit.client.0.smithi090.stdout:0/425: creat d1a/d4b/d53/d79/d82/f8d x:0 0 0 2024-04-24T08:24:56.595 INFO:tasks.workunit.client.1.smithi149.stdout:0/633: dwrite d2/d9/df/d56/f80 [0,4194304] 0 2024-04-24T08:24:56.598 INFO:tasks.workunit.client.1.smithi149.stdout:4/633: chown d2/fc 420928454 1 2024-04-24T08:24:56.598 INFO:tasks.workunit.client.0.smithi090.stdout:6/393: rmdir d8/d1d/d1f 39 2024-04-24T08:24:56.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/390: truncate d3/d5/da/dc/f87 355081 0 2024-04-24T08:24:56.598 INFO:tasks.workunit.client.1.smithi149.stdout:3/761: mknod d0/d88/c106 0 2024-04-24T08:24:56.600 INFO:tasks.workunit.client.0.smithi090.stdout:1/391: chown d3/f41 20 1 2024-04-24T08:24:56.602 INFO:tasks.workunit.client.0.smithi090.stdout:9/347: creat d0/d19/d33/d4d/f5f x:0 0 0 2024-04-24T08:24:56.603 INFO:tasks.workunit.client.1.smithi149.stdout:5/793: dwrite db/d12/d60/db2/de2/ff3 [0,4194304] 0 2024-04-24T08:24:56.604 INFO:tasks.workunit.client.0.smithi090.stdout:2/375: fdatasync d0/d7/f32 0 2024-04-24T08:24:56.609 INFO:tasks.workunit.client.1.smithi149.stdout:9/740: unlink d1/d2/d15/d1c/f3a 0 2024-04-24T08:24:56.611 INFO:tasks.workunit.client.0.smithi090.stdout:4/288: link d3/f13 d3/d38/d49/f53 0 2024-04-24T08:24:56.618 INFO:tasks.workunit.client.1.smithi149.stdout:1/806: read d0/d5/db/d18/fc1 [2049258,72762] 0 2024-04-24T08:24:56.618 INFO:tasks.workunit.client.1.smithi149.stdout:9/741: write d1/da/f27 [1128515,125015] 0 2024-04-24T08:24:56.618 INFO:tasks.workunit.client.1.smithi149.stdout:8/605: link d1/de/d18/d32/d65/d8b/d7d/l89 d1/de/d18/d32/d63/d9e/lcd 0 2024-04-24T08:24:56.620 INFO:tasks.workunit.client.1.smithi149.stdout:9/742: readlink d1/da/l87 0 2024-04-24T08:24:56.624 INFO:tasks.workunit.client.1.smithi149.stdout:1/807: write d0/da/fe4 [773592,63874] 0 2024-04-24T08:24:56.624 INFO:tasks.workunit.client.0.smithi090.stdout:0/426: unlink d1a/d4b/d53/d79/d69/d78/f8c 0 2024-04-24T08:24:56.626 INFO:tasks.workunit.client.0.smithi090.stdout:0/427: chown d1a/d4b/d53/d79/d3c/f54 28 1 2024-04-24T08:24:56.629 INFO:tasks.workunit.client.1.smithi149.stdout:0/634: dwrite d2/d9/d1a/d36/f38 [0,4194304] 0 2024-04-24T08:24:56.629 INFO:tasks.workunit.client.0.smithi090.stdout:1/392: creat d3/d5/f8d x:0 0 0 2024-04-24T08:24:56.630 INFO:tasks.workunit.client.1.smithi149.stdout:7/691: dwrite d9/fe5 [0,4194304] 0 2024-04-24T08:24:56.631 INFO:tasks.workunit.client.0.smithi090.stdout:8/313: write d2/da/d13/f5f [616272,106032] 0 2024-04-24T08:24:56.636 INFO:tasks.workunit.client.1.smithi149.stdout:2/633: dread d0/d6/db/f1c [0,4194304] 0 2024-04-24T08:24:56.636 INFO:tasks.workunit.client.1.smithi149.stdout:0/635: truncate d2/d3c/fc6 1450665 0 2024-04-24T08:24:56.637 INFO:tasks.workunit.client.0.smithi090.stdout:8/314: read d2/da/f35 [948161,101999] 0 2024-04-24T08:24:56.640 INFO:tasks.workunit.client.1.smithi149.stdout:2/634: chown d0/d6/db/d39/d3d/d58 5 1 2024-04-24T08:24:56.641 INFO:tasks.workunit.client.0.smithi090.stdout:2/376: mkdir d0/d14/d36/d77 0 2024-04-24T08:24:56.642 INFO:tasks.workunit.client.0.smithi090.stdout:2/377: stat d0/d7/d70/d31/l73 0 2024-04-24T08:24:56.644 INFO:tasks.workunit.client.0.smithi090.stdout:4/289: symlink d3/d18/d1d/l54 0 2024-04-24T08:24:56.651 INFO:tasks.workunit.client.1.smithi149.stdout:5/794: rename db/d1f to db/d12/d1e/d85/d92/d107 0 2024-04-24T08:24:56.654 INFO:tasks.workunit.client.0.smithi090.stdout:6/394: symlink d8/dc/l78 0 2024-04-24T08:24:56.658 INFO:tasks.workunit.client.0.smithi090.stdout:9/348: mknod d0/c60 0 2024-04-24T08:24:56.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.657+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ab40788c0 con 0x7f2a9806eb10 2024-04-24T08:24:56.659 INFO:tasks.workunit.client.0.smithi090.stdout:4/290: dwrite d3/d38/f42 [0,4194304] 0 2024-04-24T08:24:56.661 INFO:tasks.workunit.client.1.smithi149.stdout:7/692: dwrite d9/dc/de/d44/fbe [0,4194304] 0 2024-04-24T08:24:56.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/743: mkdir d1/d2/d33/d60/d10b 0 2024-04-24T08:24:56.665 INFO:tasks.workunit.client.1.smithi149.stdout:6/616: mknod d0/d1a/d28/d31/d4c/ccf 0 2024-04-24T08:24:56.666 INFO:tasks.workunit.client.1.smithi149.stdout:3/762: getdents d0/d5/da/d19/d80/dac 0 2024-04-24T08:24:56.667 INFO:tasks.workunit.client.1.smithi149.stdout:9/744: readlink d1/da/l8e 0 2024-04-24T08:24:56.667 INFO:tasks.workunit.client.0.smithi090.stdout:1/393: getdents d3/d68/d7f 0 2024-04-24T08:24:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:7/693: stat d9/dc/d11/d38/dac/lbc 0 2024-04-24T08:24:56.671 INFO:tasks.workunit.client.1.smithi149.stdout:2/635: mkdir d0/d6/db/d27/d81/d9f/dba 0 2024-04-24T08:24:56.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.671+0000 7f2aaaa00700 1 -- 172.21.15.90:0/4277941183 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2ab40788c0 con 0x7f2a9806eb10 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 28s ago 11m 14.2M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (12m) 28s ago 12m 7838k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (10m) 26s ago 10m 7768k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (12m) 28s ago 12m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (10m) 26s ago 10m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (9m) 28s ago 11m 75.2M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (4m) 28s ago 4m 199M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (4m) 28s ago 4m 13.9M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:24:56.673 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (4m) 26s ago 4m 184M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (4m) 26s ago 4m 14.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (53s) 28s ago 13m 477M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (114s) 26s ago 10m 65.7M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 839fd9b0cdf9 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (13m) 28s ago 13m 53.0M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (10m) 26s ago 10m 42.1M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (12m) 28s ago 12m 8116k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (10m) 26s ago 10m 7537k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (9m) 28s ago 9m 85.2M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (8m) 28s ago 8m 89.3M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (8m) 28s ago 8m 75.9M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (7m) 26s ago 7m 96.0M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:24:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (7m) 26s ago 7m 89.4M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:24:56.675 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (6m) 26s ago 6m 85.7M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:24:56.675 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (62s) 28s ago 11m 47.9M - 2.43.0 a07b618ecd1d 541c9856b6ae 2024-04-24T08:24:56.677 INFO:tasks.workunit.client.1.smithi149.stdout:1/808: rename d0/d5/db/f8c to d0/d1b/d5d/de5/d101/f109 0 2024-04-24T08:24:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 msgr2=0x7f2a98070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 0x7f2a98070fc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0005fb0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 msgr2=0x7f2ab410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400b480 tx=0x7f2aa400b840 comp rx=0 tx=0).stop 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 shutdown_connections 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f2a9806eb10 0x7f2a98070fc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab4102490 0x7f2ab410be90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 --2- 172.21.15.90:0/4277941183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab4103640 0x7f2ab410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.676+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 >> 172.21.15.90:0/4277941183 conn(0x7f2ab40fda60 msgr2=0x7f2ab40771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.677+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 shutdown_connections 2024-04-24T08:24:56.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.677+0000 7f2ab9200700 1 -- 172.21.15.90:0/4277941183 wait complete. 2024-04-24T08:24:56.680 INFO:tasks.workunit.client.1.smithi149.stdout:1/809: dread d0/ddf/da0/fe0 [0,4194304] 0 2024-04-24T08:24:56.680 INFO:tasks.workunit.client.1.smithi149.stdout:8/606: mkdir d1/dce 0 2024-04-24T08:24:56.684 INFO:tasks.workunit.client.1.smithi149.stdout:3/763: rmdir d0/d5/d1a/d70/da3 39 2024-04-24T08:24:56.687 INFO:tasks.workunit.client.0.smithi090.stdout:8/315: getdents d2/da/d13 0 2024-04-24T08:24:56.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/694: dread - d9/dc/d7b/fb2 zero size 2024-04-24T08:24:56.696 INFO:tasks.workunit.client.0.smithi090.stdout:4/291: truncate d3/f13 1511108 0 2024-04-24T08:24:56.697 INFO:tasks.workunit.client.0.smithi090.stdout:4/292: chown d3/d38/d49 171808018 1 2024-04-24T08:24:56.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/316: creat d2/f69 x:0 0 0 2024-04-24T08:24:56.698 INFO:tasks.workunit.client.1.smithi149.stdout:3/764: dread - d0/d26/d3c/db1/fb6 zero size 2024-04-24T08:24:56.700 INFO:tasks.workunit.client.0.smithi090.stdout:4/293: symlink d3/d38/l55 0 2024-04-24T08:24:56.702 INFO:tasks.workunit.client.0.smithi090.stdout:8/317: symlink d2/da/d13/d60/l6a 0 2024-04-24T08:24:56.702 INFO:tasks.workunit.client.0.smithi090.stdout:4/294: write d3/d38/d49/f4f [1549836,87742] 0 2024-04-24T08:24:56.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/335: sync 2024-04-24T08:24:56.707 INFO:tasks.workunit.client.1.smithi149.stdout:9/745: dread d1/d2/d34/ff2 [0,4194304] 0 2024-04-24T08:24:56.713 INFO:tasks.workunit.client.1.smithi149.stdout:0/636: dread d2/d9/df/d2f/d6c/dc1/d4e/f6d [0,4194304] 0 2024-04-24T08:24:56.714 INFO:tasks.workunit.client.0.smithi090.stdout:2/378: write d0/d7/f11 [101023,1657] 0 2024-04-24T08:24:56.717 INFO:tasks.workunit.client.0.smithi090.stdout:3/336: fdatasync d2/d15/f16 0 2024-04-24T08:24:56.718 INFO:tasks.workunit.client.1.smithi149.stdout:4/634: truncate d2/d10/f27 1008403 0 2024-04-24T08:24:56.719 INFO:tasks.workunit.client.0.smithi090.stdout:9/349: dread d0/d19/f56 [0,4194304] 0 2024-04-24T08:24:56.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/765: rename d0/d5/da/d19/d80/f8c to d0/d5/da/d19/d1b/d72/f107 0 2024-04-24T08:24:56.721 INFO:tasks.workunit.client.0.smithi090.stdout:4/295: mknod d3/d18/c56 0 2024-04-24T08:24:56.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/318: mkdir d2/d62/d6b 0 2024-04-24T08:24:56.725 INFO:tasks.workunit.client.1.smithi149.stdout:9/746: write d1/d2/d34/f70 [789510,87662] 0 2024-04-24T08:24:56.727 INFO:tasks.workunit.client.1.smithi149.stdout:9/747: chown d1/d2/f97 330 1 2024-04-24T08:24:56.730 INFO:tasks.workunit.client.0.smithi090.stdout:6/395: dread d8/d1d/d1f/f58 [4194304,4194304] 0 2024-04-24T08:24:56.731 INFO:tasks.workunit.client.0.smithi090.stdout:3/337: symlink d2/df/d1a/d37/d59/l6a 0 2024-04-24T08:24:56.733 INFO:tasks.workunit.client.1.smithi149.stdout:0/637: creat d2/d9/df/d2f/d6c/dc1/fcf x:0 0 0 2024-04-24T08:24:56.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/350: symlink d0/d2a/l61 0 2024-04-24T08:24:56.739 INFO:tasks.workunit.client.0.smithi090.stdout:8/319: dread d2/da/d13/f3a [0,4194304] 0 2024-04-24T08:24:56.742 INFO:tasks.workunit.client.1.smithi149.stdout:7/695: dread d9/dc/de/f31 [0,4194304] 0 2024-04-24T08:24:56.742 INFO:tasks.workunit.client.1.smithi149.stdout:5/795: dwrite db/d23/daa/fab [0,4194304] 0 2024-04-24T08:24:56.746 INFO:tasks.workunit.client.0.smithi090.stdout:3/338: dwrite d2/d25/f48 [0,4194304] 0 2024-04-24T08:24:56.749 INFO:tasks.workunit.client.0.smithi090.stdout:9/351: rmdir d0/dc/d24/d3d 39 2024-04-24T08:24:56.753 INFO:tasks.workunit.client.0.smithi090.stdout:4/296: creat d3/d18/d1d/d4b/f57 x:0 0 0 2024-04-24T08:24:56.753 INFO:tasks.workunit.client.1.smithi149.stdout:6/617: dwrite d0/f10 [0,4194304] 0 2024-04-24T08:24:56.756 INFO:tasks.workunit.client.1.smithi149.stdout:9/748: rename d1/da/d13/d74/lc3 to d1/d2/d33/d5d/d83/dbf/l10c 0 2024-04-24T08:24:56.757 INFO:tasks.workunit.client.1.smithi149.stdout:1/810: dwrite d0/d1b/d5d/f69 [0,4194304] 0 2024-04-24T08:24:56.759 INFO:tasks.workunit.client.0.smithi090.stdout:8/320: rmdir d2/da/d13 39 2024-04-24T08:24:56.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/339: creat d2/df/d1a/d37/d53/f6b x:0 0 0 2024-04-24T08:24:56.764 INFO:tasks.workunit.client.0.smithi090.stdout:9/352: rmdir d0/dc 39 2024-04-24T08:24:56.767 INFO:tasks.workunit.client.1.smithi149.stdout:8/607: link d1/d1f/d25/f78 d1/de/d18/daf/fcf 0 2024-04-24T08:24:56.772 INFO:tasks.workunit.client.0.smithi090.stdout:2/379: getdents d0/d14 0 2024-04-24T08:24:56.774 INFO:tasks.workunit.client.0.smithi090.stdout:2/380: chown d0/d7/d70/f5d 2315 1 2024-04-24T08:24:56.775 INFO:tasks.workunit.client.1.smithi149.stdout:5/796: truncate db/d35/f9d 626067 0 2024-04-24T08:24:56.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:56 smithi149 ceph-mon[31282]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:56.780 INFO:tasks.workunit.client.1.smithi149.stdout:1/811: creat d0/d1b/db5/d108/f10a x:0 0 0 2024-04-24T08:24:56.781 INFO:tasks.workunit.client.0.smithi090.stdout:2/381: rmdir d0/d14 39 2024-04-24T08:24:56.782 INFO:tasks.workunit.client.1.smithi149.stdout:7/696: mkdir d9/d43/d4f/dc8/dea 0 2024-04-24T08:24:56.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/382: read - d0/d2e/f6a zero size 2024-04-24T08:24:56.784 INFO:tasks.workunit.client.1.smithi149.stdout:9/749: dwrite d1/d2/d15/f81 [0,4194304] 0 2024-04-24T08:24:56.787 INFO:tasks.workunit.client.0.smithi090.stdout:2/383: creat d0/d7/d70/f78 x:0 0 0 2024-04-24T08:24:56.787 INFO:tasks.workunit.client.1.smithi149.stdout:7/697: dread - d9/d43/d4f/f92 zero size 2024-04-24T08:24:56.789 INFO:tasks.workunit.client.0.smithi090.stdout:8/321: dread d2/da/d13/d34/f4d [0,4194304] 0 2024-04-24T08:24:56.792 INFO:tasks.workunit.client.0.smithi090.stdout:2/384: mkdir d0/d7/d57/d79 0 2024-04-24T08:24:56.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:56 smithi090 ceph-mon[25016]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:56.793 INFO:tasks.workunit.client.1.smithi149.stdout:4/635: getdents d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae 0 2024-04-24T08:24:56.793 INFO:tasks.workunit.client.1.smithi149.stdout:8/608: mknod d1/d1f/d5e/cd0 0 2024-04-24T08:24:56.793 INFO:tasks.workunit.client.1.smithi149.stdout:3/766: dwrite d0/d5/d53/d34/f95 [0,4194304] 0 2024-04-24T08:24:56.795 INFO:tasks.workunit.client.1.smithi149.stdout:7/698: stat d9/dc/d11/d83/da6/d67/d69 0 2024-04-24T08:24:56.798 INFO:tasks.workunit.client.1.smithi149.stdout:1/812: unlink d0/d5/d4f/f77 0 2024-04-24T08:24:56.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/322: rename d2/da/d13/d64/f49 to d2/da/d13/d64/d50/f6c 0 2024-04-24T08:24:56.800 INFO:tasks.workunit.client.1.smithi149.stdout:1/813: chown d0/da 17217687 1 2024-04-24T08:24:56.800 INFO:tasks.workunit.client.1.smithi149.stdout:9/750: rmdir d1/de/d9d 39 2024-04-24T08:24:56.802 INFO:tasks.workunit.client.1.smithi149.stdout:4/636: chown d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/lc3 98983256 1 2024-04-24T08:24:56.803 INFO:tasks.workunit.client.0.smithi090.stdout:3/340: read d2/f17 [195018,38252] 0 2024-04-24T08:24:56.803 INFO:tasks.workunit.client.0.smithi090.stdout:2/385: fsync d0/d7/f32 0 2024-04-24T08:24:56.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/797: mknod db/d12/d22/d45/d5d/c108 0 2024-04-24T08:24:56.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/638: dwrite d2/d9/df/f5d [0,4194304] 0 2024-04-24T08:24:56.806 INFO:tasks.workunit.client.1.smithi149.stdout:9/751: write d1/d2/d15/f81 [3219020,57726] 0 2024-04-24T08:24:56.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/609: rmdir d1/d1f/d5e/d9c 39 2024-04-24T08:24:56.818 INFO:tasks.workunit.client.1.smithi149.stdout:4/637: read d2/d24/d2e/d38/f9c [1304189,99645] 0 2024-04-24T08:24:56.819 INFO:tasks.workunit.client.0.smithi090.stdout:2/386: dread d0/d7/f64 [0,4194304] 0 2024-04-24T08:24:56.821 INFO:tasks.workunit.client.1.smithi149.stdout:0/639: dread - d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fa8 zero size 2024-04-24T08:24:56.826 INFO:tasks.workunit.client.1.smithi149.stdout:6/618: rename d0/d1a/d28/d31/d4c/d76/fb4 to d0/d1a/d28/d31/d4c/d80/fd0 0 2024-04-24T08:24:56.835 INFO:tasks.workunit.client.1.smithi149.stdout:1/814: dread d0/d5/d4f/d60/f78 [0,4194304] 0 2024-04-24T08:24:56.835 INFO:tasks.workunit.client.0.smithi090.stdout:8/323: creat d2/da/d13/d64/f6d x:0 0 0 2024-04-24T08:24:56.842 INFO:tasks.workunit.client.0.smithi090.stdout:2/387: rename d0/d14/d4d/d50/c5f to d0/d7/d6f/c7a 0 2024-04-24T08:24:56.845 INFO:tasks.workunit.client.0.smithi090.stdout:8/324: creat d2/d62/f6e x:0 0 0 2024-04-24T08:24:56.847 INFO:tasks.workunit.client.1.smithi149.stdout:4/638: fsync d2/d10/fa6 0 2024-04-24T08:24:56.852 INFO:tasks.workunit.client.1.smithi149.stdout:0/640: rmdir d2/d9/df/d2f/d6c/dc1/d4e/d6f 39 2024-04-24T08:24:56.854 INFO:tasks.workunit.client.1.smithi149.stdout:6/619: mknod d0/d1a/d28/d31/d4c/d4e/d62/d66/cd1 0 2024-04-24T08:24:56.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/341: link d2/df/d57/l40 d2/df/l6c 0 2024-04-24T08:24:56.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/353: dread d0/dc/f2c [0,4194304] 0 2024-04-24T08:24:56.859 INFO:tasks.workunit.client.0.smithi090.stdout:8/325: rename d2/f22 to d2/da/d13/d34/f6f 0 2024-04-24T08:24:56.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/297: write d3/f9 [552986,93827] 0 2024-04-24T08:24:56.862 INFO:tasks.workunit.client.0.smithi090.stdout:3/342: creat d2/df/d1a/f6d x:0 0 0 2024-04-24T08:24:56.863 INFO:tasks.workunit.client.0.smithi090.stdout:4/298: dread - d3/d18/d1d/d22/f51 zero size 2024-04-24T08:24:56.867 INFO:tasks.workunit.client.1.smithi149.stdout:9/752: mknod d1/da/d13/d74/dbb/dc0/c10d 0 2024-04-24T08:24:56.868 INFO:tasks.workunit.client.0.smithi090.stdout:2/388: dwrite d0/d7/d70/d31/d46/d60/f74 [0,4194304] 0 2024-04-24T08:24:56.872 INFO:tasks.workunit.client.0.smithi090.stdout:9/354: truncate d0/dc/d24/d3d/f5c 1119323 0 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.876+0000 7f3095000700 1 -- 172.21.15.90:0/2236620914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090071e00 msgr2=0x7f3090072210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.876+0000 7f3095000700 1 --2- 172.21.15.90:0/2236620914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090071e00 0x7f3090072210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3080009910 tx=0x7f3080009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 -- 172.21.15.90:0/2236620914 shutdown_connections 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 --2- 172.21.15.90:0/2236620914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3090072750 0x7f3090107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 --2- 172.21.15.90:0/2236620914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090071e00 0x7f3090072210 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 -- 172.21.15.90:0/2236620914 >> 172.21.15.90:0/2236620914 conn(0x7f309006d110 msgr2=0x7f309006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 -- 172.21.15.90:0/2236620914 shutdown_connections 2024-04-24T08:24:56.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.877+0000 7f3095000700 1 -- 172.21.15.90:0/2236620914 wait complete. 2024-04-24T08:24:56.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.878+0000 7f3095000700 1 Processor -- start 2024-04-24T08:24:56.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.879+0000 7f3095000700 1 -- start start 2024-04-24T08:24:56.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.879+0000 7f3095000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.879+0000 7f3095000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30901a0d00 0x7f30901a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.879+0000 7f3095000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30901a1200 con 0x7f3090072750 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.879+0000 7f3095000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30901a1370 con 0x7f30901a0d00 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44520/0 (socket says 172.21.15.90:44520) 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30901a0d00 0x7f30901a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 -- 172.21.15.90:0/1707392271 learned_addr learned my addr 172.21.15.90:0/1707392271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 -- 172.21.15.90:0/1707392271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30901a0d00 msgr2=0x7f30901a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30901a0d00 0x7f30901a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.880+0000 7f308de00700 1 -- 172.21.15.90:0/1707392271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800095f0 con 0x7f3090072750 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f308de00700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3080000c00 tx=0x7f308000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308001d070 con 0x7f3090072750 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f308000f460 con 0x7f3090072750 2024-04-24T08:24:56.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30901a62b0 con 0x7f3090072750 2024-04-24T08:24:56.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3080021620 con 0x7f3090072750 2024-04-24T08:24:56.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.881+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30901a6770 con 0x7f3090072750 2024-04-24T08:24:56.883 INFO:tasks.workunit.client.0.smithi090.stdout:4/299: mknod d3/d18/d1d/d22/c58 0 2024-04-24T08:24:56.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.883+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3090113830 con 0x7f3090072750 2024-04-24T08:24:56.885 INFO:tasks.workunit.client.1.smithi149.stdout:4/639: fsync d2/d24/d2e/f4a 0 2024-04-24T08:24:56.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.884+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f308000fa60 con 0x7f3090072750 2024-04-24T08:24:56.886 INFO:tasks.workunit.client.0.smithi090.stdout:9/355: dread d0/dc/f2c [0,4194304] 0 2024-04-24T08:24:56.886 INFO:tasks.workunit.client.1.smithi149.stdout:0/641: symlink d2/d9/df/d48/ld0 0 2024-04-24T08:24:56.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.886+0000 7f3086a00700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 0x7f3074070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:56.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.886+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f308008fce0 con 0x7f3090072750 2024-04-24T08:24:56.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.886+0000 7f308d400700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 0x7f3074070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:56.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.887+0000 7f308d400700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 0x7f3074070f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3090072b80 tx=0x7f307c009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:56.888 INFO:tasks.workunit.client.1.smithi149.stdout:6/620: fsync d0/d1a/d28/f2c 0 2024-04-24T08:24:56.892 INFO:tasks.workunit.client.1.smithi149.stdout:6/621: stat d0/d1a/d28/d31/f5c 0 2024-04-24T08:24:56.898 INFO:tasks.workunit.client.1.smithi149.stdout:9/753: creat d1/da/d4a/dad/f10e x:0 0 0 2024-04-24T08:24:56.898 INFO:tasks.workunit.client.1.smithi149.stdout:5/798: write db/d12/f69 [124265,108456] 0 2024-04-24T08:24:56.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:56.893+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f308005c900 con 0x7f3090072750 2024-04-24T08:24:56.899 INFO:tasks.workunit.client.0.smithi090.stdout:2/389: mkdir d0/d7/d57/d79/d7b 0 2024-04-24T08:24:56.900 INFO:tasks.workunit.client.1.smithi149.stdout:7/699: dwrite d9/dc/de/d44/fab [0,4194304] 0 2024-04-24T08:24:56.902 INFO:tasks.workunit.client.0.smithi090.stdout:8/326: creat d2/da/f70 x:0 0 0 2024-04-24T08:24:56.903 INFO:tasks.workunit.client.0.smithi090.stdout:7/420: sync 2024-04-24T08:24:56.903 INFO:tasks.workunit.client.1.smithi149.stdout:3/767: link d0/d5/da/d5a/d63/d6a/c71 d0/d5/d53/d34/d3a/d93/c108 0 2024-04-24T08:24:56.907 INFO:tasks.workunit.client.0.smithi090.stdout:9/356: rename d0/d19/d33/d50 to d0/dc/d24/d2d/d49/d62 0 2024-04-24T08:24:56.911 INFO:tasks.workunit.client.0.smithi090.stdout:4/300: dwrite d3/d18/d1d/d22/f51 [0,4194304] 0 2024-04-24T08:24:56.911 INFO:tasks.workunit.client.0.smithi090.stdout:3/343: creat d2/df/f6e x:0 0 0 2024-04-24T08:24:56.914 INFO:tasks.workunit.client.0.smithi090.stdout:2/390: mknod d0/d2e/c7c 0 2024-04-24T08:24:56.915 INFO:tasks.workunit.client.0.smithi090.stdout:2/391: fdatasync d0/d7/d67/f72 0 2024-04-24T08:24:56.916 INFO:tasks.workunit.client.0.smithi090.stdout:8/327: mkdir d2/d62/d71 0 2024-04-24T08:24:56.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/328: fdatasync d2/f69 0 2024-04-24T08:24:56.918 INFO:tasks.workunit.client.0.smithi090.stdout:8/329: fdatasync d2/f3 0 2024-04-24T08:24:56.926 INFO:tasks.workunit.client.1.smithi149.stdout:4/640: dread d2/d10/d91/f9d [0,4194304] 0 2024-04-24T08:24:56.929 INFO:tasks.workunit.client.1.smithi149.stdout:9/754: write d1/de/ff [512794,89533] 0 2024-04-24T08:24:56.932 INFO:tasks.workunit.client.1.smithi149.stdout:9/755: chown d1/da/d13/l1f 2012201 1 2024-04-24T08:24:56.934 INFO:tasks.workunit.client.0.smithi090.stdout:7/421: dread db/d17/d29/d38/d3e/f3f [0,4194304] 0 2024-04-24T08:24:56.934 INFO:tasks.workunit.client.1.smithi149.stdout:5/799: rmdir db/d12/d1e/d65/de8 39 2024-04-24T08:24:56.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/636: sync 2024-04-24T08:24:56.936 INFO:tasks.workunit.client.0.smithi090.stdout:8/330: creat d2/da/d13/d34/f72 x:0 0 0 2024-04-24T08:24:56.936 INFO:tasks.workunit.client.0.smithi090.stdout:7/422: chown db/d54/d78/l7d 33478696 1 2024-04-24T08:24:56.937 INFO:tasks.workunit.client.0.smithi090.stdout:7/423: readlink l5 0 2024-04-24T08:24:56.937 INFO:tasks.workunit.client.1.smithi149.stdout:2/637: fsync d0/d6/db/f9e 0 2024-04-24T08:24:56.939 INFO:tasks.workunit.client.1.smithi149.stdout:3/768: unlink d0/d5/da/ff6 0 2024-04-24T08:24:56.944 INFO:tasks.workunit.client.0.smithi090.stdout:4/301: link d3/c19 d3/d38/c59 0 2024-04-24T08:24:56.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/424: truncate db/d17/f3a 177590 0 2024-04-24T08:24:56.947 INFO:tasks.workunit.client.0.smithi090.stdout:4/302: readlink d3/d25/l48 0 2024-04-24T08:24:56.948 INFO:tasks.workunit.client.0.smithi090.stdout:9/357: dread d0/d19/d33/f36 [0,4194304] 0 2024-04-24T08:24:56.948 INFO:tasks.workunit.client.1.smithi149.stdout:4/641: write d2/d10/d1a/fbe [4899186,123060] 0 2024-04-24T08:24:56.949 INFO:tasks.workunit.client.1.smithi149.stdout:9/756: truncate d1/de/fa8 34698 0 2024-04-24T08:24:56.950 INFO:tasks.workunit.client.1.smithi149.stdout:6/622: write d0/db7/fb9 [4219962,64165] 0 2024-04-24T08:24:56.952 INFO:tasks.workunit.client.1.smithi149.stdout:5/800: symlink db/d12/d60/db2/de2/l109 0 2024-04-24T08:24:56.954 INFO:tasks.workunit.client.0.smithi090.stdout:8/331: dwrite d2/f59 [0,4194304] 0 2024-04-24T08:24:56.955 INFO:tasks.workunit.client.0.smithi090.stdout:4/303: mknod d3/d18/d34/c5a 0 2024-04-24T08:24:56.958 INFO:tasks.workunit.client.1.smithi149.stdout:3/769: creat d0/d26/d3c/f109 x:0 0 0 2024-04-24T08:24:56.959 INFO:tasks.workunit.client.1.smithi149.stdout:3/770: chown d0/d5/da/d59 1145952 1 2024-04-24T08:24:56.959 INFO:tasks.workunit.client.0.smithi090.stdout:2/392: truncate d0/d7/d70/d31/f41 4125209 0 2024-04-24T08:24:56.960 INFO:tasks.workunit.client.1.smithi149.stdout:1/815: link d0/d1b/db5/d108/l7d d0/ddf/d53/l10b 0 2024-04-24T08:24:56.961 INFO:tasks.workunit.client.0.smithi090.stdout:2/393: chown d0/d7/d70/d31/d46/l54 12601 1 2024-04-24T08:24:56.962 INFO:tasks.workunit.client.0.smithi090.stdout:2/394: chown d0/d7/l1e 0 1 2024-04-24T08:24:56.964 INFO:tasks.workunit.client.1.smithi149.stdout:1/816: write d0/d5/db/d18/d2f/fa4 [1069475,16765] 0 2024-04-24T08:24:56.966 INFO:tasks.workunit.client.0.smithi090.stdout:8/332: mknod d2/da/d13/c73 0 2024-04-24T08:24:56.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/610: getdents d1/d1f 0 2024-04-24T08:24:56.971 INFO:tasks.workunit.client.1.smithi149.stdout:4/642: fdatasync d2/d24/d2e/f4a 0 2024-04-24T08:24:56.972 INFO:tasks.workunit.client.0.smithi090.stdout:9/358: symlink d0/dc/d24/l63 0 2024-04-24T08:24:56.973 INFO:tasks.workunit.client.0.smithi090.stdout:7/425: getdents db/d17/d29/d38 0 2024-04-24T08:24:56.974 INFO:tasks.workunit.client.0.smithi090.stdout:8/333: mknod d2/d62/c74 0 2024-04-24T08:24:56.976 INFO:tasks.workunit.client.0.smithi090.stdout:8/334: fdatasync d2/f69 0 2024-04-24T08:24:56.976 INFO:tasks.workunit.client.1.smithi149.stdout:6/623: fsync d0/d1a/d28/d31/f37 0 2024-04-24T08:24:56.977 INFO:tasks.workunit.client.0.smithi090.stdout:7/426: read db/d17/d29/d32/f4c [294985,117173] 0 2024-04-24T08:24:56.979 INFO:tasks.workunit.client.1.smithi149.stdout:2/638: dwrite d0/d6/db/d27/f5f [0,4194304] 0 2024-04-24T08:24:56.979 INFO:tasks.workunit.client.0.smithi090.stdout:7/427: readlink db/d17/d29/d32/d5a/d75/l1e 0 2024-04-24T08:24:56.980 INFO:tasks.workunit.client.1.smithi149.stdout:5/801: creat db/d12/d2f/d78/f10a x:0 0 0 2024-04-24T08:24:56.980 INFO:tasks.workunit.client.0.smithi090.stdout:2/395: dwrite d0/d2e/f37 [0,4194304] 0 2024-04-24T08:24:56.983 INFO:tasks.workunit.client.1.smithi149.stdout:6/624: fdatasync d0/d1a/d28/d31/f5c 0 2024-04-24T08:24:56.983 INFO:tasks.workunit.client.1.smithi149.stdout:5/802: readlink db/d12/d2f/d95/l101 0 2024-04-24T08:24:56.984 INFO:tasks.workunit.client.1.smithi149.stdout:3/771: rmdir d0/d5/da/d19/d1b/d72 39 2024-04-24T08:24:56.987 INFO:tasks.workunit.client.1.smithi149.stdout:3/772: chown d0/d5/da/d5a/d9c/cf7 315 1 2024-04-24T08:24:56.990 INFO:tasks.workunit.client.0.smithi090.stdout:4/304: creat d3/d18/f5b x:0 0 0 2024-04-24T08:24:56.993 INFO:tasks.workunit.client.0.smithi090.stdout:4/305: write d3/d38/d49/f4f [1791912,67297] 0 2024-04-24T08:24:56.995 INFO:tasks.workunit.client.1.smithi149.stdout:0/642: rename d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fa8 to d2/d17/fd1 0 2024-04-24T08:24:56.999 INFO:tasks.workunit.client.0.smithi090.stdout:7/428: dread db/d17/f2e [0,4194304] 0 2024-04-24T08:24:56.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/817: creat d0/d1b/db5/d108/f10c x:0 0 0 2024-04-24T08:24:57.002 INFO:tasks.workunit.client.1.smithi149.stdout:8/611: read - d1/de/d18/d91/d9f/fa3 zero size 2024-04-24T08:24:57.004 INFO:tasks.workunit.client.0.smithi090.stdout:2/396: mknod d0/d2c/c7d 0 2024-04-24T08:24:57.005 INFO:tasks.workunit.client.0.smithi090.stdout:2/397: fsync d0/d14/f58 0 2024-04-24T08:24:57.008 INFO:tasks.workunit.client.1.smithi149.stdout:4/643: truncate d2/d18/d1e/d3c/d48/d69/f77 126481 0 2024-04-24T08:24:57.014 INFO:tasks.workunit.client.1.smithi149.stdout:2/639: dread d0/d6/db/d39/d3d/d44/f53 [0,4194304] 0 2024-04-24T08:24:57.019 INFO:tasks.workunit.client.0.smithi090.stdout:9/359: creat d0/dc/f64 x:0 0 0 2024-04-24T08:24:57.020 INFO:tasks.workunit.client.1.smithi149.stdout:5/803: rename db/d12/d2f/d78/l99 to db/d12/d1e/d85/l10b 0 2024-04-24T08:24:57.020 INFO:tasks.workunit.client.1.smithi149.stdout:6/625: rename d0/d1a/d28/d31/d4c to d0/d1a/d28/d31/d4c/d8a/dd2 22 2024-04-24T08:24:57.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/643: symlink d2/d9/df/d2f/d6c/d72/d91/ld2 0 2024-04-24T08:24:57.023 INFO:tasks.workunit.client.1.smithi149.stdout:8/612: dwrite d1/d1f/d31/f67 [0,4194304] 0 2024-04-24T08:24:57.023 INFO:tasks.workunit.client.0.smithi090.stdout:7/429: truncate db/d17/d39/f3c 2533636 0 2024-04-24T08:24:57.025 INFO:tasks.workunit.client.0.smithi090.stdout:9/360: write d0/f1b [3399039,85201] 0 2024-04-24T08:24:57.028 INFO:tasks.workunit.client.0.smithi090.stdout:8/335: rename d2/da/d13/d64/f26 to d2/da/d13/d60/f75 0 2024-04-24T08:24:57.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/757: creat d1/de/f10f x:0 0 0 2024-04-24T08:24:57.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/398: dread d0/d14/d17/f62 [0,4194304] 0 2024-04-24T08:24:57.034 INFO:tasks.workunit.client.0.smithi090.stdout:4/306: rename d3/d18/d34 to d3/d18/d5c 0 2024-04-24T08:24:57.036 INFO:tasks.workunit.client.1.smithi149.stdout:7/700: dread d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:57.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/307: read d3/d38/f42 [1813718,1789] 0 2024-04-24T08:24:57.038 INFO:tasks.workunit.client.1.smithi149.stdout:4/644: mknod d2/d18/cc5 0 2024-04-24T08:24:57.038 INFO:tasks.workunit.client.0.smithi090.stdout:4/308: fdatasync d3/d18/d1d/d4b/f57 0 2024-04-24T08:24:57.039 INFO:tasks.workunit.client.1.smithi149.stdout:7/701: readlink d9/dc/d7b/laa 0 2024-04-24T08:24:57.042 INFO:tasks.workunit.client.0.smithi090.stdout:4/309: read d3/d25/f2f [3034032,106300] 0 2024-04-24T08:24:57.045 INFO:tasks.workunit.client.1.smithi149.stdout:3/773: symlink d0/d5/da/d5a/d63/d6a/df5/l10a 0 2024-04-24T08:24:57.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/774: chown d0/d5/fc7 54 1 2024-04-24T08:24:57.047 INFO:tasks.workunit.client.1.smithi149.stdout:2/640: unlink d0/d6/db/d39/fa6 0 2024-04-24T08:24:57.050 INFO:tasks.workunit.client.1.smithi149.stdout:3/775: write d0/d26/d3c/f109 [297580,66507] 0 2024-04-24T08:24:57.051 INFO:tasks.workunit.client.0.smithi090.stdout:7/430: creat db/d17/d39/d86/f92 x:0 0 0 2024-04-24T08:24:57.052 INFO:tasks.workunit.client.1.smithi149.stdout:0/644: mknod d2/d9/df/d2f/d6c/dc1/cd3 0 2024-04-24T08:24:57.054 INFO:tasks.workunit.client.1.smithi149.stdout:1/818: dwrite d0/d1b/f91 [0,4194304] 0 2024-04-24T08:24:57.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/361: link d0/d19/f28 d0/dc/d24/d2d/f65 0 2024-04-24T08:24:57.056 INFO:tasks.workunit.client.1.smithi149.stdout:8/613: mkdir d1/de/d18/d32/d65/d8b/d7d/dd1 0 2024-04-24T08:24:57.059 INFO:tasks.workunit.client.1.smithi149.stdout:9/758: read - d1/de/d9d/dfd/f101 zero size 2024-04-24T08:24:57.060 INFO:tasks.workunit.client.0.smithi090.stdout:2/399: unlink d0/c75 0 2024-04-24T08:24:57.061 INFO:tasks.workunit.client.1.smithi149.stdout:4/645: symlink d2/d10/d1a/da2/lc6 0 2024-04-24T08:24:57.061 INFO:tasks.workunit.client.0.smithi090.stdout:0/428: sync 2024-04-24T08:24:57.061 INFO:tasks.workunit.client.0.smithi090.stdout:5/448: sync 2024-04-24T08:24:57.062 INFO:tasks.workunit.client.0.smithi090.stdout:4/310: dwrite d3/d18/d1d/d22/f35 [0,4194304] 0 2024-04-24T08:24:57.063 INFO:tasks.workunit.client.1.smithi149.stdout:7/702: symlink d9/dc/d7b/leb 0 2024-04-24T08:24:57.063 INFO:tasks.workunit.client.1.smithi149.stdout:4/646: fdatasync d2/d24/d2e/d3f/fa9 0 2024-04-24T08:24:57.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/449: rename d8/da/d14/d2f/f42 to d8/da/d18/d21/d4c/f9a 0 2024-04-24T08:24:57.078 INFO:tasks.workunit.client.1.smithi149.stdout:3/776: creat d0/d5/d53/d4c/d5f/f10b x:0 0 0 2024-04-24T08:24:57.080 INFO:tasks.workunit.client.0.smithi090.stdout:9/362: dwrite d0/dc/fd [0,4194304] 0 2024-04-24T08:24:57.081 INFO:tasks.workunit.client.1.smithi149.stdout:0/645: creat d2/d3c/fd4 x:0 0 0 2024-04-24T08:24:57.083 INFO:tasks.workunit.client.0.smithi090.stdout:5/450: write d8/da/d14/d2f/d44/f70 [983007,15115] 0 2024-04-24T08:24:57.083 INFO:tasks.workunit.client.1.smithi149.stdout:0/646: write d2/d3c/d57/f78 [4999676,23663] 0 2024-04-24T08:24:57.085 INFO:tasks.workunit.client.1.smithi149.stdout:1/819: unlink d0/ddf/da0/f5e 0 2024-04-24T08:24:57.088 INFO:tasks.workunit.client.0.smithi090.stdout:0/429: dwrite d1a/f30 [4194304,4194304] 0 2024-04-24T08:24:57.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/451: chown d8/da/d14/d2f/d44/f49 62 1 2024-04-24T08:24:57.089 INFO:tasks.workunit.client.1.smithi149.stdout:8/614: fsync d1/de/d18/f4c 0 2024-04-24T08:24:57.093 INFO:tasks.workunit.client.0.smithi090.stdout:7/431: rmdir db/d17/d39/d86 39 2024-04-24T08:24:57.093 INFO:tasks.workunit.client.0.smithi090.stdout:0/430: chown d1a/d4b/d53/d79/d69/d78 306 1 2024-04-24T08:24:57.095 INFO:tasks.workunit.client.1.smithi149.stdout:6/626: link d0/fb d0/d1a/d28/d31/d4c/d80/d6f/dc2/fd3 0 2024-04-24T08:24:57.100 INFO:tasks.workunit.client.0.smithi090.stdout:2/400: getdents d0/d14/d4d/d50 0 2024-04-24T08:24:57.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/431: rmdir d1a/d4b 39 2024-04-24T08:24:57.108 INFO:tasks.workunit.client.1.smithi149.stdout:7/703: symlink d9/d43/d95/lec 0 2024-04-24T08:24:57.109 INFO:tasks.workunit.client.0.smithi090.stdout:7/432: symlink db/d17/l93 0 2024-04-24T08:24:57.115 INFO:tasks.workunit.client.1.smithi149.stdout:4/647: write d2/d10/d91/f9d [7141978,122252] 0 2024-04-24T08:24:57.116 INFO:tasks.workunit.client.1.smithi149.stdout:2/641: mknod d0/d6/db/d27/d81/d9f/dba/cbb 0 2024-04-24T08:24:57.117 INFO:tasks.workunit.client.0.smithi090.stdout:5/452: symlink d8/d59/l9b 0 2024-04-24T08:24:57.118 INFO:tasks.workunit.client.1.smithi149.stdout:2/642: dread - d0/d6/db/d39/d3d/fb5 zero size 2024-04-24T08:24:57.119 INFO:tasks.workunit.client.0.smithi090.stdout:2/401: readlink d0/d7/d70/d31/d46/l4f 0 2024-04-24T08:24:57.122 INFO:tasks.workunit.client.1.smithi149.stdout:5/804: dwrite db/d35/f9d [0,4194304] 0 2024-04-24T08:24:57.123 INFO:tasks.workunit.client.0.smithi090.stdout:0/432: read - d1a/d4b/d53/d79/d69/f6d zero size 2024-04-24T08:24:57.124 INFO:tasks.workunit.client.1.smithi149.stdout:5/805: write db/da4/ff1 [881197,73862] 0 2024-04-24T08:24:57.125 INFO:tasks.workunit.client.0.smithi090.stdout:5/453: rmdir d8/da/d3a/d2d/d37/d93/d7a 39 2024-04-24T08:24:57.126 INFO:tasks.workunit.client.0.smithi090.stdout:0/433: dread - d1a/d4b/d53/d79/d3c/f84 zero size 2024-04-24T08:24:57.126 INFO:tasks.workunit.client.0.smithi090.stdout:4/311: dwrite d3/d24/f2b [0,4194304] 0 2024-04-24T08:24:57.129 INFO:tasks.workunit.client.1.smithi149.stdout:1/820: unlink d0/d5/db/d18/d2f/fd1 0 2024-04-24T08:24:57.129 INFO:tasks.workunit.client.0.smithi090.stdout:5/454: read d8/da/d14/d2f/d44/f70 [513817,43405] 0 2024-04-24T08:24:57.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/434: chown d1a/d4b/d53/d79 26526163 1 2024-04-24T08:24:57.135 INFO:tasks.workunit.client.1.smithi149.stdout:8/615: read d1/f8e [1808772,39788] 0 2024-04-24T08:24:57.140 INFO:tasks.workunit.client.1.smithi149.stdout:6/627: symlink d0/d1a/d28/d31/d38/d54/d87/ld4 0 2024-04-24T08:24:57.144 INFO:tasks.workunit.client.0.smithi090.stdout:5/455: unlink d8/da/d3a/d2d/d37/d93/d7b/f8b 0 2024-04-24T08:24:57.148 INFO:tasks.workunit.client.1.smithi149.stdout:3/777: unlink d0/d26/d3c/db1/ce5 0 2024-04-24T08:24:57.154 INFO:tasks.workunit.client.1.smithi149.stdout:0/647: getdents d2/d9/df/d56/dca 0 2024-04-24T08:24:57.155 INFO:tasks.workunit.client.0.smithi090.stdout:2/402: getdents d0/d7/d70/d31 0 2024-04-24T08:24:57.155 INFO:tasks.workunit.client.0.smithi090.stdout:4/312: dread d3/d18/d5c/f3d [0,4194304] 0 2024-04-24T08:24:57.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/806: creat db/d12/d60/f10c x:0 0 0 2024-04-24T08:24:57.157 INFO:tasks.workunit.client.0.smithi090.stdout:4/313: read - d3/d18/d1d/f4e zero size 2024-04-24T08:24:57.159 INFO:tasks.workunit.client.0.smithi090.stdout:4/314: dread - d3/d18/f5b zero size 2024-04-24T08:24:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:5/807: chown db/d12/d1e/d65/f7a 25996 1 2024-04-24T08:24:57.160 INFO:tasks.workunit.client.1.smithi149.stdout:9/759: dwrite d1/de/fa8 [0,4194304] 0 2024-04-24T08:24:57.165 INFO:tasks.workunit.client.0.smithi090.stdout:7/433: dwrite db/d17/d29/f30 [0,4194304] 0 2024-04-24T08:24:57.165 INFO:tasks.workunit.client.0.smithi090.stdout:2/403: symlink d0/d7/l7e 0 2024-04-24T08:24:57.168 INFO:tasks.workunit.client.1.smithi149.stdout:3/778: dread d0/d5/d53/d34/d3a/d93/f105 [0,4194304] 0 2024-04-24T08:24:57.170 INFO:tasks.workunit.client.1.smithi149.stdout:8/616: fsync d1/d1f/d5e/fa0 0 2024-04-24T08:24:57.171 INFO:tasks.workunit.client.0.smithi090.stdout:4/315: creat d3/d24/f5d x:0 0 0 2024-04-24T08:24:57.173 INFO:tasks.workunit.client.0.smithi090.stdout:4/316: fdatasync d3/d18/d1d/f4e 0 2024-04-24T08:24:57.175 INFO:tasks.workunit.client.1.smithi149.stdout:7/704: symlink d9/dc/d11/d83/da6/d67/dba/led 0 2024-04-24T08:24:57.176 INFO:tasks.workunit.client.0.smithi090.stdout:5/456: dread d8/f31 [4194304,4194304] 0 2024-04-24T08:24:57.177 INFO:tasks.workunit.client.1.smithi149.stdout:4/648: link d2/d24/d2e/d3f/fa9 d2/d10/d91/d9f/fc7 0 2024-04-24T08:24:57.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/434: mknod db/d17/d39/c94 0 2024-04-24T08:24:57.178 INFO:tasks.workunit.client.1.smithi149.stdout:9/760: rename d1/d2/d15/f8c to d1/d2/d34/f110 0 2024-04-24T08:24:57.179 INFO:tasks.workunit.client.1.smithi149.stdout:3/779: stat d0/d5/da/d5a/d63/d6a/dd9/ffe 0 2024-04-24T08:24:57.181 INFO:tasks.workunit.client.1.smithi149.stdout:5/808: truncate db/d12/d2f/d74/dc6/fcd 280297 0 2024-04-24T08:24:57.182 INFO:tasks.workunit.client.0.smithi090.stdout:5/457: write d8/da/d3a/d2d/d37/d93/d7a/d50/f8e [13825,122994] 0 2024-04-24T08:24:57.185 INFO:tasks.workunit.client.0.smithi090.stdout:7/435: truncate db/f10 4240176 0 2024-04-24T08:24:57.185 INFO:tasks.workunit.client.0.smithi090.stdout:5/458: dread - d8/da/d18/f65 zero size 2024-04-24T08:24:57.187 INFO:tasks.workunit.client.0.smithi090.stdout:5/459: chown d8/d7c 34326105 1 2024-04-24T08:24:57.189 INFO:tasks.workunit.client.0.smithi090.stdout:2/404: dwrite d0/d7/d6f/f29 [0,4194304] 0 2024-04-24T08:24:57.192 INFO:tasks.workunit.client.1.smithi149.stdout:5/809: chown db/f20 457 1 2024-04-24T08:24:57.193 INFO:tasks.workunit.client.0.smithi090.stdout:4/317: dread d3/d18/d1d/f40 [0,4194304] 0 2024-04-24T08:24:57.194 INFO:tasks.workunit.client.1.smithi149.stdout:3/780: creat d0/d26/d3c/f10c x:0 0 0 2024-04-24T08:24:57.195 INFO:tasks.workunit.client.1.smithi149.stdout:9/761: symlink d1/d2/d15/d1c/d42/dae/dd5/l111 0 2024-04-24T08:24:57.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/436: rmdir db/d17/d29/d38/d3e 39 2024-04-24T08:24:57.201 INFO:tasks.workunit.client.1.smithi149.stdout:2/643: dwrite d0/d6/db/d39/d3d/f6b [0,4194304] 0 2024-04-24T08:24:57.202 INFO:tasks.workunit.client.0.smithi090.stdout:2/405: rename d0/d7/d70 to d0/d7/d67/d7f 0 2024-04-24T08:24:57.204 INFO:tasks.workunit.client.1.smithi149.stdout:1/821: dwrite d0/d1b/d5d/de5/d101/f109 [0,4194304] 0 2024-04-24T08:24:57.212 INFO:tasks.workunit.client.1.smithi149.stdout:5/810: write db/d12/d60/db2/d83/dc8/ffe [88772,86585] 0 2024-04-24T08:24:57.214 INFO:tasks.workunit.client.0.smithi090.stdout:2/406: mknod d0/d14/d4d/d50/c80 0 2024-04-24T08:24:57.216 INFO:tasks.workunit.client.0.smithi090.stdout:7/437: symlink db/d17/d29/d38/d3e/d46/l95 0 2024-04-24T08:24:57.222 INFO:tasks.workunit.client.1.smithi149.stdout:5/811: stat db/d12/d60/db2/d83/faf 0 2024-04-24T08:24:57.222 INFO:tasks.workunit.client.1.smithi149.stdout:7/705: dwrite d9/dc/d7b/f7d [0,4194304] 0 2024-04-24T08:24:57.223 INFO:tasks.workunit.client.1.smithi149.stdout:9/762: symlink d1/da/d13/d74/l112 0 2024-04-24T08:24:57.225 INFO:tasks.workunit.client.1.smithi149.stdout:3/781: readlink d0/d5/da/d19/d1b/d72/ldd 0 2024-04-24T08:24:57.229 INFO:tasks.workunit.client.0.smithi090.stdout:6/396: sync 2024-04-24T08:24:57.229 INFO:tasks.workunit.client.0.smithi090.stdout:1/394: sync 2024-04-24T08:24:57.229 INFO:tasks.workunit.client.0.smithi090.stdout:3/344: sync 2024-04-24T08:24:57.230 INFO:tasks.workunit.client.1.smithi149.stdout:6/628: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:24:57.230 INFO:tasks.workunit.client.0.smithi090.stdout:2/407: rename d0/d7/d57/d79/d7b to d0/d7/d67/d7f/d81 0 2024-04-24T08:24:57.231 INFO:tasks.workunit.client.1.smithi149.stdout:4/649: dwrite d2/f6c [0,4194304] 0 2024-04-24T08:24:57.234 INFO:tasks.workunit.client.1.smithi149.stdout:0/648: write d2/d3c/f61 [526864,17601] 0 2024-04-24T08:24:57.234 INFO:tasks.workunit.client.0.smithi090.stdout:7/438: mknod db/d17/d29/d32/c96 0 2024-04-24T08:24:57.236 INFO:tasks.workunit.client.0.smithi090.stdout:4/318: link d3/d18/c2d d3/c5e 0 2024-04-24T08:24:57.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/782: fdatasync d0/d5/d73/fad 0 2024-04-24T08:24:57.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/395: chown d3/d5/l77 6714832 1 2024-04-24T08:24:57.242 INFO:tasks.workunit.client.0.smithi090.stdout:3/345: mkdir d2/d15/d36/d6f 0 2024-04-24T08:24:57.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/460: dwrite d8/da/d14/d2f/f47 [0,4194304] 0 2024-04-24T08:24:57.244 INFO:tasks.workunit.client.0.smithi090.stdout:2/408: symlink d0/d7/d67/d7f/d31/l82 0 2024-04-24T08:24:57.245 INFO:tasks.workunit.client.1.smithi149.stdout:5/812: rmdir db/d12/d60/db2/d83 39 2024-04-24T08:24:57.249 INFO:tasks.workunit.client.1.smithi149.stdout:4/650: mknod d2/d24/da7/cc8 0 2024-04-24T08:24:57.252 INFO:tasks.workunit.client.1.smithi149.stdout:6/629: fdatasync d0/d1a/d28/d31/d38/fc6 0 2024-04-24T08:24:57.257 INFO:tasks.workunit.client.1.smithi149.stdout:6/630: stat d0/d1a/d28/d31/d33/l49 0 2024-04-24T08:24:57.258 INFO:tasks.workunit.client.0.smithi090.stdout:1/396: fdatasync d3/d1d/d34/f3b 0 2024-04-24T08:24:57.259 INFO:tasks.workunit.client.1.smithi149.stdout:9/763: rename d1/f59 to d1/d2/dc8/f113 0 2024-04-24T08:24:57.260 INFO:tasks.workunit.client.1.smithi149.stdout:8/617: dwrite d1/de/d18/f7e [0,4194304] 0 2024-04-24T08:24:57.260 INFO:tasks.workunit.client.1.smithi149.stdout:0/649: mknod d2/d9/cd5 0 2024-04-24T08:24:57.262 INFO:tasks.workunit.client.1.smithi149.stdout:0/650: readlink d2/d9/l14 0 2024-04-24T08:24:57.263 INFO:tasks.workunit.client.1.smithi149.stdout:1/822: dread d0/ddd/fe9 [0,4194304] 0 2024-04-24T08:24:57.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/409: symlink d0/d14/d36/d77/l83 0 2024-04-24T08:24:57.267 INFO:tasks.workunit.client.0.smithi090.stdout:1/397: symlink d3/d5/d3e/l8e 0 2024-04-24T08:24:57.269 INFO:tasks.workunit.client.0.smithi090.stdout:5/461: dwrite d8/d59/f86 [0,4194304] 0 2024-04-24T08:24:57.274 INFO:tasks.workunit.client.0.smithi090.stdout:3/346: link d2/df/d1a/d37/f47 d2/df/d1a/d37/d53/f70 0 2024-04-24T08:24:57.279 INFO:tasks.workunit.client.1.smithi149.stdout:4/651: mknod d2/d10/cc9 0 2024-04-24T08:24:57.280 INFO:tasks.workunit.client.0.smithi090.stdout:3/347: truncate d2/df/f6e 66531 0 2024-04-24T08:24:57.288 INFO:tasks.workunit.client.1.smithi149.stdout:6/631: symlink d0/d1a/d28/d31/d4c/d80/d6f/d97/ld5 0 2024-04-24T08:24:57.290 INFO:tasks.workunit.client.1.smithi149.stdout:0/651: dread - d2/d9/df/d48/f52 zero size 2024-04-24T08:24:57.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/823: mkdir d0/d1b/d10d 0 2024-04-24T08:24:57.300 INFO:tasks.workunit.client.1.smithi149.stdout:7/706: rename d9/dc/d11/d84/ld7 to d9/dc/d11/d83/da6/d67/lee 0 2024-04-24T08:24:57.307 INFO:tasks.workunit.client.1.smithi149.stdout:8/618: creat d1/de/d18/fd2 x:0 0 0 2024-04-24T08:24:57.308 INFO:tasks.workunit.client.1.smithi149.stdout:5/813: getdents db/d12/d22/d45 0 2024-04-24T08:24:57.309 INFO:tasks.workunit.client.0.smithi090.stdout:7/439: rename db/d17/f2d to db/d17/d39/f97 0 2024-04-24T08:24:57.312 INFO:tasks.workunit.client.0.smithi090.stdout:7/440: truncate db/d90/f91 720359 0 2024-04-24T08:24:57.316 INFO:tasks.workunit.client.1.smithi149.stdout:9/764: dread d1/d4c/d91/fce [0,4194304] 0 2024-04-24T08:24:57.316 INFO:tasks.workunit.client.1.smithi149.stdout:0/652: mkdir d2/d9/df/d2f/dd6 0 2024-04-24T08:24:57.318 INFO:tasks.workunit.client.1.smithi149.stdout:2/644: write d0/d6/db/d39/d3d/d44/f61 [587362,125355] 0 2024-04-24T08:24:57.319 INFO:tasks.workunit.client.1.smithi149.stdout:7/707: fsync d9/dc/d72/fa1 0 2024-04-24T08:24:57.320 INFO:tasks.workunit.client.1.smithi149.stdout:0/653: chown d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6 1 1 2024-04-24T08:24:57.323 INFO:tasks.workunit.client.1.smithi149.stdout:8/619: truncate d1/d1f/f2c 1037388 0 2024-04-24T08:24:57.324 INFO:tasks.workunit.client.1.smithi149.stdout:2/645: chown d0/d6/db/f4a 288062350 1 2024-04-24T08:24:57.325 INFO:tasks.workunit.client.1.smithi149.stdout:5/814: truncate db/d12/d2f/d78/fd4 638962 0 2024-04-24T08:24:57.325 INFO:tasks.workunit.client.1.smithi149.stdout:0/654: fdatasync d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fad 0 2024-04-24T08:24:57.326 INFO:tasks.workunit.client.1.smithi149.stdout:3/783: write d0/d5/d53/d34/d3a/d93/fb5 [171556,87105] 0 2024-04-24T08:24:57.328 INFO:tasks.workunit.client.1.smithi149.stdout:6/632: dwrite d0/d1a/d28/d31/d4c/d4e/f64 [4194304,4194304] 0 2024-04-24T08:24:57.330 INFO:tasks.workunit.client.0.smithi090.stdout:1/398: rename d3/d5/da/dc/f57 to d3/d68/f8f 0 2024-04-24T08:24:57.331 INFO:tasks.workunit.client.1.smithi149.stdout:3/784: fsync d0/d88/fbe 0 2024-04-24T08:24:57.332 INFO:tasks.workunit.client.1.smithi149.stdout:6/633: stat d0/c51 0 2024-04-24T08:24:57.334 INFO:tasks.workunit.client.1.smithi149.stdout:2/646: truncate d0/d6/db/d27/fab 1090736 0 2024-04-24T08:24:57.337 INFO:tasks.workunit.client.0.smithi090.stdout:3/348: getdents d2/d25/d2c 0 2024-04-24T08:24:57.339 INFO:tasks.workunit.client.0.smithi090.stdout:2/410: dwrite d0/d7/f64 [0,4194304] 0 2024-04-24T08:24:57.343 INFO:tasks.workunit.client.0.smithi090.stdout:2/411: read d0/d7/fd [3697638,60183] 0 2024-04-24T08:24:57.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/399: creat d3/d5/da/dc/d80/f90 x:0 0 0 2024-04-24T08:24:57.344 INFO:tasks.workunit.client.0.smithi090.stdout:1/400: stat d3/d1d/d34 0 2024-04-24T08:24:57.346 INFO:tasks.workunit.client.0.smithi090.stdout:1/401: stat d3/d25/d30/l3d 0 2024-04-24T08:24:57.346 INFO:tasks.workunit.client.0.smithi090.stdout:2/412: truncate d0/d14/f34 1147247 0 2024-04-24T08:24:57.349 INFO:tasks.workunit.client.0.smithi090.stdout:2/413: write d0/d7/f33 [3383776,2487] 0 2024-04-24T08:24:57.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/765: mknod d1/da/dd/d55/c114 0 2024-04-24T08:24:57.351 INFO:tasks.workunit.client.0.smithi090.stdout:2/414: write d0/d7/d67/d7f/f5d [972722,80787] 0 2024-04-24T08:24:57.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.353+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30901a7030 con 0x7f3090072750 2024-04-24T08:24:57.354 INFO:tasks.workunit.client.0.smithi090.stdout:5/462: truncate f7 2526284 0 2024-04-24T08:24:57.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.353+0000 7f3086a00700 1 -- 172.21.15.90:0/1707392271 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f308005c490 con 0x7f3090072750 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:24:57.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:24:57.356 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:24:57.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 msgr2=0x7f3074070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 0x7f3074070f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3090072b80 tx=0x7f307c009190 comp rx=0 tx=0).stop 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 msgr2=0x7f30901a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3080000c00 tx=0x7f308000bd70 comp rx=0 tx=0).stop 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 shutdown_connections 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f307406ea70 0x7f3074070f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3090072750 0x7f30901a07c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 --2- 172.21.15.90:0/1707392271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30901a0d00 0x7f30901a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 >> 172.21.15.90:0/1707392271 conn(0x7f309006d110 msgr2=0x7f309010d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 shutdown_connections 2024-04-24T08:24:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.357+0000 7f3095000700 1 -- 172.21.15.90:0/1707392271 wait complete. 2024-04-24T08:24:57.359 INFO:tasks.workunit.client.0.smithi090.stdout:1/402: fsync d3/d5/da/f82 0 2024-04-24T08:24:57.360 INFO:tasks.workunit.client.1.smithi149.stdout:5/815: rename db/d35/f7c to db/d12/d22/d45/d5d/de6/f10d 0 2024-04-24T08:24:57.361 INFO:tasks.workunit.client.1.smithi149.stdout:0/655: fsync d2/d9/df/d2f/d6c/dc1/d4e/d6f/f8a 0 2024-04-24T08:24:57.363 INFO:tasks.workunit.client.1.smithi149.stdout:6/634: readlink d0/d1a/d28/d31/d38/d54/d87/lb0 0 2024-04-24T08:24:57.365 INFO:tasks.workunit.client.0.smithi090.stdout:2/415: dwrite d0/d2e/f6a [0,4194304] 0 2024-04-24T08:24:57.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/441: write db/d54/d83/f8f [1542972,123708] 0 2024-04-24T08:24:57.370 INFO:tasks.workunit.client.1.smithi149.stdout:8/620: mkdir d1/de/d18/dd3 0 2024-04-24T08:24:57.375 INFO:tasks.workunit.client.1.smithi149.stdout:1/824: dwrite d0/ddf/f7c [0,4194304] 0 2024-04-24T08:24:57.376 INFO:tasks.workunit.client.0.smithi090.stdout:1/403: dwrite d3/d1d/d34/d5a/f5b [0,4194304] 0 2024-04-24T08:24:57.377 INFO:tasks.workunit.client.0.smithi090.stdout:9/363: sync 2024-04-24T08:24:57.380 INFO:tasks.workunit.client.0.smithi090.stdout:8/336: sync 2024-04-24T08:24:57.383 INFO:tasks.workunit.client.0.smithi090.stdout:9/364: write d0/d19/f1e [1337548,124627] 0 2024-04-24T08:24:57.385 INFO:tasks.workunit.client.0.smithi090.stdout:5/463: mkdir d8/d9c 0 2024-04-24T08:24:57.386 INFO:tasks.workunit.client.1.smithi149.stdout:5/816: dwrite db/d12/d60/db2/de2/ff3 [0,4194304] 0 2024-04-24T08:24:57.391 INFO:tasks.workunit.client.0.smithi090.stdout:2/416: link d0/d2e/f6a d0/d2e/f84 0 2024-04-24T08:24:57.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/442: dwrite f9 [0,4194304] 0 2024-04-24T08:24:57.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/337: mkdir d2/da/d13/d60/d76 0 2024-04-24T08:24:57.401 INFO:tasks.workunit.client.0.smithi090.stdout:8/338: read - d2/da/d13/d64/d24/f5b zero size 2024-04-24T08:24:57.406 INFO:tasks.workunit.client.0.smithi090.stdout:8/339: read - d2/da/d13/d64/d24/f46 zero size 2024-04-24T08:24:57.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/417: rename d0/ff to d0/d5e/f85 0 2024-04-24T08:24:57.410 INFO:tasks.workunit.client.0.smithi090.stdout:9/365: dread d0/dc/d24/d2d/f3f [0,4194304] 0 2024-04-24T08:24:57.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/464: creat d8/d99/f9d x:0 0 0 2024-04-24T08:24:57.415 INFO:tasks.workunit.client.0.smithi090.stdout:8/340: readlink d2/da/d13/l25 0 2024-04-24T08:24:57.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/443: chown db/d17/d29/d32/l82 675 1 2024-04-24T08:24:57.420 INFO:tasks.workunit.client.0.smithi090.stdout:9/366: symlink d0/d19/d33/d4d/l66 0 2024-04-24T08:24:57.421 INFO:tasks.workunit.client.0.smithi090.stdout:9/367: chown d0/f5 234819 1 2024-04-24T08:24:57.421 INFO:tasks.workunit.client.0.smithi090.stdout:2/418: mkdir d0/d7/d67/d7f/d31/d46/d60/d65/d86 0 2024-04-24T08:24:57.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/368: write d0/fb [1787228,48410] 0 2024-04-24T08:24:57.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/465: fsync d8/da/d14/f69 0 2024-04-24T08:24:57.427 INFO:tasks.workunit.client.0.smithi090.stdout:1/404: write d3/d5/da/dc/f35 [1765010,90532] 0 2024-04-24T08:24:57.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/466: fdatasync d8/d99/f9d 0 2024-04-24T08:24:57.429 INFO:tasks.workunit.client.0.smithi090.stdout:1/405: fdatasync d3/d5/f16 0 2024-04-24T08:24:57.431 INFO:tasks.workunit.client.0.smithi090.stdout:8/341: link d2/da/d13/d64/c5a d2/d62/d6b/c77 0 2024-04-24T08:24:57.434 INFO:tasks.workunit.client.0.smithi090.stdout:1/406: symlink d3/d1d/d34/d5a/l91 0 2024-04-24T08:24:57.436 INFO:tasks.workunit.client.0.smithi090.stdout:5/467: creat d8/f9e x:0 0 0 2024-04-24T08:24:57.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/444: dwrite db/f12 [0,4194304] 0 2024-04-24T08:24:57.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/419: dread d0/d2e/f6a [0,4194304] 0 2024-04-24T08:24:57.442 INFO:tasks.workunit.client.0.smithi090.stdout:8/342: mknod d2/da/c78 0 2024-04-24T08:24:57.447 INFO:tasks.workunit.client.0.smithi090.stdout:1/407: creat d3/d5/da/d84/f92 x:0 0 0 2024-04-24T08:24:57.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/445: symlink db/d54/d83/l98 0 2024-04-24T08:24:57.457 INFO:tasks.workunit.client.0.smithi090.stdout:5/468: mkdir d8/da/d3a/d2d/d37/d93/d7a/d50/d9f 0 2024-04-24T08:24:57.457 INFO:tasks.workunit.client.0.smithi090.stdout:1/408: fsync d3/d5/da/dc/f81 0 2024-04-24T08:24:57.458 INFO:tasks.workunit.client.0.smithi090.stdout:7/446: chown db/d17/f2e 63455782 1 2024-04-24T08:24:57.458 INFO:tasks.workunit.client.0.smithi090.stdout:8/343: fdatasync d2/da/d13/f3a 0 2024-04-24T08:24:57.460 INFO:tasks.workunit.client.0.smithi090.stdout:5/469: dread - d8/d25/f72 zero size 2024-04-24T08:24:57.462 INFO:tasks.workunit.client.0.smithi090.stdout:1/409: fsync d3/d5/da/f6a 0 2024-04-24T08:24:57.463 INFO:tasks.workunit.client.1.smithi149.stdout:9/766: mkdir d1/da/dd/d79/db6/d115 0 2024-04-24T08:24:57.466 INFO:tasks.workunit.client.0.smithi090.stdout:7/447: truncate db/d17/d29/d32/d5a/d75/f7f 234087 0 2024-04-24T08:24:57.467 INFO:tasks.workunit.client.1.smithi149.stdout:2/647: mknod d0/d6/db/d39/d3d/cbc 0 2024-04-24T08:24:57.467 INFO:tasks.workunit.client.0.smithi090.stdout:2/420: dwrite d0/d2e/f6a [0,4194304] 0 2024-04-24T08:24:57.468 INFO:tasks.workunit.client.0.smithi090.stdout:1/410: write d3/d68/f8f [4465433,25716] 0 2024-04-24T08:24:57.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/708: creat d9/dc/d11/fef x:0 0 0 2024-04-24T08:24:57.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/470: truncate d8/d56/f5f 742158 0 2024-04-24T08:24:57.474 INFO:tasks.workunit.client.1.smithi149.stdout:6/635: mknod d0/d1a/d28/d31/d33/cd6 0 2024-04-24T08:24:57.476 INFO:tasks.workunit.client.0.smithi090.stdout:2/421: truncate d0/f22 3865917 0 2024-04-24T08:24:57.477 INFO:tasks.workunit.client.1.smithi149.stdout:6/636: chown d0/f12 14530 1 2024-04-24T08:24:57.478 INFO:tasks.workunit.client.0.smithi090.stdout:2/422: truncate d0/d5e/f85 4570789 0 2024-04-24T08:24:57.478 INFO:tasks.workunit.client.0.smithi090.stdout:8/344: dwrite d2/da/d13/d64/f4e [0,4194304] 0 2024-04-24T08:24:57.479 INFO:tasks.workunit.client.1.smithi149.stdout:8/621: symlink d1/de/d18/d32/d65/d8b/ld4 0 2024-04-24T08:24:57.480 INFO:tasks.workunit.client.0.smithi090.stdout:2/423: chown d0/d2e/f84 30613828 1 2024-04-24T08:24:57.481 INFO:tasks.workunit.client.0.smithi090.stdout:1/411: unlink d3/d5/da/dc/f51 0 2024-04-24T08:24:57.485 INFO:tasks.workunit.client.0.smithi090.stdout:2/424: truncate d0/d7/d67/d7f/d31/f56 881009 0 2024-04-24T08:24:57.487 INFO:tasks.workunit.client.0.smithi090.stdout:2/425: chown d0/d7/d57/d79 12 1 2024-04-24T08:24:57.487 INFO:tasks.workunit.client.0.smithi090.stdout:7/448: mkdir db/d17/d29/d38/d99 0 2024-04-24T08:24:57.487 INFO:tasks.workunit.client.0.smithi090.stdout:5/471: dwrite d8/da/fc [0,4194304] 0 2024-04-24T08:24:57.490 INFO:tasks.workunit.client.0.smithi090.stdout:5/472: write d8/d59/f86 [1680209,112236] 0 2024-04-24T08:24:57.490 INFO:tasks.workunit.client.0.smithi090.stdout:2/426: write d0/d7/d57/f66 [1940697,61558] 0 2024-04-24T08:24:57.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/648: dwrite d0/d6/db/d27/d64/f74 [0,4194304] 0 2024-04-24T08:24:57.498 INFO:tasks.workunit.client.1.smithi149.stdout:9/767: fsync d1/da/d13/d67/de4/fed 0 2024-04-24T08:24:57.504 INFO:tasks.workunit.client.1.smithi149.stdout:7/709: mknod d9/dc/d11/d83/da6/cf0 0 2024-04-24T08:24:57.505 INFO:tasks.workunit.client.0.smithi090.stdout:8/345: write d2/da/d13/f18 [3947214,27499] 0 2024-04-24T08:24:57.507 INFO:tasks.workunit.client.1.smithi149.stdout:3/785: getdents d0 0 2024-04-24T08:24:57.508 INFO:tasks.workunit.client.0.smithi090.stdout:8/346: chown d2/l8 1222 1 2024-04-24T08:24:57.509 INFO:tasks.workunit.client.1.smithi149.stdout:8/622: dwrite d1/de/d18/d91/d9f/fc6 [0,4194304] 0 2024-04-24T08:24:57.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/412: rename d3/d5/d58/l88 to d3/d5/da/d84/l93 0 2024-04-24T08:24:57.510 INFO:tasks.workunit.client.1.smithi149.stdout:6/637: mkdir d0/d1a/d28/d31/d4c/d80/d6f/dd7 0 2024-04-24T08:24:57.526 INFO:tasks.workunit.client.0.smithi090.stdout:5/473: creat d8/da/d18/fa0 x:0 0 0 2024-04-24T08:24:57.529 INFO:tasks.workunit.client.0.smithi090.stdout:5/474: write d8/da/d3a/d2d/d37/f6a [1599336,103177] 0 2024-04-24T08:24:57.539 INFO:tasks.workunit.client.0.smithi090.stdout:1/413: rmdir d3/d68 39 2024-04-24T08:24:57.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/347: fdatasync d2/da/f35 0 2024-04-24T08:24:57.546 INFO:tasks.workunit.client.1.smithi149.stdout:9/768: link d1/da/dd/fac d1/da/dd/d79/db6/d115/f116 0 2024-04-24T08:24:57.549 INFO:tasks.workunit.client.1.smithi149.stdout:5/817: dwrite db/d12/d1e/f41 [0,4194304] 0 2024-04-24T08:24:57.550 INFO:tasks.workunit.client.0.smithi090.stdout:0/435: sync 2024-04-24T08:24:57.550 INFO:tasks.workunit.client.0.smithi090.stdout:5/475: mkdir d8/da/d3a/d2d/d37/d93/d7a/da1 0 2024-04-24T08:24:57.556 INFO:tasks.workunit.client.1.smithi149.stdout:2/649: creat d0/d6/db/fbd x:0 0 0 2024-04-24T08:24:57.558 INFO:tasks.workunit.client.0.smithi090.stdout:1/414: rename d3/d5/da/d84 to d3/d5/da/d94 0 2024-04-24T08:24:57.561 INFO:tasks.workunit.client.1.smithi149.stdout:5/818: rename db/d12/d22/l70 to db/d23/daa/l10e 0 2024-04-24T08:24:57.566 INFO:tasks.workunit.client.1.smithi149.stdout:8/623: write d1/de/d18/d32/d65/d8b/d7d/f96 [2004397,37318] 0 2024-04-24T08:24:57.567 INFO:tasks.workunit.client.1.smithi149.stdout:9/769: symlink d1/d2/d33/d5d/l117 0 2024-04-24T08:24:57.568 INFO:tasks.workunit.client.0.smithi090.stdout:2/427: link d0/d7/d67/d7f/d31/f41 d0/d7/d67/d7f/f87 0 2024-04-24T08:24:57.572 INFO:tasks.workunit.client.0.smithi090.stdout:2/428: dread d0/d7/d67/d7f/d31/f56 [0,4194304] 0 2024-04-24T08:24:57.574 INFO:tasks.workunit.client.0.smithi090.stdout:0/436: mkdir d1a/d4b/d53/d79/d35/d8e 0 2024-04-24T08:24:57.576 INFO:tasks.workunit.client.1.smithi149.stdout:6/638: dwrite d0/d1a/d28/d31/d4c/d80/f74 [0,4194304] 0 2024-04-24T08:24:57.577 INFO:tasks.workunit.client.0.smithi090.stdout:6/397: sync 2024-04-24T08:24:57.579 INFO:tasks.workunit.client.1.smithi149.stdout:2/650: creat d0/d6/db/d28/d7f/d8f/fbe x:0 0 0 2024-04-24T08:24:57.579 INFO:tasks.workunit.client.0.smithi090.stdout:7/449: rename db/d17/d29/d38/d3e/d46/l95 to db/d17/l9a 0 2024-04-24T08:24:57.580 INFO:tasks.workunit.client.0.smithi090.stdout:6/398: write d8/d1d/d1f/f2e [569067,95177] 0 2024-04-24T08:24:57.581 INFO:tasks.workunit.client.1.smithi149.stdout:5/819: dwrite db/d12/d2f/d74/fa2 [0,4194304] 0 2024-04-24T08:24:57.584 INFO:tasks.workunit.client.0.smithi090.stdout:6/399: write d8/d1d/d1f/d3e/f6a [6800,69321] 0 2024-04-24T08:24:57.585 INFO:tasks.workunit.client.1.smithi149.stdout:8/624: mknod d1/de/d18/d32/d63/d9e/cd5 0 2024-04-24T08:24:57.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.586+0000 7f03d3a00700 1 -- 172.21.15.90:0/3012194885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc105710 msgr2=0x7f03cc107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:57.587 INFO:tasks.workunit.client.0.smithi090.stdout:0/437: dread d1a/d4b/f4d [0,4194304] 0 2024-04-24T08:24:57.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.586+0000 7f03d3a00700 1 --2- 172.21.15.90:0/3012194885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc105710 0x7f03cc107af0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009910 tx=0x7f03b8009c20 comp rx=0 tx=0).stop 2024-04-24T08:24:57.588 INFO:tasks.workunit.client.0.smithi090.stdout:6/400: truncate d8/f3f 460528 0 2024-04-24T08:24:57.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.587+0000 7f03d3a00700 1 -- 172.21.15.90:0/3012194885 shutdown_connections 2024-04-24T08:24:57.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.587+0000 7f03d3a00700 1 --2- 172.21.15.90:0/3012194885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc105710 0x7f03cc107af0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.587+0000 7f03d3a00700 1 --2- 172.21.15.90:0/3012194885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc069270 0x7f03cc1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.587+0000 7f03d3a00700 1 -- 172.21.15.90:0/3012194885 >> 172.21.15.90:0/3012194885 conn(0x7f03cc0fa780 msgr2=0x7f03cc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:57.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.590+0000 7f03d3a00700 1 -- 172.21.15.90:0/3012194885 shutdown_connections 2024-04-24T08:24:57.592 INFO:tasks.workunit.client.0.smithi090.stdout:5/476: unlink d8/d59/f86 0 2024-04-24T08:24:57.592 INFO:tasks.workunit.client.0.smithi090.stdout:2/429: mknod d0/d14/d4d/d50/c88 0 2024-04-24T08:24:57.593 INFO:tasks.workunit.client.1.smithi149.stdout:7/710: dwrite d9/dc/d11/d83/da6/d67/d69/fcf [0,4194304] 0 2024-04-24T08:24:57.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.590+0000 7f03d3a00700 1 -- 172.21.15.90:0/3012194885 wait complete. 2024-04-24T08:24:57.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.592+0000 7f03d3a00700 1 Processor -- start 2024-04-24T08:24:57.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.592+0000 7f03d3a00700 1 -- start start 2024-04-24T08:24:57.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.593+0000 7f03d3a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 0x7f03cc1938e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:57.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.593+0000 7f03d3a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 0x7f03cc196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:57.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.593+0000 7f03d3a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc194320 con 0x7f03cc193e20 2024-04-24T08:24:57.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.593+0000 7f03d3a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc194490 con 0x7f03cc105710 2024-04-24T08:24:57.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.594+0000 7f03d1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 0x7f03cc1938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:57.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.594+0000 7f03d1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 0x7f03cc1938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:58348/0 (socket says 172.21.15.90:58348) 2024-04-24T08:24:57.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.594+0000 7f03d1000700 1 -- 172.21.15.90:0/4089326342 learned_addr learned my addr 172.21.15.90:0/4089326342 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:24:57.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.595+0000 7f03cbe00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 0x7f03cc196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:57.596 INFO:tasks.workunit.client.1.smithi149.stdout:6/639: dread d0/fb [0,4194304] 0 2024-04-24T08:24:57.597 INFO:tasks.workunit.client.0.smithi090.stdout:2/430: chown d0/d14/d36/l42 489242 1 2024-04-24T08:24:57.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.596+0000 7f03cbe00700 1 -- 172.21.15.90:0/4089326342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 msgr2=0x7f03cc1938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:57.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.596+0000 7f03cbe00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 0x7f03cc1938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:57.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.597+0000 7f03cbe00700 1 -- 172.21.15.90:0/4089326342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80095f0 con 0x7f03cc193e20 2024-04-24T08:24:57.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.598+0000 7f03cbe00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 0x7f03cc196e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009910 tx=0x7f03b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.599+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b801d070 con 0x7f03cc193e20 2024-04-24T08:24:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.599+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03b8005dc0 con 0x7f03cc193e20 2024-04-24T08:24:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.599+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b8021410 con 0x7f03cc193e20 2024-04-24T08:24:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.599+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc1973c0 con 0x7f03cc193e20 2024-04-24T08:24:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.600+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc197880 con 0x7f03cc193e20 2024-04-24T08:24:57.603 INFO:tasks.workunit.client.1.smithi149.stdout:2/651: symlink d0/d6/db/d27/d81/d9f/lbf 0 2024-04-24T08:24:57.607 INFO:tasks.workunit.client.0.smithi090.stdout:1/415: mknod d3/d1d/d34/c95 0 2024-04-24T08:24:57.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/820: mkdir db/d12/d2f/d78/d10f 0 2024-04-24T08:24:57.612 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:57 smithi090 ceph-mon[25016]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:57.612 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:57 smithi090 ceph-mon[25016]: pgmap v21: 65 pgs: 65 active+clean; 2.0 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 199 MiB/s wr, 376 op/s 2024-04-24T08:24:57.612 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:57 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1707392271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:24:57.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.608+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc198c10 con 0x7f03cc193e20 2024-04-24T08:24:57.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.608+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f03b800f7d0 con 0x7f03cc193e20 2024-04-24T08:24:57.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.610+0000 7f03c9600700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 0x7f03b4070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:24:57.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.610+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f03b808fd60 con 0x7f03cc193e20 2024-04-24T08:24:57.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.613+0000 7f03d1000700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 0x7f03b4070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:24:57.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/450: fdatasync db/d17/d29/d32/d5a/d75/f7f 0 2024-04-24T08:24:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.615+0000 7f03d1000700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 0x7f03b4070f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005c50 tx=0x7f03c0007480 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:24:57.617 INFO:tasks.workunit.client.1.smithi149.stdout:8/625: dread d1/de/d18/d43/f88 [0,4194304] 0 2024-04-24T08:24:57.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/652: unlink d0/d6/db/d27/d81/d9f/lbf 0 2024-04-24T08:24:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:57.618+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f03b805c8e0 con 0x7f03cc193e20 2024-04-24T08:24:57.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/626: chown d1/de/d18/daf/lcc 1 1 2024-04-24T08:24:57.620 INFO:tasks.workunit.client.1.smithi149.stdout:5/821: rmdir db/d12 39 2024-04-24T08:24:57.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/348: rename d2/da/d13/d60/f75 to d2/da/d13/d64/d24/f79 0 2024-04-24T08:24:57.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/416: unlink d3/d5/da/f59 0 2024-04-24T08:24:57.626 INFO:tasks.workunit.client.1.smithi149.stdout:9/770: getdents d1/de/d9d/dfd 0 2024-04-24T08:24:57.627 INFO:tasks.workunit.client.1.smithi149.stdout:2/653: unlink d0/d6/db/d39/l62 0 2024-04-24T08:24:57.628 INFO:tasks.workunit.client.1.smithi149.stdout:9/771: chown d1/d2/d15/d1c/d42/f7f 740162 1 2024-04-24T08:24:57.629 INFO:tasks.workunit.client.1.smithi149.stdout:2/654: write d0/d6/db/d39/f4e [4234262,83913] 0 2024-04-24T08:24:57.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/627: creat d1/de/d18/d32/d63/d9e/fd6 x:0 0 0 2024-04-24T08:24:57.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/628: write d1/d1f/f8d [3279861,121105] 0 2024-04-24T08:24:57.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/401: dwrite d8/d1d/d1f/f2e [4194304,4194304] 0 2024-04-24T08:24:57.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/431: dwrite d0/d5e/f63 [0,4194304] 0 2024-04-24T08:24:57.636 INFO:tasks.workunit.client.0.smithi090.stdout:5/477: rename d8/da/d18/d8c to d8/da/d3a/d96/da2 0 2024-04-24T08:24:57.639 INFO:tasks.workunit.client.0.smithi090.stdout:1/417: symlink d3/l96 0 2024-04-24T08:24:57.641 INFO:tasks.workunit.client.0.smithi090.stdout:0/438: getdents d1a/d4b/d53/d79/d3c 0 2024-04-24T08:24:57.642 INFO:tasks.workunit.client.0.smithi090.stdout:0/439: stat d1a/d4b/f6f 0 2024-04-24T08:24:57.643 INFO:tasks.workunit.client.1.smithi149.stdout:7/711: link d9/dc/d2e/l30 d9/d43/d4f/da5/lf1 0 2024-04-24T08:24:57.646 INFO:tasks.workunit.client.0.smithi090.stdout:5/478: chown d8/da/d18/f1f 267257 1 2024-04-24T08:24:57.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/712: dread - d9/dc/d11/d83/fd9 zero size 2024-04-24T08:24:57.654 INFO:tasks.workunit.client.0.smithi090.stdout:5/479: write d8/da/d3a/d2d/d37/d93/d7b/f80 [226103,96386] 0 2024-04-24T08:24:57.654 INFO:tasks.workunit.client.1.smithi149.stdout:5/822: unlink db/d12/d1e/d65/de8/f100 0 2024-04-24T08:24:57.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/432: mknod d0/d7/d57/d79/c89 0 2024-04-24T08:24:57.656 INFO:tasks.workunit.client.1.smithi149.stdout:6/640: write d0/d1a/d28/f95 [422531,70510] 0 2024-04-24T08:24:57.657 INFO:tasks.workunit.client.1.smithi149.stdout:8/629: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:24:57.658 INFO:tasks.workunit.client.1.smithi149.stdout:9/772: dwrite d1/d2/d15/d1c/d42/dae/dd5/fde [0,4194304] 0 2024-04-24T08:24:57.660 INFO:tasks.workunit.client.0.smithi090.stdout:6/402: symlink d8/d1d/d1f/l79 0 2024-04-24T08:24:57.660 INFO:tasks.workunit.client.1.smithi149.stdout:8/630: chown d1/d1f/d25/c4e 1646457 1 2024-04-24T08:24:57.662 INFO:tasks.workunit.client.1.smithi149.stdout:8/631: fdatasync d1/de/d18/d43/fb6 0 2024-04-24T08:24:57.664 INFO:tasks.workunit.client.0.smithi090.stdout:6/403: write d8/d1d/d1f/d3e/f5c [377993,39675] 0 2024-04-24T08:24:57.664 INFO:tasks.workunit.client.0.smithi090.stdout:8/349: rename d2/da/d13/f18 to d2/d62/f7a 0 2024-04-24T08:24:57.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/418: dread d3/d1d/f55 [0,4194304] 0 2024-04-24T08:24:57.667 INFO:tasks.workunit.client.0.smithi090.stdout:8/350: write d2/da/d13/d64/f4e [5111972,1578] 0 2024-04-24T08:24:57.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/419: dread - d3/d1d/f86 zero size 2024-04-24T08:24:57.669 INFO:tasks.workunit.client.1.smithi149.stdout:7/713: fdatasync d9/dc/de/f31 0 2024-04-24T08:24:57.669 INFO:tasks.workunit.client.0.smithi090.stdout:2/433: readlink d0/lb 0 2024-04-24T08:24:57.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/632: chown d1/d1f/d25/c4e 371408 1 2024-04-24T08:24:57.677 INFO:tasks.workunit.client.1.smithi149.stdout:5/823: write db/d12/d1e/d85/d92/d107/d7f/fd0 [3640969,128966] 0 2024-04-24T08:24:57.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/714: readlink d9/dc/d11/d83/da6/l99 0 2024-04-24T08:24:57.682 INFO:tasks.workunit.client.1.smithi149.stdout:6/641: unlink d0/d1a/d28/d31/d4c/d4e/laa 0 2024-04-24T08:24:57.682 INFO:tasks.workunit.client.1.smithi149.stdout:7/715: dread - d9/dc/d11/d84/da7/fd3 zero size 2024-04-24T08:24:57.684 INFO:tasks.workunit.client.1.smithi149.stdout:8/633: rename d1/de/d18/f4c to d1/de/d18/d32/d63/d9e/fd7 0 2024-04-24T08:24:57.685 INFO:tasks.workunit.client.1.smithi149.stdout:8/634: dread - d1/de/d18/fd2 zero size 2024-04-24T08:24:57.685 INFO:tasks.workunit.client.1.smithi149.stdout:5/824: mkdir db/d12/d1e/d85/d92/d107/d110 0 2024-04-24T08:24:57.686 INFO:tasks.workunit.client.0.smithi090.stdout:3/349: sync 2024-04-24T08:24:57.686 INFO:tasks.workunit.client.0.smithi090.stdout:4/319: sync 2024-04-24T08:24:57.686 INFO:tasks.workunit.client.0.smithi090.stdout:9/369: sync 2024-04-24T08:24:57.688 INFO:tasks.workunit.client.1.smithi149.stdout:5/825: fsync db/d12/fb0 0 2024-04-24T08:24:57.693 INFO:tasks.workunit.client.0.smithi090.stdout:5/480: write d8/da/d3a/d2d/d37/d93/d7a/d50/f79 [464080,60499] 0 2024-04-24T08:24:57.696 INFO:tasks.workunit.client.1.smithi149.stdout:6/642: rename d0/d1a/d28/d31/d38/d54/d87 to d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8 0 2024-04-24T08:24:57.697 INFO:tasks.workunit.client.0.smithi090.stdout:6/404: mkdir d8/d1d/d1f/d3e/d7a 0 2024-04-24T08:24:57.698 INFO:tasks.workunit.client.0.smithi090.stdout:3/350: dread - d2/df/d1a/d37/f55 zero size 2024-04-24T08:24:57.699 INFO:tasks.workunit.client.0.smithi090.stdout:7/451: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:57.699 INFO:tasks.workunit.client.1.smithi149.stdout:6/643: read d0/d1a/f69 [660122,73747] 0 2024-04-24T08:24:57.706 INFO:tasks.workunit.client.1.smithi149.stdout:8/635: rename d1/de/d18/d32/d65/d8b/dc9/lcb to d1/de/d18/daf/ld8 0 2024-04-24T08:24:57.706 INFO:tasks.workunit.client.0.smithi090.stdout:4/320: unlink d3/d18/d1d/f2a 0 2024-04-24T08:24:57.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/351: dwrite d2/f12 [0,4194304] 0 2024-04-24T08:24:57.708 INFO:tasks.workunit.client.0.smithi090.stdout:2/434: mknod d0/d7/d67/d7f/d31/d46/c8a 0 2024-04-24T08:24:57.709 INFO:tasks.workunit.client.0.smithi090.stdout:3/351: truncate d2/df/d1a/d37/f47 118902 0 2024-04-24T08:24:57.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/370: mknod d0/d19/d33/d4d/c67 0 2024-04-24T08:24:57.713 INFO:tasks.workunit.client.0.smithi090.stdout:3/352: truncate d2/df/f6e 231505 0 2024-04-24T08:24:57.715 INFO:tasks.workunit.client.1.smithi149.stdout:6/644: rmdir d0/d1a 39 2024-04-24T08:24:57.719 INFO:tasks.workunit.client.0.smithi090.stdout:6/405: creat d8/d1d/d2f/d30/f7b x:0 0 0 2024-04-24T08:24:57.721 INFO:tasks.workunit.client.1.smithi149.stdout:5/826: creat db/d12/d1e/d65/f111 x:0 0 0 2024-04-24T08:24:57.724 INFO:tasks.workunit.client.1.smithi149.stdout:9/773: write d1/d2/d33/d5d/d83/dbf/dc4/ff5 [1876129,97764] 0 2024-04-24T08:24:57.724 INFO:tasks.workunit.client.0.smithi090.stdout:5/481: dwrite d8/da/d3a/d2d/d37/f6a [0,4194304] 0 2024-04-24T08:24:57.729 INFO:tasks.workunit.client.0.smithi090.stdout:4/321: write d3/d25/f2f [523693,45949] 0 2024-04-24T08:24:57.729 INFO:tasks.workunit.client.1.smithi149.stdout:8/636: mknod d1/de/d18/d32/d65/cd9 0 2024-04-24T08:24:57.731 INFO:tasks.workunit.client.1.smithi149.stdout:8/637: chown d1/d1f/d22 55499 1 2024-04-24T08:24:57.732 INFO:tasks.workunit.client.1.smithi149.stdout:6/645: fdatasync d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 0 2024-04-24T08:24:57.733 INFO:tasks.workunit.client.1.smithi149.stdout:5/827: symlink db/d12/d22/d45/d5d/de6/l112 0 2024-04-24T08:24:57.735 INFO:tasks.workunit.client.1.smithi149.stdout:6/646: fsync d0/fa 0 2024-04-24T08:24:57.735 INFO:tasks.workunit.client.0.smithi090.stdout:7/452: read db/d17/d29/d38/f44 [565606,60164] 0 2024-04-24T08:24:57.740 INFO:tasks.workunit.client.1.smithi149.stdout:9/774: rename d1/d99/dfb to d1/d2/d15/d1c/dca/d118 0 2024-04-24T08:24:57.742 INFO:tasks.workunit.client.1.smithi149.stdout:9/775: dread - d1/de/f10f zero size 2024-04-24T08:24:57.743 INFO:tasks.workunit.client.0.smithi090.stdout:2/435: mknod d0/d7/d67/d7f/d31/c8b 0 2024-04-24T08:24:57.745 INFO:tasks.workunit.client.1.smithi149.stdout:7/716: write d9/d43/d4f/d66/f6e [2462192,27827] 0 2024-04-24T08:24:57.745 INFO:tasks.workunit.client.0.smithi090.stdout:3/353: symlink d2/df/d21/l71 0 2024-04-24T08:24:57.748 INFO:tasks.workunit.client.0.smithi090.stdout:9/371: creat d0/dc/d24/d2d/d49/d62/f68 x:0 0 0 2024-04-24T08:24:57.749 INFO:tasks.workunit.client.0.smithi090.stdout:4/322: chown d3/d18/d1d/d22/c28 47506191 1 2024-04-24T08:24:57.752 INFO:tasks.workunit.client.0.smithi090.stdout:4/323: chown d3/d18/d1d/f4e 1069208562 1 2024-04-24T08:24:57.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/420: dread d3/d68/f8f [0,4194304] 0 2024-04-24T08:24:57.754 INFO:tasks.workunit.client.1.smithi149.stdout:8/638: fdatasync d1/de/d18/d32/fb3 0 2024-04-24T08:24:57.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/406: dread d8/f16 [4194304,4194304] 0 2024-04-24T08:24:57.755 INFO:tasks.workunit.client.0.smithi090.stdout:7/453: creat db/d17/d29/d38/d99/f9b x:0 0 0 2024-04-24T08:24:57.756 INFO:tasks.workunit.client.1.smithi149.stdout:5/828: dwrite db/d12/d1e/d85/d92/da6/fce [0,4194304] 0 2024-04-24T08:24:57.757 INFO:tasks.workunit.client.1.smithi149.stdout:7/717: mknod d9/cf2 0 2024-04-24T08:24:57.761 INFO:tasks.workunit.client.1.smithi149.stdout:7/718: dread - d9/dc/d11/d83/da6/d67/d69/dad/fb9 zero size 2024-04-24T08:24:57.761 INFO:tasks.workunit.client.1.smithi149.stdout:8/639: write d1/d1f/f68 [1305501,6306] 0 2024-04-24T08:24:57.763 INFO:tasks.workunit.client.0.smithi090.stdout:2/436: symlink d0/l8c 0 2024-04-24T08:24:57.765 INFO:tasks.workunit.client.0.smithi090.stdout:6/407: dread d8/d1d/d1f/f2e [4194304,4194304] 0 2024-04-24T08:24:57.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/408: stat d8 0 2024-04-24T08:24:57.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/372: dwrite d0/d19/f56 [0,4194304] 0 2024-04-24T08:24:57.771 INFO:tasks.workunit.client.0.smithi090.stdout:4/324: symlink d3/d24/l5f 0 2024-04-24T08:24:57.773 INFO:tasks.workunit.client.0.smithi090.stdout:4/325: write d3/d24/f2b [3649653,79202] 0 2024-04-24T08:24:57.775 INFO:tasks.workunit.client.0.smithi090.stdout:4/326: truncate d3/d18/f5b 614973 0 2024-04-24T08:24:57.777 INFO:tasks.workunit.client.1.smithi149.stdout:9/776: dread d1/de/ff [0,4194304] 0 2024-04-24T08:24:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:57 smithi149 ceph-mon[31282]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:57 smithi149 ceph-mon[31282]: pgmap v21: 65 pgs: 65 active+clean; 2.0 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 199 MiB/s wr, 376 op/s 2024-04-24T08:24:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:57 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1707392271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:24:57.780 INFO:tasks.workunit.client.1.smithi149.stdout:9/777: dread - d1/da/dd/d55/f105 zero size 2024-04-24T08:24:57.780 INFO:tasks.workunit.client.1.smithi149.stdout:5/829: rename db/d12/d22/d45/d5d/de6 to db/d12/d2f/d74/dc6/d113 0 2024-04-24T08:24:57.782 INFO:tasks.workunit.client.0.smithi090.stdout:7/454: unlink db/d17/d29/d38/d3e/f3f 0 2024-04-24T08:24:57.786 INFO:tasks.workunit.client.1.smithi149.stdout:6/647: getdents d0/d1a/d28/d31/d4c/d76 0 2024-04-24T08:24:57.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/421: dwrite d3/f4 [0,4194304] 0 2024-04-24T08:24:57.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/352: getdents d2/d62 0 2024-04-24T08:24:57.788 INFO:tasks.workunit.client.1.smithi149.stdout:8/640: fdatasync d1/d1f/d25/f78 0 2024-04-24T08:24:57.790 INFO:tasks.workunit.client.0.smithi090.stdout:2/437: creat d0/d14/d4d/f8d x:0 0 0 2024-04-24T08:24:57.793 INFO:tasks.workunit.client.1.smithi149.stdout:8/641: truncate d1/de/d18/fd2 640164 0 2024-04-24T08:24:57.795 INFO:tasks.workunit.client.0.smithi090.stdout:3/354: truncate d2/f9 2358395 0 2024-04-24T08:24:57.801 INFO:tasks.workunit.client.1.smithi149.stdout:6/648: rmdir d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8 39 2024-04-24T08:24:57.803 INFO:tasks.workunit.client.0.smithi090.stdout:4/327: rename d3/d25/f2f to d3/d38/f60 0 2024-04-24T08:24:57.806 INFO:tasks.workunit.client.1.smithi149.stdout:5/830: link db/da4/fb5 db/d23/f114 0 2024-04-24T08:24:57.808 INFO:tasks.workunit.client.0.smithi090.stdout:6/409: dwrite d8/d1d/d2f/d30/d38/f49 [0,4194304] 0 2024-04-24T08:24:57.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/422: mkdir d3/d5/da/d97 0 2024-04-24T08:24:57.810 INFO:tasks.workunit.client.1.smithi149.stdout:9/778: creat d1/d2/d33/d60/d10b/f119 x:0 0 0 2024-04-24T08:24:57.812 INFO:tasks.workunit.client.0.smithi090.stdout:8/353: symlink d2/da/d13/d60/l7b 0 2024-04-24T08:24:57.815 INFO:tasks.workunit.client.0.smithi090.stdout:2/438: mknod d0/d14/d17/c8e 0 2024-04-24T08:24:57.818 INFO:tasks.workunit.client.0.smithi090.stdout:3/355: chown d2/df/d21/f32 259857426 1 2024-04-24T08:24:57.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/373: mkdir d0/d69 0 2024-04-24T08:24:57.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/779: mkdir d1/da/d13/d74/d11a 0 2024-04-24T08:24:57.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/642: write d1/d1f/f69 [639053,101666] 0 2024-04-24T08:24:57.825 INFO:tasks.workunit.client.1.smithi149.stdout:7/719: dwrite d9/dc/d11/d26/f28 [4194304,4194304] 0 2024-04-24T08:24:57.827 INFO:tasks.workunit.client.1.smithi149.stdout:1/825: sync 2024-04-24T08:24:57.828 INFO:tasks.workunit.client.1.smithi149.stdout:3/786: sync 2024-04-24T08:24:57.828 INFO:tasks.workunit.client.1.smithi149.stdout:0/656: sync 2024-04-24T08:24:57.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/423: unlink d3/d5/f14 0 2024-04-24T08:24:57.829 INFO:tasks.workunit.client.0.smithi090.stdout:8/354: symlink d2/da/d13/d64/d50/l7c 0 2024-04-24T08:24:57.830 INFO:tasks.workunit.client.0.smithi090.stdout:1/424: fsync d3/d5/da/dc/f87 0 2024-04-24T08:24:57.830 INFO:tasks.workunit.client.1.smithi149.stdout:6/649: symlink d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/ld9 0 2024-04-24T08:24:57.832 INFO:tasks.workunit.client.0.smithi090.stdout:2/439: symlink d0/d14/d4d/l8f 0 2024-04-24T08:24:57.833 INFO:tasks.workunit.client.1.smithi149.stdout:3/787: chown d0/l68 23 1 2024-04-24T08:24:57.834 INFO:tasks.workunit.client.1.smithi149.stdout:6/650: fdatasync d0/f10 0 2024-04-24T08:24:57.835 INFO:tasks.workunit.client.1.smithi149.stdout:3/788: readlink d0/d5/da/d5e/la6 0 2024-04-24T08:24:57.836 INFO:tasks.workunit.client.0.smithi090.stdout:3/356: creat d2/df/d57/d4f/f72 x:0 0 0 2024-04-24T08:24:57.842 INFO:tasks.workunit.client.0.smithi090.stdout:4/328: mknod d3/d18/c61 0 2024-04-24T08:24:57.847 INFO:tasks.workunit.client.1.smithi149.stdout:8/643: dread d1/d1f/d31/f67 [0,4194304] 0 2024-04-24T08:24:57.851 INFO:tasks.workunit.client.1.smithi149.stdout:5/831: rmdir db/d12/d2f/d78/d10f 0 2024-04-24T08:24:57.853 INFO:tasks.workunit.client.0.smithi090.stdout:9/374: mknod d0/d5e/c6a 0 2024-04-24T08:24:57.856 INFO:tasks.workunit.client.0.smithi090.stdout:2/440: mkdir d0/d7/d67/d7f/d31/d90 0 2024-04-24T08:24:57.858 INFO:tasks.workunit.client.0.smithi090.stdout:7/455: getdents db/d17/d29/d38/d3e 0 2024-04-24T08:24:57.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/652: sync 2024-04-24T08:24:57.860 INFO:tasks.workunit.client.1.smithi149.stdout:3/789: creat d0/d5/da/d19/d1b/d72/f10d x:0 0 0 2024-04-24T08:24:57.861 INFO:tasks.workunit.client.0.smithi090.stdout:4/329: dread d3/f9 [0,4194304] 0 2024-04-24T08:24:57.862 INFO:tasks.workunit.client.1.smithi149.stdout:9/780: symlink d1/da/d13/d74/d11a/l11b 0 2024-04-24T08:24:57.862 INFO:tasks.workunit.client.1.smithi149.stdout:3/790: stat d0/d5/da/d5e/f101 0 2024-04-24T08:24:57.864 INFO:tasks.workunit.client.0.smithi090.stdout:8/355: creat d2/d62/d6b/f7d x:0 0 0 2024-04-24T08:24:57.866 INFO:tasks.workunit.client.1.smithi149.stdout:9/781: chown d1/da/d13/d67/de4/fed 11 1 2024-04-24T08:24:57.867 INFO:tasks.workunit.client.0.smithi090.stdout:6/410: truncate d8/d1d/f6f 18495 0 2024-04-24T08:24:57.867 INFO:tasks.workunit.client.0.smithi090.stdout:8/356: chown d2/da/d13/d60/l7b 3573 1 2024-04-24T08:24:57.868 INFO:tasks.workunit.client.0.smithi090.stdout:9/375: dread d0/d19/d33/f36 [0,4194304] 0 2024-04-24T08:24:57.868 INFO:tasks.workunit.client.0.smithi090.stdout:2/441: creat d0/d7/d28/f91 x:0 0 0 2024-04-24T08:24:57.869 INFO:tasks.workunit.client.0.smithi090.stdout:1/425: write d3/d1d/d54/f72 [1699060,45092] 0 2024-04-24T08:24:57.869 INFO:tasks.workunit.client.0.smithi090.stdout:8/357: write d2/d62/f6e [738449,48015] 0 2024-04-24T08:24:57.869 INFO:tasks.workunit.client.1.smithi149.stdout:1/826: symlink d0/d1b/l10e 0 2024-04-24T08:24:57.869 INFO:tasks.workunit.client.1.smithi149.stdout:5/832: dwrite db/d12/f84 [4194304,4194304] 0 2024-04-24T08:24:57.874 INFO:tasks.workunit.client.0.smithi090.stdout:9/376: write d0/f53 [3019037,8052] 0 2024-04-24T08:24:57.875 INFO:tasks.workunit.client.1.smithi149.stdout:5/833: readlink db/d12/d60/d8f/lba 0 2024-04-24T08:24:57.875 INFO:tasks.workunit.client.1.smithi149.stdout:0/657: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/fd7 x:0 0 0 2024-04-24T08:24:57.875 INFO:tasks.workunit.client.1.smithi149.stdout:7/720: dread f1 [4194304,4194304] 0 2024-04-24T08:24:57.875 INFO:tasks.workunit.client.1.smithi149.stdout:6/651: creat d0/db7/fda x:0 0 0 2024-04-24T08:24:57.875 INFO:tasks.workunit.client.0.smithi090.stdout:8/358: write d2/d62/d6b/f7d [273216,82337] 0 2024-04-24T08:24:57.876 INFO:tasks.workunit.client.1.smithi149.stdout:0/658: chown d2/d3c/fc6 65188 1 2024-04-24T08:24:57.876 INFO:tasks.workunit.client.0.smithi090.stdout:4/330: fsync d3/d38/d49/f53 0 2024-04-24T08:24:57.878 INFO:tasks.workunit.client.1.smithi149.stdout:4/653: mknod d2/da8/cca 0 2024-04-24T08:24:57.878 INFO:tasks.workunit.client.1.smithi149.stdout:7/721: dread - d9/d43/d95/fd0 zero size 2024-04-24T08:24:57.879 INFO:tasks.workunit.client.1.smithi149.stdout:4/654: chown d2/d18/da4 18999777 1 2024-04-24T08:24:57.881 INFO:tasks.workunit.client.0.smithi090.stdout:7/456: dwrite db/d17/d29/d38/d3e/f58 [0,4194304] 0 2024-04-24T08:24:57.882 INFO:tasks.workunit.client.1.smithi149.stdout:3/791: mkdir d0/d5/da/d5a/d63/d6a/d10e 0 2024-04-24T08:24:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:9/782: mknod d1/d2/d33/d5d/c11c 0 2024-04-24T08:24:57.884 INFO:tasks.workunit.client.0.smithi090.stdout:8/359: dread - d2/da/d13/d64/d24/f46 zero size 2024-04-24T08:24:57.885 INFO:tasks.workunit.client.1.smithi149.stdout:8/644: mknod d1/de/d18/d32/cda 0 2024-04-24T08:24:57.890 INFO:tasks.workunit.client.1.smithi149.stdout:6/652: symlink d0/d1a/d28/d31/d4c/d80/d6f/ldb 0 2024-04-24T08:24:57.891 INFO:tasks.workunit.client.0.smithi090.stdout:9/377: mknod d0/c6b 0 2024-04-24T08:24:57.891 INFO:tasks.workunit.client.0.smithi090.stdout:9/378: readlink d0/l27 0 2024-04-24T08:24:57.892 INFO:tasks.workunit.client.0.smithi090.stdout:6/411: dwrite d8/d1d/d29/f6c [0,4194304] 0 2024-04-24T08:24:57.892 INFO:tasks.workunit.client.1.smithi149.stdout:4/655: creat d2/d10/d91/fcb x:0 0 0 2024-04-24T08:24:57.894 INFO:tasks.workunit.client.1.smithi149.stdout:3/792: mknod d0/d5/da/d5a/d63/d6a/dd9/c10f 0 2024-04-24T08:24:57.896 INFO:tasks.workunit.client.1.smithi149.stdout:3/793: chown d0/d5/d53/d34/d3a/d93/ff1 1677411268 1 2024-04-24T08:24:57.897 INFO:tasks.workunit.client.1.smithi149.stdout:9/783: mkdir d1/d2/d34/d11d 0 2024-04-24T08:24:57.900 INFO:tasks.workunit.client.1.smithi149.stdout:3/794: chown d0/d5/da/d19/d1b/d22/d38/ld7 141594188 1 2024-04-24T08:24:57.900 INFO:tasks.workunit.client.0.smithi090.stdout:6/412: truncate d8/f46 381839 0 2024-04-24T08:24:57.902 INFO:tasks.workunit.client.1.smithi149.stdout:1/827: creat d0/d5/d4f/dac/f10f x:0 0 0 2024-04-24T08:24:57.913 INFO:tasks.workunit.client.0.smithi090.stdout:8/360: creat d2/da/d13/f7e x:0 0 0 2024-04-24T08:24:57.923 INFO:tasks.workunit.client.0.smithi090.stdout:9/379: dread d0/f1b [0,4194304] 0 2024-04-24T08:24:57.927 INFO:tasks.workunit.client.1.smithi149.stdout:9/784: truncate d1/da/d13/d74/dbb/fc5 9086 0 2024-04-24T08:24:57.928 INFO:tasks.workunit.client.1.smithi149.stdout:6/653: read d0/d1a/d28/d31/d33/f90 [487210,32429] 0 2024-04-24T08:24:57.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/361: truncate d2/da/f35 1694660 0 2024-04-24T08:24:57.929 INFO:tasks.workunit.client.0.smithi090.stdout:2/442: write d0/d14/d17/f1b [3093585,115977] 0 2024-04-24T08:24:57.931 INFO:tasks.workunit.client.0.smithi090.stdout:8/362: write d2/f12 [2659474,36933] 0 2024-04-24T08:24:57.933 INFO:tasks.workunit.client.0.smithi090.stdout:7/457: link db/d17/d29/d32/l45 db/d17/d29/d38/l9c 0 2024-04-24T08:24:57.934 INFO:tasks.workunit.client.0.smithi090.stdout:7/458: stat db/c80 0 2024-04-24T08:24:57.934 INFO:tasks.workunit.client.0.smithi090.stdout:8/363: write d2/da/d13/d64/d50/f57 [953684,22424] 0 2024-04-24T08:24:57.935 INFO:tasks.workunit.client.0.smithi090.stdout:7/459: read - db/d17/d29/f55 zero size 2024-04-24T08:24:57.937 INFO:tasks.workunit.client.0.smithi090.stdout:9/380: mkdir d0/dc/d24/d2d/d6c 0 2024-04-24T08:24:57.941 INFO:tasks.workunit.client.1.smithi149.stdout:5/834: rename db/d23/f43 to db/d12/d60/db2/f115 0 2024-04-24T08:24:57.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/659: creat d2/d9/fd8 x:0 0 0 2024-04-24T08:24:57.946 INFO:tasks.workunit.client.1.smithi149.stdout:7/722: write d9/dc/d72/fa1 [845123,83399] 0 2024-04-24T08:24:57.947 INFO:tasks.workunit.client.0.smithi090.stdout:4/331: truncate d3/f32 3078530 0 2024-04-24T08:24:57.948 INFO:tasks.workunit.client.0.smithi090.stdout:4/332: write d3/d18/d5c/f4a [1715617,54140] 0 2024-04-24T08:24:57.950 INFO:tasks.workunit.client.0.smithi090.stdout:7/460: symlink db/d54/d83/l9d 0 2024-04-24T08:24:57.952 INFO:tasks.workunit.client.1.smithi149.stdout:1/828: dread d0/ddf/d53/d61/fc7 [0,4194304] 0 2024-04-24T08:24:57.955 INFO:tasks.workunit.client.0.smithi090.stdout:2/443: mknod d0/d7/d67/d7f/d31/d46/c92 0 2024-04-24T08:24:57.956 INFO:tasks.workunit.client.0.smithi090.stdout:8/364: mkdir d2/da/d13/d60/d76/d7f 0 2024-04-24T08:24:57.956 INFO:tasks.workunit.client.0.smithi090.stdout:2/444: chown d0/f21 906808699 1 2024-04-24T08:24:57.958 INFO:tasks.workunit.client.1.smithi149.stdout:0/660: mknod d2/d9/d1a/d36/cd9 0 2024-04-24T08:24:57.959 INFO:tasks.workunit.client.0.smithi090.stdout:6/413: write d8/d1d/d1f/f2e [2686431,79974] 0 2024-04-24T08:24:57.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/661: readlink d2/d9/d1a/d36/d85/l9f 0 2024-04-24T08:24:57.961 INFO:tasks.workunit.client.0.smithi090.stdout:6/414: stat d8/d1d/d2f/d30/c32 0 2024-04-24T08:24:57.961 INFO:tasks.workunit.client.0.smithi090.stdout:2/445: creat d0/d2e/f93 x:0 0 0 2024-04-24T08:24:57.961 INFO:tasks.workunit.client.1.smithi149.stdout:0/662: chown d2/d3c 1288 1 2024-04-24T08:24:57.963 INFO:tasks.workunit.client.1.smithi149.stdout:1/829: creat d0/d1b/d5d/f110 x:0 0 0 2024-04-24T08:24:57.965 INFO:tasks.workunit.client.0.smithi090.stdout:4/333: link d3/d18/d1d/f40 d3/d18/d1d/d22/f62 0 2024-04-24T08:24:57.966 INFO:tasks.workunit.client.1.smithi149.stdout:4/656: write d2/f9 [3230430,14442] 0 2024-04-24T08:24:57.966 INFO:tasks.workunit.client.0.smithi090.stdout:8/365: mknod d2/d62/d71/c80 0 2024-04-24T08:24:57.967 INFO:tasks.workunit.client.0.smithi090.stdout:9/381: dwrite d0/d19/f1d [0,4194304] 0 2024-04-24T08:24:57.967 INFO:tasks.workunit.client.1.smithi149.stdout:4/657: chown d2/d18/d1e/d3b 707 1 2024-04-24T08:24:57.968 INFO:tasks.workunit.client.1.smithi149.stdout:7/723: symlink d9/d43/d4f/dc8/dea/lf3 0 2024-04-24T08:24:57.971 INFO:tasks.workunit.client.1.smithi149.stdout:7/724: write d9/fe5 [3197061,12012] 0 2024-04-24T08:24:57.973 INFO:tasks.workunit.client.1.smithi149.stdout:5/835: dwrite db/d12/d22/f31 [0,4194304] 0 2024-04-24T08:24:57.978 INFO:tasks.workunit.client.1.smithi149.stdout:6/654: rename d0/d1a/d28/d31/d33/l5e to d0/d1a/ldc 0 2024-04-24T08:24:57.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/725: write d9/dc/d11/d83/fd9 [101060,37461] 0 2024-04-24T08:24:57.979 INFO:tasks.workunit.client.0.smithi090.stdout:2/446: dwrite d0/d2e/f84 [4194304,4194304] 0 2024-04-24T08:24:57.980 INFO:tasks.workunit.client.0.smithi090.stdout:9/382: dread d0/f1b [0,4194304] 0 2024-04-24T08:24:57.983 INFO:tasks.workunit.client.0.smithi090.stdout:8/366: rmdir d2/da/d13/d64 39 2024-04-24T08:24:57.993 INFO:tasks.workunit.client.1.smithi149.stdout:4/658: creat d2/d24/d2e/d3f/fcc x:0 0 0 2024-04-24T08:24:57.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/440: sync 2024-04-24T08:24:57.994 INFO:tasks.workunit.client.1.smithi149.stdout:5/836: creat db/d12/d22/d45/f116 x:0 0 0 2024-04-24T08:24:57.997 INFO:tasks.workunit.client.0.smithi090.stdout:4/334: dwrite d3/d18/d1d/f46 [0,4194304] 0 2024-04-24T08:24:57.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/830: link d0/d1b/d5d/f9a d0/d1b/d5d/de5/f111 0 2024-04-24T08:24:57.999 INFO:tasks.workunit.client.1.smithi149.stdout:3/795: dwrite d0/d5/da/d5e/fa2 [0,4194304] 0 2024-04-24T08:24:57.999 INFO:tasks.workunit.client.1.smithi149.stdout:4/659: fdatasync d2/d18/d1e/d3c/d59/fb1 0 2024-04-24T08:24:58.005 INFO:tasks.workunit.client.1.smithi149.stdout:8/645: dwrite d1/d1f/d25/f8f [0,4194304] 0 2024-04-24T08:24:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.005+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03cc102140 con 0x7f03b406eac0 2024-04-24T08:24:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.007+0000 7f03c9600700 1 -- 172.21.15.90:0/4089326342 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f03cc102140 con 0x7f03b406eac0 2024-04-24T08:24:58.008 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:24:58.008 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:24:58.008 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:24:58.008 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:24:58.009 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:24:58.010 INFO:tasks.workunit.client.0.smithi090.stdout:6/415: creat d8/d1d/f7c x:0 0 0 2024-04-24T08:24:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 msgr2=0x7f03b4070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 0x7f03b4070f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005c50 tx=0x7f03c0007480 comp rx=0 tx=0).stop 2024-04-24T08:24:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 msgr2=0x7f03cc196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:24:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 0x7f03cc196e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009910 tx=0x7f03b800bd70 comp rx=0 tx=0).stop 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 shutdown_connections 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f03b406eac0 0x7f03b4070f70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc105710 0x7f03cc1938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 --2- 172.21.15.90:0/4089326342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03cc193e20 0x7f03cc196e80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.012+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 >> 172.21.15.90:0/4089326342 conn(0x7f03cc0fa780 msgr2=0x7f03cc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.013+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 shutdown_connections 2024-04-24T08:24:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:24:58.013+0000 7f03d3a00700 1 -- 172.21.15.90:0/4089326342 wait complete. 2024-04-24T08:24:58.024 INFO:tasks.workunit.client.1.smithi149.stdout:6/655: fdatasync d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fb6 0 2024-04-24T08:24:58.025 INFO:tasks.workunit.client.1.smithi149.stdout:4/660: creat d2/d18/d1e/fcd x:0 0 0 2024-04-24T08:24:58.029 INFO:tasks.workunit.client.0.smithi090.stdout:4/335: write d3/d38/f60 [3992239,41206] 0 2024-04-24T08:24:58.030 INFO:tasks.workunit.client.1.smithi149.stdout:4/661: chown d2/l8 4091 1 2024-04-24T08:24:58.033 INFO:tasks.workunit.client.0.smithi090.stdout:6/416: symlink d8/d1d/d2f/d30/l7d 0 2024-04-24T08:24:58.036 INFO:tasks.workunit.client.0.smithi090.stdout:2/447: creat d0/d7/f94 x:0 0 0 2024-04-24T08:24:58.037 INFO:tasks.workunit.client.0.smithi090.stdout:8/367: dread d2/da/d13/d34/f3c [0,4194304] 0 2024-04-24T08:24:58.037 INFO:tasks.workunit.client.1.smithi149.stdout:5/837: unlink db/d12/f5b 0 2024-04-24T08:24:58.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/417: read d8/dc/f3a [2515720,123622] 0 2024-04-24T08:24:58.043 INFO:tasks.workunit.client.0.smithi090.stdout:2/448: write d0/d7/d67/d7f/f5d [1053352,29327] 0 2024-04-24T08:24:58.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/441: write d1a/f71 [746682,96254] 0 2024-04-24T08:24:58.043 INFO:tasks.workunit.client.0.smithi090.stdout:4/336: rmdir d3/d24 39 2024-04-24T08:24:58.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/418: chown d8/d1d/f7c 6 1 2024-04-24T08:24:58.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/796: dread d0/d5/d53/d4c/d5f/f8b [0,4194304] 0 2024-04-24T08:24:58.046 INFO:tasks.workunit.client.1.smithi149.stdout:0/663: dwrite d2/d9/f89 [0,4194304] 0 2024-04-24T08:24:58.048 INFO:tasks.workunit.client.1.smithi149.stdout:4/662: creat d2/d18/da4/fce x:0 0 0 2024-04-24T08:24:58.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/797: chown d0/d5/da/d19/faf 454212668 1 2024-04-24T08:24:58.051 INFO:tasks.workunit.client.1.smithi149.stdout:7/726: dwrite d9/dc/d2e/d51/f59 [0,4194304] 0 2024-04-24T08:24:58.052 INFO:tasks.workunit.client.0.smithi090.stdout:8/368: write d2/da/d13/d64/d24/f46 [889098,50863] 0 2024-04-24T08:24:58.053 INFO:tasks.workunit.client.0.smithi090.stdout:6/419: dread d8/d1d/d1f/f58 [4194304,4194304] 0 2024-04-24T08:24:58.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/383: getdents d0/dc/d24 0 2024-04-24T08:24:58.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/420: write d8/dc/f10 [2040016,92777] 0 2024-04-24T08:24:58.063 INFO:tasks.workunit.client.1.smithi149.stdout:6/656: symlink d0/d1a/d28/d31/d4c/d80/d6f/dd7/ldd 0 2024-04-24T08:24:58.070 INFO:tasks.workunit.client.1.smithi149.stdout:3/798: creat d0/d5/d53/dd6/f110 x:0 0 0 2024-04-24T08:24:58.070 INFO:tasks.workunit.client.0.smithi090.stdout:9/384: unlink d0/c30 0 2024-04-24T08:24:58.076 INFO:tasks.workunit.client.1.smithi149.stdout:4/663: dwrite d2/d10/d91/fcb [0,4194304] 0 2024-04-24T08:24:58.076 INFO:tasks.workunit.client.0.smithi090.stdout:8/369: mkdir d2/da/d13/d64/d24/d81 0 2024-04-24T08:24:58.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/370: read d2/f12 [3628137,111451] 0 2024-04-24T08:24:58.079 INFO:tasks.workunit.client.1.smithi149.stdout:1/831: link d0/d1b/d4b/c89 d0/d5/df/c112 0 2024-04-24T08:24:58.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/371: truncate d2/f69 495429 0 2024-04-24T08:24:58.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/442: symlink d1a/l8f 0 2024-04-24T08:24:58.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/372: truncate d2/da/d13/d64/d50/f57 1182355 0 2024-04-24T08:24:58.085 INFO:tasks.workunit.client.1.smithi149.stdout:3/799: unlink d0/d5/d53/d34/f3b 0 2024-04-24T08:24:58.085 INFO:tasks.workunit.client.0.smithi090.stdout:9/385: dwrite d0/f5 [4194304,4194304] 0 2024-04-24T08:24:58.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/449: dread d0/d14/f38 [0,4194304] 0 2024-04-24T08:24:58.094 INFO:tasks.workunit.client.1.smithi149.stdout:1/832: creat d0/ddf/da0/db8/dc5/f113 x:0 0 0 2024-04-24T08:24:58.095 INFO:tasks.workunit.client.0.smithi090.stdout:2/450: truncate d0/d7/d67/d7f/d31/d46/d60/f74 4389782 0 2024-04-24T08:24:58.098 INFO:tasks.workunit.client.0.smithi090.stdout:9/386: dread d0/f18 [0,4194304] 0 2024-04-24T08:24:58.100 INFO:tasks.workunit.client.1.smithi149.stdout:4/664: dwrite d2/d18/d1e/d3c/f43 [0,4194304] 0 2024-04-24T08:24:58.101 INFO:tasks.workunit.client.1.smithi149.stdout:6/657: creat d0/d1a/d28/d31/d4c/d4e/d62/fde x:0 0 0 2024-04-24T08:24:58.101 INFO:tasks.workunit.client.0.smithi090.stdout:9/387: truncate d0/d19/d33/f57 426906 0 2024-04-24T08:24:58.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/373: creat d2/da/f82 x:0 0 0 2024-04-24T08:24:58.114 INFO:tasks.workunit.client.1.smithi149.stdout:5/838: dwrite db/d23/f80 [0,4194304] 0 2024-04-24T08:24:58.114 INFO:tasks.workunit.client.1.smithi149.stdout:6/658: chown d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/ld9 1 1 2024-04-24T08:24:58.115 INFO:tasks.workunit.client.1.smithi149.stdout:4/665: creat d2/d18/d1e/d3b/fcf x:0 0 0 2024-04-24T08:24:58.119 INFO:tasks.workunit.client.0.smithi090.stdout:9/388: mknod d0/dc/d24/c6d 0 2024-04-24T08:24:58.120 INFO:tasks.workunit.client.0.smithi090.stdout:9/389: chown d0/f2 0 1 2024-04-24T08:24:58.121 INFO:tasks.workunit.client.1.smithi149.stdout:5/839: chown db/c105 173531 1 2024-04-24T08:24:58.121 INFO:tasks.workunit.client.1.smithi149.stdout:0/664: truncate d2/d3c/d57/f78 1794429 0 2024-04-24T08:24:58.121 INFO:tasks.workunit.client.1.smithi149.stdout:6/659: write d0/d1a/d28/d31/f82 [10772,114706] 0 2024-04-24T08:24:58.122 INFO:tasks.workunit.client.1.smithi149.stdout:7/727: dwrite d9/dc/d11/d83/f93 [0,4194304] 0 2024-04-24T08:24:58.126 INFO:tasks.workunit.client.1.smithi149.stdout:0/665: readlink d2/d9/d1a/d36/d85/lbe 0 2024-04-24T08:24:58.126 INFO:tasks.workunit.client.0.smithi090.stdout:9/390: fdatasync d0/d19/f25 0 2024-04-24T08:24:58.134 INFO:tasks.workunit.client.0.smithi090.stdout:2/451: rename d0/d5e/f85 to d0/d7/f95 0 2024-04-24T08:24:58.136 INFO:tasks.workunit.client.1.smithi149.stdout:2/655: sync 2024-04-24T08:24:58.137 INFO:tasks.workunit.client.0.smithi090.stdout:0/443: dwrite d1a/d4b/f6f [0,4194304] 0 2024-04-24T08:24:58.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/444: chown d1a/d4b/d53/d79/d69 278 1 2024-04-24T08:24:58.140 INFO:tasks.workunit.client.1.smithi149.stdout:6/660: unlink d0/d1a/d28/d31/d4c/d4e/f8d 0 2024-04-24T08:24:58.146 INFO:tasks.workunit.client.1.smithi149.stdout:0/666: truncate d2/d9/df/f59 3360711 0 2024-04-24T08:24:58.148 INFO:tasks.workunit.client.0.smithi090.stdout:2/452: dread d0/d7/d67/d7f/d31/d46/d60/f74 [0,4194304] 0 2024-04-24T08:24:58.151 INFO:tasks.workunit.client.1.smithi149.stdout:0/667: fdatasync d2/d9/f2d 0 2024-04-24T08:24:58.152 INFO:tasks.workunit.client.1.smithi149.stdout:3/800: dwrite d0/d5/d53/d34/f35 [0,4194304] 0 2024-04-24T08:24:58.157 INFO:tasks.workunit.client.0.smithi090.stdout:0/445: mkdir d1a/d4b/d53/d79/d69/d90 0 2024-04-24T08:24:58.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/453: write d0/d7/d67/d7f/d31/d46/d60/f74 [3724204,101866] 0 2024-04-24T08:24:58.164 INFO:tasks.workunit.client.1.smithi149.stdout:1/833: dwrite d0/ddd/f4e [0,4194304] 0 2024-04-24T08:24:58.170 INFO:tasks.workunit.client.0.smithi090.stdout:9/391: rename d0/c6 to d0/c6e 0 2024-04-24T08:24:58.171 INFO:tasks.workunit.client.0.smithi090.stdout:9/392: fsync d0/d19/f1a 0 2024-04-24T08:24:58.173 INFO:tasks.workunit.client.1.smithi149.stdout:4/666: dread d2/d18/f1b [0,4194304] 0 2024-04-24T08:24:58.175 INFO:tasks.workunit.client.0.smithi090.stdout:0/446: readlink d1a/l60 0 2024-04-24T08:24:58.177 INFO:tasks.workunit.client.1.smithi149.stdout:5/840: creat db/d12/d22/f117 x:0 0 0 2024-04-24T08:24:58.180 INFO:tasks.workunit.client.1.smithi149.stdout:5/841: readlink db/d12/d2f/d74/lb3 0 2024-04-24T08:24:58.181 INFO:tasks.workunit.client.0.smithi090.stdout:2/454: rename d0/d7/d67/d7f/d81 to d0/d14/d4d/d50/d96 0 2024-04-24T08:24:58.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/842: dread - db/d12/d1e/d65/fe1 zero size 2024-04-24T08:24:58.185 INFO:tasks.workunit.client.1.smithi149.stdout:0/668: truncate d2/f7f 305913 0 2024-04-24T08:24:58.188 INFO:tasks.workunit.client.0.smithi090.stdout:0/447: chown d1a/d4b/d53/d79/d35/d68/f76 6502 1 2024-04-24T08:24:58.190 INFO:tasks.workunit.client.1.smithi149.stdout:3/801: mknod d0/d5/da/d19/d76/c111 0 2024-04-24T08:24:58.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/455: dread d0/d2e/f6a [4194304,4194304] 0 2024-04-24T08:24:58.195 INFO:tasks.workunit.client.1.smithi149.stdout:2/656: creat d0/d6/db/d28/db0/fc0 x:0 0 0 2024-04-24T08:24:58.195 INFO:tasks.workunit.client.0.smithi090.stdout:5/482: sync 2024-04-24T08:24:58.195 INFO:tasks.workunit.client.0.smithi090.stdout:3/357: sync 2024-04-24T08:24:58.195 INFO:tasks.workunit.client.0.smithi090.stdout:1/426: sync 2024-04-24T08:24:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:4/667: dwrite d2/d10/d1a/fba [0,4194304] 0 2024-04-24T08:24:58.202 INFO:tasks.workunit.client.1.smithi149.stdout:4/668: write d2/d10/d1a/f36 [6514604,67189] 0 2024-04-24T08:24:58.202 INFO:tasks.workunit.client.1.smithi149.stdout:6/661: dwrite d0/f3f [0,4194304] 0 2024-04-24T08:24:58.203 INFO:tasks.workunit.client.0.smithi090.stdout:0/448: dwrite d1a/d4b/d53/d79/d35/d62/f6e [0,4194304] 0 2024-04-24T08:24:58.204 INFO:tasks.workunit.client.1.smithi149.stdout:7/728: getdents d9/dc 0 2024-04-24T08:24:58.204 INFO:tasks.workunit.client.0.smithi090.stdout:2/456: symlink d0/d2c/l97 0 2024-04-24T08:24:58.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/358: symlink d2/d25/l73 0 2024-04-24T08:24:58.210 INFO:tasks.workunit.client.1.smithi149.stdout:6/662: dread - d0/d1a/d28/d31/d38/fc6 zero size 2024-04-24T08:24:58.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/449: dread - d1a/f3f zero size 2024-04-24T08:24:58.215 INFO:tasks.workunit.client.1.smithi149.stdout:5/843: dwrite db/f94 [0,4194304] 0 2024-04-24T08:24:58.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/785: sync 2024-04-24T08:24:58.224 INFO:tasks.workunit.client.1.smithi149.stdout:9/786: chown d1/da/dd/f24 49 1 2024-04-24T08:24:58.229 INFO:tasks.workunit.client.0.smithi090.stdout:0/450: creat d1a/d4b/d53/d79/d3c/f91 x:0 0 0 2024-04-24T08:24:58.231 INFO:tasks.workunit.client.1.smithi149.stdout:6/663: unlink d0/f73 0 2024-04-24T08:24:58.232 INFO:tasks.workunit.client.0.smithi090.stdout:9/393: rmdir d0/dc/d24/d2d/d6c 0 2024-04-24T08:24:58.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/394: write d0/fb [1445593,107089] 0 2024-04-24T08:24:58.235 INFO:tasks.workunit.client.0.smithi090.stdout:3/359: link d2/df/d5b/f63 d2/df/d1a/d37/d53/f74 0 2024-04-24T08:24:58.236 INFO:tasks.workunit.client.1.smithi149.stdout:5/844: mknod db/d12/d2f/d74/df6/c118 0 2024-04-24T08:24:58.239 INFO:tasks.workunit.client.1.smithi149.stdout:3/802: creat d0/d5/d1a/d70/da3/f112 x:0 0 0 2024-04-24T08:24:58.240 INFO:tasks.workunit.client.1.smithi149.stdout:2/657: rename d0/d6/db/d27/l2e to d0/d6/db/d39/d3d/d44/d8b/lc1 0 2024-04-24T08:24:58.246 INFO:tasks.workunit.client.0.smithi090.stdout:0/451: dwrite d1a/f1e [0,4194304] 0 2024-04-24T08:24:58.247 INFO:tasks.workunit.client.1.smithi149.stdout:7/729: mknod d9/d43/d4f/da5/cf4 0 2024-04-24T08:24:58.247 INFO:tasks.workunit.client.0.smithi090.stdout:1/427: rename d3/d1d/f6e to d3/d1d/d34/f98 0 2024-04-24T08:24:58.248 INFO:tasks.workunit.client.0.smithi090.stdout:9/395: rmdir d0/d19 39 2024-04-24T08:24:58.248 INFO:tasks.workunit.client.0.smithi090.stdout:3/360: symlink d2/d25/d33/l75 0 2024-04-24T08:24:58.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/457: getdents d0/d14/d36 0 2024-04-24T08:24:58.249 INFO:tasks.workunit.client.1.smithi149.stdout:9/787: dread d1/de/fa8 [0,4194304] 0 2024-04-24T08:24:58.251 INFO:tasks.workunit.client.0.smithi090.stdout:3/361: truncate d2/df/d1a/d37/d59/f61 901644 0 2024-04-24T08:24:58.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/803: rmdir d0/d5/d53/d4c/d5f 39 2024-04-24T08:24:58.254 INFO:tasks.workunit.client.1.smithi149.stdout:3/804: stat d0/d26/d3c/db1/deb 0 2024-04-24T08:24:58.255 INFO:tasks.workunit.client.0.smithi090.stdout:0/452: creat d1a/d4b/d53/d79/d69/d78/f92 x:0 0 0 2024-04-24T08:24:58.255 INFO:tasks.workunit.client.0.smithi090.stdout:0/453: chown d1a/f52 5440 1 2024-04-24T08:24:58.258 INFO:tasks.workunit.client.0.smithi090.stdout:9/396: fdatasync d0/dc/f2c 0 2024-04-24T08:24:58.258 INFO:tasks.workunit.client.1.smithi149.stdout:4/669: rename d2/d10/d1a/f6f to d2/d24/da7/fd0 0 2024-04-24T08:24:58.260 INFO:tasks.workunit.client.1.smithi149.stdout:4/670: truncate d2/fc2 709776 0 2024-04-24T08:24:58.260 INFO:tasks.workunit.client.1.smithi149.stdout:9/788: fsync d1/da/d4a/dad/fc2 0 2024-04-24T08:24:58.260 INFO:tasks.workunit.client.0.smithi090.stdout:2/458: creat d0/d14/d4d/d50/d96/f98 x:0 0 0 2024-04-24T08:24:58.262 INFO:tasks.workunit.client.0.smithi090.stdout:1/428: link d3/d5/f1a d3/d1d/d34/d73/f99 0 2024-04-24T08:24:58.263 INFO:tasks.workunit.client.0.smithi090.stdout:2/459: dread - d0/d14/d4d/d50/d96/f98 zero size 2024-04-24T08:24:58.266 INFO:tasks.workunit.client.0.smithi090.stdout:3/362: mknod d2/d25/d2c/c76 0 2024-04-24T08:24:58.268 INFO:tasks.workunit.client.1.smithi149.stdout:5/845: rename db/d35/f9d to db/d12/d1e/d85/d92/f119 0 2024-04-24T08:24:58.268 INFO:tasks.workunit.client.0.smithi090.stdout:0/454: creat d1a/d4b/d53/d79/d58/f93 x:0 0 0 2024-04-24T08:24:58.278 INFO:tasks.workunit.client.0.smithi090.stdout:1/429: rmdir d3/d5/d58 39 2024-04-24T08:24:58.279 INFO:tasks.workunit.client.0.smithi090.stdout:2/460: chown d0/d7/d67/d7f/d31/f41 848987705 1 2024-04-24T08:24:58.280 INFO:tasks.workunit.client.1.smithi149.stdout:1/834: write d0/d1b/db5/f8b [1060129,25664] 0 2024-04-24T08:24:58.283 INFO:tasks.workunit.client.0.smithi090.stdout:3/363: symlink d2/df/d57/l77 0 2024-04-24T08:24:58.284 INFO:tasks.workunit.client.1.smithi149.stdout:4/671: dwrite d2/fc2 [0,4194304] 0 2024-04-24T08:24:58.285 INFO:tasks.workunit.client.0.smithi090.stdout:3/364: readlink d2/d25/d2c/l41 0 2024-04-24T08:24:58.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/365: dread - d2/df/d57/d4f/f72 zero size 2024-04-24T08:24:58.288 INFO:tasks.workunit.client.1.smithi149.stdout:9/789: dwrite d1/da/dd/d79/db6/feb [0,4194304] 0 2024-04-24T08:24:58.292 INFO:tasks.workunit.client.0.smithi090.stdout:9/397: unlink d0/dc/d24/f4b 0 2024-04-24T08:24:58.310 INFO:tasks.workunit.client.1.smithi149.stdout:4/672: dread d2/d10/d1a/fba [0,4194304] 0 2024-04-24T08:24:58.312 INFO:tasks.workunit.client.1.smithi149.stdout:1/835: mknod d0/d5/d4f/d60/c114 0 2024-04-24T08:24:58.315 INFO:tasks.workunit.client.1.smithi149.stdout:3/805: dread d0/d5/da/d19/d1b/f2c [0,4194304] 0 2024-04-24T08:24:58.316 INFO:tasks.workunit.client.0.smithi090.stdout:3/366: dread d2/df/f10 [0,4194304] 0 2024-04-24T08:24:58.325 INFO:tasks.workunit.client.1.smithi149.stdout:7/730: dwrite d9/dc/d2e/d51/fc0 [0,4194304] 0 2024-04-24T08:24:58.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/836: chown d0/d1b/db5/fb1 92880 1 2024-04-24T08:24:58.327 INFO:tasks.workunit.client.1.smithi149.stdout:4/673: creat d2/d24/d2e/d38/fd1 x:0 0 0 2024-04-24T08:24:58.339 INFO:tasks.workunit.client.1.smithi149.stdout:1/837: truncate d0/ddd/fd4 1799764 0 2024-04-24T08:24:58.341 INFO:tasks.workunit.client.1.smithi149.stdout:5/846: write db/d12/d60/d8f/fac [9179947,87045] 0 2024-04-24T08:24:58.344 INFO:tasks.workunit.client.1.smithi149.stdout:4/674: symlink d2/d24/d2e/d38/ld2 0 2024-04-24T08:24:58.345 INFO:tasks.workunit.client.1.smithi149.stdout:7/731: symlink d9/dc/d7b/dc2/lf5 0 2024-04-24T08:24:58.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/790: write d1/d2/d33/d60/f77 [728086,515] 0 2024-04-24T08:24:58.354 INFO:tasks.workunit.client.1.smithi149.stdout:4/675: creat d2/da8/fd3 x:0 0 0 2024-04-24T08:24:58.359 INFO:tasks.workunit.client.1.smithi149.stdout:9/791: symlink d1/d99/l11e 0 2024-04-24T08:24:58.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/455: mknod d1a/d4b/d53/d79/d69/d8a/c94 0 2024-04-24T08:24:58.364 INFO:tasks.workunit.client.1.smithi149.stdout:0/669: write d2/d17/f24 [486338,34266] 0 2024-04-24T08:24:58.364 INFO:tasks.workunit.client.1.smithi149.stdout:4/676: creat d2/d18/d1e/d3c/d48/d69/d7d/d8d/fd4 x:0 0 0 2024-04-24T08:24:58.377 INFO:tasks.workunit.client.0.smithi090.stdout:4/337: sync 2024-04-24T08:24:58.378 INFO:tasks.workunit.client.0.smithi090.stdout:7/461: sync 2024-04-24T08:24:58.433 INFO:tasks.workunit.client.1.smithi149.stdout:8/646: sync 2024-04-24T08:24:58.451 INFO:tasks.workunit.client.0.smithi090.stdout:6/421: sync 2024-04-24T08:24:58.451 INFO:tasks.workunit.client.0.smithi090.stdout:8/374: sync 2024-04-24T08:24:58.453 INFO:tasks.workunit.client.0.smithi090.stdout:6/422: dread - d8/d1d/f7c zero size 2024-04-24T08:24:58.470 INFO:tasks.workunit.client.1.smithi149.stdout:5/847: creat db/d12/d60/db2/d83/dc5/f11a x:0 0 0 2024-04-24T08:24:58.474 INFO:tasks.workunit.client.0.smithi090.stdout:9/398: stat d0/d19/d33/d4d 0 2024-04-24T08:24:58.475 INFO:tasks.workunit.client.0.smithi090.stdout:3/367: unlink d2/df/d57/l5a 0 2024-04-24T08:24:58.475 INFO:tasks.workunit.client.1.smithi149.stdout:5/848: chown db/d12/d2f/d74/dc6/f103 2 1 2024-04-24T08:24:58.477 INFO:tasks.workunit.client.1.smithi149.stdout:1/838: write d0/d5/db/d18/f6f [981743,97077] 0 2024-04-24T08:24:58.483 INFO:tasks.workunit.client.0.smithi090.stdout:0/456: mknod d1a/d4b/d53/d79/d3c/c95 0 2024-04-24T08:24:58.491 INFO:tasks.workunit.client.0.smithi090.stdout:8/375: dwrite d2/f69 [0,4194304] 0 2024-04-24T08:24:58.491 INFO:tasks.workunit.client.0.smithi090.stdout:7/462: mknod db/d54/d83/c9e 0 2024-04-24T08:24:58.495 INFO:tasks.workunit.client.0.smithi090.stdout:4/338: creat d3/d38/d49/f63 x:0 0 0 2024-04-24T08:24:58.505 INFO:tasks.workunit.client.0.smithi090.stdout:6/423: dread d8/d1d/d1f/d3e/f5c [0,4194304] 0 2024-04-24T08:24:58.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/376: dread d2/d62/d6b/f7d [0,4194304] 0 2024-04-24T08:24:58.508 INFO:tasks.workunit.client.0.smithi090.stdout:4/339: truncate d3/d18/d1d/f40 2499965 0 2024-04-24T08:24:58.510 INFO:tasks.workunit.client.0.smithi090.stdout:1/430: dwrite d3/d5/f16 [0,4194304] 0 2024-04-24T08:24:58.512 INFO:tasks.workunit.client.0.smithi090.stdout:3/368: rmdir d2/df/d57 39 2024-04-24T08:24:58.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/377: write d2/da/f82 [344316,26976] 0 2024-04-24T08:24:58.516 INFO:tasks.workunit.client.0.smithi090.stdout:8/378: stat d2/da/d13/d34/f3c 0 2024-04-24T08:24:58.516 INFO:tasks.workunit.client.0.smithi090.stdout:2/461: truncate d0/d7/f95 1459 0 2024-04-24T08:24:58.518 INFO:tasks.workunit.client.0.smithi090.stdout:2/462: chown d0/d2c/f76 789 1 2024-04-24T08:24:58.519 INFO:tasks.workunit.client.0.smithi090.stdout:0/457: mkdir d1a/d4b/d53/d79/d58/d96 0 2024-04-24T08:24:58.522 INFO:tasks.workunit.client.0.smithi090.stdout:2/463: write d0/d14/d4d/d50/f5c [1307739,46885] 0 2024-04-24T08:24:58.536 INFO:tasks.workunit.client.0.smithi090.stdout:6/424: unlink d8/dc/d19/l5b 0 2024-04-24T08:24:58.539 INFO:tasks.workunit.client.0.smithi090.stdout:1/431: unlink d3/d5/l6 0 2024-04-24T08:24:58.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/340: write d3/f3f [3478662,60781] 0 2024-04-24T08:24:58.542 INFO:tasks.workunit.client.0.smithi090.stdout:1/432: fsync d3/d5/f17 0 2024-04-24T08:24:58.543 INFO:tasks.workunit.client.0.smithi090.stdout:9/399: creat d0/d69/f6f x:0 0 0 2024-04-24T08:24:58.545 INFO:tasks.workunit.client.0.smithi090.stdout:8/379: fdatasync d2/da/d13/d34/f6f 0 2024-04-24T08:24:58.546 INFO:tasks.workunit.client.0.smithi090.stdout:1/433: read d3/d1d/d34/d73/f99 [3012451,121077] 0 2024-04-24T08:24:58.548 INFO:tasks.workunit.client.0.smithi090.stdout:0/458: symlink d1a/d4b/d53/d79/d35/l97 0 2024-04-24T08:24:58.553 INFO:tasks.workunit.client.0.smithi090.stdout:2/464: mkdir d0/d2c/d99 0 2024-04-24T08:24:58.553 INFO:tasks.workunit.client.0.smithi090.stdout:1/434: write d3/d5/da/f82 [783926,99938] 0 2024-04-24T08:24:58.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/425: mkdir d8/d1d/d1f/d3e/d4f/d7e 0 2024-04-24T08:24:58.555 INFO:tasks.workunit.client.0.smithi090.stdout:5/483: sync 2024-04-24T08:24:58.556 INFO:tasks.workunit.client.0.smithi090.stdout:2/465: chown d0/d14/d17/f18 20 1 2024-04-24T08:24:58.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/463: dwrite db/f1a [0,4194304] 0 2024-04-24T08:24:58.558 INFO:tasks.workunit.client.0.smithi090.stdout:1/435: fsync d3/d5/da/dc/f81 0 2024-04-24T08:24:58.559 INFO:tasks.workunit.client.0.smithi090.stdout:4/341: fdatasync d3/d24/f44 0 2024-04-24T08:24:58.560 INFO:tasks.workunit.client.0.smithi090.stdout:5/484: stat d8/da/d3a/f55 0 2024-04-24T08:24:58.560 INFO:tasks.workunit.client.0.smithi090.stdout:3/369: symlink d2/l78 0 2024-04-24T08:24:58.561 INFO:tasks.workunit.client.0.smithi090.stdout:1/436: chown d3/d5/f17 828584779 1 2024-04-24T08:24:58.562 INFO:tasks.workunit.client.0.smithi090.stdout:2/466: dread - d0/d7/d67/d7f/f6e zero size 2024-04-24T08:24:58.563 INFO:tasks.workunit.client.0.smithi090.stdout:9/400: symlink d0/d19/d33/d4d/l70 0 2024-04-24T08:24:58.563 INFO:tasks.workunit.client.0.smithi090.stdout:8/380: rmdir d2/da/d13 39 2024-04-24T08:24:58.565 INFO:tasks.workunit.client.0.smithi090.stdout:2/467: write d0/d14/d17/f1b [4793228,43680] 0 2024-04-24T08:24:58.568 INFO:tasks.workunit.client.0.smithi090.stdout:0/459: creat d1a/d4b/d53/d79/d82/f98 x:0 0 0 2024-04-24T08:24:58.570 INFO:tasks.workunit.client.0.smithi090.stdout:0/460: write d1a/d4b/f85 [878048,3973] 0 2024-04-24T08:24:58.571 INFO:tasks.workunit.client.0.smithi090.stdout:0/461: chown d1a/f64 70842596 1 2024-04-24T08:24:58.578 INFO:tasks.workunit.client.1.smithi149.stdout:7/732: creat d9/dc/d11/d83/da6/ff6 x:0 0 0 2024-04-24T08:24:58.579 INFO:tasks.workunit.client.0.smithi090.stdout:4/342: creat d3/d18/d1d/d4b/f64 x:0 0 0 2024-04-24T08:24:58.580 INFO:tasks.workunit.client.1.smithi149.stdout:9/792: rmdir d1/da/d13/d74/dbb/dc0 39 2024-04-24T08:24:58.581 INFO:tasks.workunit.client.0.smithi090.stdout:5/485: fdatasync d8/da/d14/d2f/d44/f49 0 2024-04-24T08:24:58.587 INFO:tasks.workunit.client.0.smithi090.stdout:9/401: mknod d0/dc/d24/d2d/d49/c71 0 2024-04-24T08:24:58.588 INFO:tasks.workunit.client.1.smithi149.stdout:5/849: read db/d12/d1e/fc1 [1395947,5725] 0 2024-04-24T08:24:58.593 INFO:tasks.workunit.client.0.smithi090.stdout:2/468: creat d0/d5e/f9a x:0 0 0 2024-04-24T08:24:58.595 INFO:tasks.workunit.client.1.smithi149.stdout:5/850: creat db/d12/d2f/d74/f11b x:0 0 0 2024-04-24T08:24:58.596 INFO:tasks.workunit.client.0.smithi090.stdout:3/370: dread - d2/df/d21/f30 zero size 2024-04-24T08:24:58.597 INFO:tasks.workunit.client.0.smithi090.stdout:1/437: truncate d3/d1d/d34/d73/f99 4972008 0 2024-04-24T08:24:58.599 INFO:tasks.workunit.client.0.smithi090.stdout:4/343: unlink d3/f3f 0 2024-04-24T08:24:58.600 INFO:tasks.workunit.client.0.smithi090.stdout:0/462: dread f8 [0,4194304] 0 2024-04-24T08:24:58.600 INFO:tasks.workunit.client.0.smithi090.stdout:4/344: write d3/d38/f39 [7516884,20873] 0 2024-04-24T08:24:58.603 INFO:tasks.workunit.client.0.smithi090.stdout:4/345: fdatasync d3/d18/d1d/d4b/f57 0 2024-04-24T08:24:58.603 INFO:tasks.workunit.client.0.smithi090.stdout:0/463: dread - d1a/d4b/d53/d79/d35/d68/f7b zero size 2024-04-24T08:24:58.607 INFO:tasks.workunit.client.0.smithi090.stdout:6/426: rename d8/d1d/d2f/d30/c55 to d8/dc/d19/c7f 0 2024-04-24T08:24:58.608 INFO:tasks.workunit.client.1.smithi149.stdout:4/677: write d2/d24/d2e/d3f/f4d [1063092,60586] 0 2024-04-24T08:24:58.609 INFO:tasks.workunit.client.0.smithi090.stdout:2/469: readlink d0/la 0 2024-04-24T08:24:58.609 INFO:tasks.workunit.client.1.smithi149.stdout:7/733: getdents d9/d43/d4f/d66/d96 0 2024-04-24T08:24:58.611 INFO:tasks.workunit.client.0.smithi090.stdout:7/464: dwrite f7 [4194304,4194304] 0 2024-04-24T08:24:58.616 INFO:tasks.workunit.client.1.smithi149.stdout:4/678: truncate d2/f44 9061980 0 2024-04-24T08:24:58.617 INFO:tasks.workunit.client.0.smithi090.stdout:1/438: symlink d3/d1d/d34/d5a/l9a 0 2024-04-24T08:24:58.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/647: dwrite d1/de/d18/d32/d63/d76/f9a [0,4194304] 0 2024-04-24T08:24:58.619 INFO:tasks.workunit.client.1.smithi149.stdout:7/734: symlink d9/d43/d4f/da5/lf7 0 2024-04-24T08:24:58.623 INFO:tasks.workunit.client.0.smithi090.stdout:5/486: write d8/d59/f61 [5114073,67532] 0 2024-04-24T08:24:58.623 INFO:tasks.workunit.client.1.smithi149.stdout:8/648: chown d1/d1f 10390695 1 2024-04-24T08:24:58.625 INFO:tasks.workunit.client.1.smithi149.stdout:5/851: link db/d12/d2f/d78/fd4 db/d12/d2f/d74/dc6/f11c 0 2024-04-24T08:24:58.627 INFO:tasks.workunit.client.0.smithi090.stdout:4/346: creat d3/d38/d49/f65 x:0 0 0 2024-04-24T08:24:58.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/649: readlink d1/de/d18/d32/d63/d9e/lcd 0 2024-04-24T08:24:58.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/381: dwrite d2/da/d13/d64/f16 [0,4194304] 0 2024-04-24T08:24:58.635 INFO:tasks.workunit.client.1.smithi149.stdout:7/735: mknod d9/cf8 0 2024-04-24T08:24:58.636 INFO:tasks.workunit.client.1.smithi149.stdout:5/852: mkdir db/d12/d1e/d11d 0 2024-04-24T08:24:58.638 INFO:tasks.workunit.client.0.smithi090.stdout:6/427: dread d8/dc/d19/f3d [0,4194304] 0 2024-04-24T08:24:58.639 INFO:tasks.workunit.client.0.smithi090.stdout:3/371: mknod d2/d25/d33/c79 0 2024-04-24T08:24:58.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/439: creat d3/d5/d3e/f9b x:0 0 0 2024-04-24T08:24:58.644 INFO:tasks.workunit.client.0.smithi090.stdout:4/347: creat d3/d18/d5c/f66 x:0 0 0 2024-04-24T08:24:58.644 INFO:tasks.workunit.client.1.smithi149.stdout:5/853: read db/d12/d60/db2/d83/dc8/ffe [34376,103686] 0 2024-04-24T08:24:58.644 INFO:tasks.workunit.client.1.smithi149.stdout:8/650: mknod d1/d1f/d5e/d9c/cdb 0 2024-04-24T08:24:58.645 INFO:tasks.workunit.client.0.smithi090.stdout:4/348: write d3/d18/d1d/f46 [2749222,5130] 0 2024-04-24T08:24:58.648 INFO:tasks.workunit.client.0.smithi090.stdout:4/349: chown d3/d18/d1d/d4b/f57 3 1 2024-04-24T08:24:58.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/428: dread d8/dc/f72 [0,4194304] 0 2024-04-24T08:24:58.648 INFO:tasks.workunit.client.0.smithi090.stdout:8/382: rename d2/da/d13/d60/d76/d7f to d2/d62/d83 0 2024-04-24T08:24:58.652 INFO:tasks.workunit.client.1.smithi149.stdout:5/854: creat db/d12/d60/db2/de2/f11e x:0 0 0 2024-04-24T08:24:58.657 INFO:tasks.workunit.client.1.smithi149.stdout:8/651: unlink d1/de/f54 0 2024-04-24T08:24:58.660 INFO:tasks.workunit.client.1.smithi149.stdout:7/736: dwrite d9/dc/d72/f76 [0,4194304] 0 2024-04-24T08:24:58.666 INFO:tasks.workunit.client.0.smithi090.stdout:9/402: truncate d0/fb 1467009 0 2024-04-24T08:24:58.666 INFO:tasks.workunit.client.0.smithi090.stdout:7/465: dread db/d17/d39/d59/f6d [0,4194304] 0 2024-04-24T08:24:58.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/652: mkdir d1/de/d18/d32/d65/d8b/d7d/ddc 0 2024-04-24T08:24:58.668 INFO:tasks.workunit.client.0.smithi090.stdout:7/466: write db/f15 [4669128,116071] 0 2024-04-24T08:24:58.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/440: read d3/d5/da/dc/f40 [1413851,41882] 0 2024-04-24T08:24:58.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/464: write d1a/d4b/d53/f6a [105789,75185] 0 2024-04-24T08:24:58.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/653: write d1/de/d18/fb7 [77937,52769] 0 2024-04-24T08:24:58.671 INFO:tasks.workunit.client.0.smithi090.stdout:4/350: rename d3/l8 to d3/d38/d49/l67 0 2024-04-24T08:24:58.672 INFO:tasks.workunit.client.0.smithi090.stdout:4/351: readlink d3/d18/d1d/l54 0 2024-04-24T08:24:58.675 INFO:tasks.workunit.client.0.smithi090.stdout:2/470: link d0/d14/d36/c39 d0/d7/d6f/c9b 0 2024-04-24T08:24:58.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/679: dwrite d2/d18/d1e/d3c/d48/d69/f70 [0,4194304] 0 2024-04-24T08:24:58.678 INFO:tasks.workunit.client.1.smithi149.stdout:8/654: creat d1/de/d18/d91/d9f/fdd x:0 0 0 2024-04-24T08:24:58.680 INFO:tasks.workunit.client.0.smithi090.stdout:8/383: dread d2/da/d13/f3a [0,4194304] 0 2024-04-24T08:24:58.684 INFO:tasks.workunit.client.0.smithi090.stdout:0/465: creat d1a/d4b/d53/d79/d69/d78/f99 x:0 0 0 2024-04-24T08:24:58.687 INFO:tasks.workunit.client.0.smithi090.stdout:3/372: dread d2/df/d1a/d37/f49 [0,4194304] 0 2024-04-24T08:24:58.689 INFO:tasks.workunit.client.1.smithi149.stdout:8/655: write d1/de/d18/d43/f55 [5212457,63293] 0 2024-04-24T08:24:58.690 INFO:tasks.workunit.client.0.smithi090.stdout:6/429: link d8/dc/f10 d8/d1d/d1f/d3e/d4f/d7e/f80 0 2024-04-24T08:24:58.690 INFO:tasks.workunit.client.1.smithi149.stdout:8/656: fsync d1/d1f/d5e/f73 0 2024-04-24T08:24:58.692 INFO:tasks.workunit.client.0.smithi090.stdout:2/471: symlink d0/d14/d4d/d50/l9c 0 2024-04-24T08:24:58.692 INFO:tasks.workunit.client.1.smithi149.stdout:8/657: chown d1/d1f/d25 109 1 2024-04-24T08:24:58.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/472: stat d0/d7/d67/f72 0 2024-04-24T08:24:58.703 INFO:tasks.workunit.client.0.smithi090.stdout:2/473: dread d0/d14/f34 [0,4194304] 0 2024-04-24T08:24:58.704 INFO:tasks.workunit.client.0.smithi090.stdout:7/467: fdatasync db/d17/f3a 0 2024-04-24T08:24:58.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/680: link d2/l3 d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/ld5 0 2024-04-24T08:24:58.706 INFO:tasks.workunit.client.1.smithi149.stdout:4/681: fsync d2/d18/da4/fce 0 2024-04-24T08:24:58.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/384: truncate d2/da/d13/d64/d24/f21 1086544 0 2024-04-24T08:24:58.708 INFO:tasks.workunit.client.1.smithi149.stdout:8/658: mkdir d1/de/d18/d32/dde 0 2024-04-24T08:24:58.711 INFO:tasks.workunit.client.1.smithi149.stdout:7/737: truncate d9/d43/d4f/d66/f6e 821353 0 2024-04-24T08:24:58.712 INFO:tasks.workunit.client.1.smithi149.stdout:7/738: readlink d9/d43/l62 0 2024-04-24T08:24:58.712 INFO:tasks.workunit.client.0.smithi090.stdout:5/487: dwrite d8/da/d3a/d2d/d37/d93/d7a/d50/f6c [0,4194304] 0 2024-04-24T08:24:58.713 INFO:tasks.workunit.client.1.smithi149.stdout:4/682: mkdir d2/d18/dd6 0 2024-04-24T08:24:58.713 INFO:tasks.workunit.client.0.smithi090.stdout:9/403: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:58.714 INFO:tasks.workunit.client.0.smithi090.stdout:3/373: chown d2/cb 995768 1 2024-04-24T08:24:58.715 INFO:tasks.workunit.client.0.smithi090.stdout:6/430: mkdir d8/d1d/d2f/d30/d81 0 2024-04-24T08:24:58.716 INFO:tasks.workunit.client.0.smithi090.stdout:6/431: fsync d8/f16 0 2024-04-24T08:24:58.718 INFO:tasks.workunit.client.0.smithi090.stdout:6/432: dread - d8/d1d/d2f/d30/f62 zero size 2024-04-24T08:24:58.721 INFO:tasks.workunit.client.0.smithi090.stdout:6/433: truncate d8/d1d/d2f/d30/f62 631611 0 2024-04-24T08:24:58.723 INFO:tasks.workunit.client.0.smithi090.stdout:6/434: fdatasync d8/d1d/d29/f40 0 2024-04-24T08:24:58.728 INFO:tasks.workunit.client.0.smithi090.stdout:7/468: creat db/d17/d29/d38/d99/f9f x:0 0 0 2024-04-24T08:24:58.730 INFO:tasks.workunit.client.0.smithi090.stdout:4/352: creat d3/df/f68 x:0 0 0 2024-04-24T08:24:58.733 INFO:tasks.workunit.client.0.smithi090.stdout:5/488: unlink d8/d25/c40 0 2024-04-24T08:24:58.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/404: mkdir d0/d5e/d72 0 2024-04-24T08:24:58.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/405: write d0/dc/f2c [2644536,47346] 0 2024-04-24T08:24:58.736 INFO:tasks.workunit.client.0.smithi090.stdout:6/435: dread d8/d1d/d29/f2c [0,4194304] 0 2024-04-24T08:24:58.737 INFO:tasks.workunit.client.0.smithi090.stdout:6/436: stat d8/dc/f5f 0 2024-04-24T08:24:58.739 INFO:tasks.workunit.client.0.smithi090.stdout:2/474: creat d0/d7/d67/d7f/d31/d90/f9d x:0 0 0 2024-04-24T08:24:58.740 INFO:tasks.workunit.client.0.smithi090.stdout:0/466: creat d1a/f9a x:0 0 0 2024-04-24T08:24:58.740 INFO:tasks.workunit.client.0.smithi090.stdout:7/469: rmdir db/d17/d29/d38/d3e 39 2024-04-24T08:24:58.744 INFO:tasks.workunit.client.0.smithi090.stdout:3/374: chown d2/d15/d36/l64 1633984457 1 2024-04-24T08:24:58.745 INFO:tasks.workunit.client.0.smithi090.stdout:1/441: getdents d3/d5/d58 0 2024-04-24T08:24:58.747 INFO:tasks.workunit.client.0.smithi090.stdout:1/442: read - d3/d1d/d34/d5a/f8b zero size 2024-04-24T08:24:58.749 INFO:tasks.workunit.client.0.smithi090.stdout:0/467: dread d1a/d4b/f85 [0,4194304] 0 2024-04-24T08:24:58.750 INFO:tasks.workunit.client.0.smithi090.stdout:6/437: creat d8/d1d/d1f/d3e/d44/f82 x:0 0 0 2024-04-24T08:24:58.751 INFO:tasks.workunit.client.1.smithi149.stdout:8/659: write d1/de/d18/d32/d63/d9e/fd7 [575388,2576] 0 2024-04-24T08:24:58.753 INFO:tasks.workunit.client.0.smithi090.stdout:4/353: dread d3/f13 [0,4194304] 0 2024-04-24T08:24:58.753 INFO:tasks.workunit.client.0.smithi090.stdout:5/489: mknod d8/ca3 0 2024-04-24T08:24:58.755 INFO:tasks.workunit.client.0.smithi090.stdout:7/470: creat db/d17/d39/d59/fa0 x:0 0 0 2024-04-24T08:24:58.758 INFO:tasks.workunit.client.1.smithi149.stdout:7/739: write d9/dc/d11/d83/da6/f5c [1409053,32702] 0 2024-04-24T08:24:58.758 INFO:tasks.workunit.client.1.smithi149.stdout:4/683: truncate d2/fc2 1945300 0 2024-04-24T08:24:58.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/406: link d0/fb d0/dc/d24/d2d/d49/f73 0 2024-04-24T08:24:58.763 INFO:tasks.workunit.client.0.smithi090.stdout:6/438: chown d8/dc/l26 12288 1 2024-04-24T08:24:58.770 INFO:tasks.workunit.client.0.smithi090.stdout:7/471: readlink db/d17/l2b 0 2024-04-24T08:24:58.772 INFO:tasks.workunit.client.0.smithi090.stdout:3/375: mkdir d2/d15/d36/d6f/d7a 0 2024-04-24T08:24:58.773 INFO:tasks.workunit.client.0.smithi090.stdout:1/443: mkdir d3/d5/da/d97/d9c 0 2024-04-24T08:24:58.774 INFO:tasks.workunit.client.1.smithi149.stdout:7/740: creat d9/dc/d11/d83/da6/d67/ff9 x:0 0 0 2024-04-24T08:24:58.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:58 smithi149 ceph-mon[31282]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:58.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/407: read d0/dc/f3e [44117,108545] 0 2024-04-24T08:24:58.777 INFO:tasks.workunit.client.0.smithi090.stdout:2/475: rename d0/d7/d67/d7f/d31/f52 to d0/f9e 0 2024-04-24T08:24:58.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/664: sync 2024-04-24T08:24:58.781 INFO:tasks.workunit.client.1.smithi149.stdout:2/658: sync 2024-04-24T08:24:58.784 INFO:tasks.workunit.client.0.smithi090.stdout:4/354: dwrite d3/d18/d1d/f1f [0,4194304] 0 2024-04-24T08:24:58.785 INFO:tasks.workunit.client.1.smithi149.stdout:2/659: truncate d0/f55 2489635 0 2024-04-24T08:24:58.785 INFO:tasks.workunit.client.1.smithi149.stdout:7/741: fdatasync d9/dc/d11/d83/da6/f71 0 2024-04-24T08:24:58.788 INFO:tasks.workunit.client.1.smithi149.stdout:4/684: write d2/d18/d1e/d3c/d48/d69/d7d/d8d/fb8 [2429717,55154] 0 2024-04-24T08:24:58.789 INFO:tasks.workunit.client.1.smithi149.stdout:8/660: dwrite d1/d1f/d22/f33 [4194304,4194304] 0 2024-04-24T08:24:58.789 INFO:tasks.workunit.client.0.smithi090.stdout:1/444: mkdir d3/d25/d30/d9d 0 2024-04-24T08:24:58.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/445: dread - d3/d1d/f86 zero size 2024-04-24T08:24:58.791 INFO:tasks.workunit.client.1.smithi149.stdout:4/685: dread - d2/d18/d1e/d3c/d48/d69/d7d/d8d/fd4 zero size 2024-04-24T08:24:58.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:58 smithi090 ceph-mon[25016]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:58.794 INFO:tasks.workunit.client.1.smithi149.stdout:4/686: chown d2/d18/d1e/d3c/d48/l83 18156 1 2024-04-24T08:24:58.797 INFO:tasks.workunit.client.1.smithi149.stdout:4/687: write d2/d18/d1e/d3b/f79 [395149,8995] 0 2024-04-24T08:24:58.800 INFO:tasks.workunit.client.0.smithi090.stdout:6/439: rename d8/d1d/f24 to d8/d1d/d2f/f83 0 2024-04-24T08:24:58.801 INFO:tasks.workunit.client.0.smithi090.stdout:5/490: write d8/f9 [529854,103211] 0 2024-04-24T08:24:58.801 INFO:tasks.workunit.client.1.smithi149.stdout:2/660: read - d0/d6/db/d27/d64/fa4 zero size 2024-04-24T08:24:58.807 INFO:tasks.workunit.client.0.smithi090.stdout:5/491: read d8/d25/f8f [691554,54085] 0 2024-04-24T08:24:58.808 INFO:tasks.workunit.client.0.smithi090.stdout:7/472: rmdir db/d17/d39/d86 39 2024-04-24T08:24:58.811 INFO:tasks.workunit.client.1.smithi149.stdout:7/742: fdatasync d9/dc/d2e/f6f 0 2024-04-24T08:24:58.812 INFO:tasks.workunit.client.0.smithi090.stdout:4/355: unlink d3/lc 0 2024-04-24T08:24:58.813 INFO:tasks.workunit.client.0.smithi090.stdout:5/492: read d8/da/d18/f22 [8314977,39935] 0 2024-04-24T08:24:58.815 INFO:tasks.workunit.client.1.smithi149.stdout:4/688: mknod d2/d18/d1e/cd7 0 2024-04-24T08:24:58.819 INFO:tasks.workunit.client.0.smithi090.stdout:7/473: dread db/d17/d29/f69 [0,4194304] 0 2024-04-24T08:24:58.820 INFO:tasks.workunit.client.0.smithi090.stdout:2/476: dread d0/f9e [0,4194304] 0 2024-04-24T08:24:58.821 INFO:tasks.workunit.client.0.smithi090.stdout:9/408: dwrite d0/dc/fd [0,4194304] 0 2024-04-24T08:24:58.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/440: creat d8/d1d/d2f/f84 x:0 0 0 2024-04-24T08:24:58.824 INFO:tasks.workunit.client.1.smithi149.stdout:7/743: creat d9/dc/d2e/d3c/ffa x:0 0 0 2024-04-24T08:24:58.826 INFO:tasks.workunit.client.0.smithi090.stdout:3/376: write d2/f2f [376274,58955] 0 2024-04-24T08:24:58.827 INFO:tasks.workunit.client.0.smithi090.stdout:6/441: read d8/d1d/d2f/d30/f52 [2135615,59782] 0 2024-04-24T08:24:58.828 INFO:tasks.workunit.client.1.smithi149.stdout:2/661: write d0/d6/db/f10 [1866874,46233] 0 2024-04-24T08:24:58.828 INFO:tasks.workunit.client.0.smithi090.stdout:3/377: readlink d2/df/d1a/l34 0 2024-04-24T08:24:58.830 INFO:tasks.workunit.client.1.smithi149.stdout:8/661: truncate d1/d1f/d5e/f73 287755 0 2024-04-24T08:24:58.834 INFO:tasks.workunit.client.1.smithi149.stdout:8/662: write d1/d1f/f50 [429339,48041] 0 2024-04-24T08:24:58.835 INFO:tasks.workunit.client.0.smithi090.stdout:9/409: symlink d0/d19/d33/d4d/l74 0 2024-04-24T08:24:58.837 INFO:tasks.workunit.client.0.smithi090.stdout:7/474: read db/d17/d39/f3c [51053,21354] 0 2024-04-24T08:24:58.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/356: dwrite d3/d18/d1d/d4b/f64 [0,4194304] 0 2024-04-24T08:24:58.840 INFO:tasks.workunit.client.1.smithi149.stdout:8/663: mknod d1/d1f/d31/cdf 0 2024-04-24T08:24:58.842 INFO:tasks.workunit.client.0.smithi090.stdout:2/477: dwrite d0/d7/d67/d7f/f5d [0,4194304] 0 2024-04-24T08:24:58.843 INFO:tasks.workunit.client.1.smithi149.stdout:8/664: fsync d1/de/d18/d32/d65/d8b/f7c 0 2024-04-24T08:24:58.846 INFO:tasks.workunit.client.0.smithi090.stdout:1/446: write d3/d5/f1a [4745038,60827] 0 2024-04-24T08:24:58.847 INFO:tasks.workunit.client.0.smithi090.stdout:1/447: chown d3/f8c 863475 1 2024-04-24T08:24:58.848 INFO:tasks.workunit.client.1.smithi149.stdout:4/689: dwrite d2/d18/d1e/d3c/d48/f87 [0,4194304] 0 2024-04-24T08:24:58.852 INFO:tasks.workunit.client.0.smithi090.stdout:6/442: fdatasync d8/dc/f3a 0 2024-04-24T08:24:58.852 INFO:tasks.workunit.client.0.smithi090.stdout:5/493: mknod d8/da/d3a/d2d/d37/d93/d7a/d50/d9f/ca4 0 2024-04-24T08:24:58.853 INFO:tasks.workunit.client.1.smithi149.stdout:2/662: rename d0/d6/db/c7d to d0/d6/db/d39/cc2 0 2024-04-24T08:24:58.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/378: creat d2/df/d1a/d37/d59/f7b x:0 0 0 2024-04-24T08:24:58.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/690: rmdir d2/d24/d2e/d38 39 2024-04-24T08:24:58.859 INFO:tasks.workunit.client.1.smithi149.stdout:8/665: creat d1/de/d18/d32/d65/d8b/d7d/ddc/fe0 x:0 0 0 2024-04-24T08:24:58.863 INFO:tasks.workunit.client.1.smithi149.stdout:2/663: rename d0/d6/db/d28/d7f/f87 to d0/d6/db/d27/fc3 0 2024-04-24T08:24:58.864 INFO:tasks.workunit.client.1.smithi149.stdout:8/666: chown d1/de/d18/d32/fca 14256 1 2024-04-24T08:24:58.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/357: rmdir d3/d18/d1d/d4b 39 2024-04-24T08:24:58.871 INFO:tasks.workunit.client.0.smithi090.stdout:3/379: dwrite d2/df/d1a/f2b [4194304,4194304] 0 2024-04-24T08:24:58.875 INFO:tasks.workunit.client.1.smithi149.stdout:4/691: rename d2/d18/d1e/d3b to d2/d18/db9/dd8 0 2024-04-24T08:24:58.878 INFO:tasks.workunit.client.1.smithi149.stdout:4/692: chown d2/d10/d1a/f3e 0 1 2024-04-24T08:24:58.880 INFO:tasks.workunit.client.0.smithi090.stdout:6/443: mkdir d8/d1d/d2f/d85 0 2024-04-24T08:24:58.883 INFO:tasks.workunit.client.0.smithi090.stdout:2/478: dwrite d0/d14/d17/f62 [0,4194304] 0 2024-04-24T08:24:58.887 INFO:tasks.workunit.client.1.smithi149.stdout:8/667: creat d1/de/d18/d32/d63/d76/fe1 x:0 0 0 2024-04-24T08:24:58.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/494: truncate d8/da/d3a/f55 716174 0 2024-04-24T08:24:58.892 INFO:tasks.workunit.client.1.smithi149.stdout:8/668: truncate d1/de/d18/d32/d63/d9e/fd6 279310 0 2024-04-24T08:24:58.895 INFO:tasks.workunit.client.1.smithi149.stdout:2/664: mknod d0/d6/db/d39/d3d/d58/d98/cc4 0 2024-04-24T08:24:58.895 INFO:tasks.workunit.client.0.smithi090.stdout:1/448: write d3/d5/da/dc/f40 [3743468,70989] 0 2024-04-24T08:24:58.896 INFO:tasks.workunit.client.0.smithi090.stdout:7/475: symlink db/d17/d29/d38/d3e/la1 0 2024-04-24T08:24:58.897 INFO:tasks.workunit.client.0.smithi090.stdout:1/449: stat d3/d5/da/dc 0 2024-04-24T08:24:58.899 INFO:tasks.workunit.client.0.smithi090.stdout:1/450: fsync d3/d5/da/dc/d80/f90 0 2024-04-24T08:24:58.899 INFO:tasks.workunit.client.0.smithi090.stdout:3/380: creat d2/d25/f7c x:0 0 0 2024-04-24T08:24:58.903 INFO:tasks.workunit.client.1.smithi149.stdout:8/669: link d1/de/d18/d43/f55 d1/de/d18/d32/d65/d8b/d7d/dd1/fe2 0 2024-04-24T08:24:58.904 INFO:tasks.workunit.client.1.smithi149.stdout:2/665: dread d0/d6/db/d27/fc3 [0,4194304] 0 2024-04-24T08:24:58.904 INFO:tasks.workunit.client.0.smithi090.stdout:6/444: chown d8/d1d/d1f/d3e/d4d/c6d 71 1 2024-04-24T08:24:58.906 INFO:tasks.workunit.client.1.smithi149.stdout:2/666: write d0/f4d [1287381,37257] 0 2024-04-24T08:24:58.907 INFO:tasks.workunit.client.1.smithi149.stdout:4/693: getdents d2/d10/d91/d9f 0 2024-04-24T08:24:58.908 INFO:tasks.workunit.client.1.smithi149.stdout:4/694: chown d2/l8 15935 1 2024-04-24T08:24:58.910 INFO:tasks.workunit.client.1.smithi149.stdout:8/670: symlink d1/de/d18/d32/d63/d9e/le3 0 2024-04-24T08:24:58.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/495: creat d8/da/d18/fa5 x:0 0 0 2024-04-24T08:24:58.911 INFO:tasks.workunit.client.0.smithi090.stdout:4/358: mkdir d3/df/d69 0 2024-04-24T08:24:58.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/381: mknod d2/df/d1a/d37/d4b/c7d 0 2024-04-24T08:24:58.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/410: getdents d0/dc/d24/d2d 0 2024-04-24T08:24:58.915 INFO:tasks.workunit.client.0.smithi090.stdout:4/359: fsync d3/d38/d49/f4f 0 2024-04-24T08:24:58.916 INFO:tasks.workunit.client.1.smithi149.stdout:4/695: getdents d2/d18/dd6 0 2024-04-24T08:24:58.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/671: read d1/de/d18/d32/d65/d8b/d7d/f96 [568149,123460] 0 2024-04-24T08:24:58.918 INFO:tasks.workunit.client.0.smithi090.stdout:1/451: creat d3/d68/d7f/f9e x:0 0 0 2024-04-24T08:24:58.920 INFO:tasks.workunit.client.0.smithi090.stdout:4/360: write d3/d18/d1d/d22/f51 [2468120,28572] 0 2024-04-24T08:24:58.923 INFO:tasks.workunit.client.0.smithi090.stdout:9/411: rename d0/l1 to d0/d19/d33/l75 0 2024-04-24T08:24:58.925 INFO:tasks.workunit.client.0.smithi090.stdout:5/496: dread d8/da/fc [0,4194304] 0 2024-04-24T08:24:58.925 INFO:tasks.workunit.client.1.smithi149.stdout:8/672: getdents d1/dce 0 2024-04-24T08:24:58.926 INFO:tasks.workunit.client.0.smithi090.stdout:3/382: mknod d2/d15/d36/d6f/d7a/c7e 0 2024-04-24T08:24:58.927 INFO:tasks.workunit.client.0.smithi090.stdout:9/412: symlink d0/d19/d33/l76 0 2024-04-24T08:24:58.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/445: dwrite d8/dc/d19/f64 [8388608,4194304] 0 2024-04-24T08:24:58.928 INFO:tasks.workunit.client.0.smithi090.stdout:5/497: stat d8/da/d3a/f43 0 2024-04-24T08:24:58.929 INFO:tasks.workunit.client.1.smithi149.stdout:8/673: write d1/f9 [9362959,24276] 0 2024-04-24T08:24:58.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/476: dread db/d17/d39/f3c [0,4194304] 0 2024-04-24T08:24:58.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/667: dread d0/d6/db/d39/f4e [0,4194304] 0 2024-04-24T08:24:58.933 INFO:tasks.workunit.client.1.smithi149.stdout:4/696: dread d2/d18/d1e/f2c [0,4194304] 0 2024-04-24T08:24:58.934 INFO:tasks.workunit.client.0.smithi090.stdout:1/452: dread d3/f4 [0,4194304] 0 2024-04-24T08:24:58.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/668: mkdir d0/d6/db/d39/d78/d8a/dc5 0 2024-04-24T08:24:58.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/446: rename d8/d1d/c4a to d8/d1d/d1f/d3e/d4f/d7e/c86 0 2024-04-24T08:24:58.939 INFO:tasks.workunit.client.1.smithi149.stdout:2/669: symlink d0/d6/db/d28/lc6 0 2024-04-24T08:24:58.941 INFO:tasks.workunit.client.1.smithi149.stdout:4/697: rename d2/d10/c11 to d2/d24/cd9 0 2024-04-24T08:24:58.944 INFO:tasks.workunit.client.1.smithi149.stdout:8/674: rename d1/de/d18/d32/d65/f81 to d1/de/d18/d32/d65/d8b/d7d/dd1/fe4 0 2024-04-24T08:24:58.944 INFO:tasks.workunit.client.1.smithi149.stdout:8/675: fsync d1/d1f/d31/fc3 0 2024-04-24T08:24:58.946 INFO:tasks.workunit.client.0.smithi090.stdout:6/447: fdatasync d8/dc/f3a 0 2024-04-24T08:24:58.947 INFO:tasks.workunit.client.1.smithi149.stdout:4/698: link d2/da8/fd3 d2/d10/d1a/fda 0 2024-04-24T08:24:58.949 INFO:tasks.workunit.client.1.smithi149.stdout:8/676: fsync d1/d1f/d25/f78 0 2024-04-24T08:24:58.951 INFO:tasks.workunit.client.0.smithi090.stdout:7/477: getdents db/d17/d29/d32/d5a 0 2024-04-24T08:24:58.951 INFO:tasks.workunit.client.0.smithi090.stdout:1/453: dread d3/d5/da/dc/f47 [0,4194304] 0 2024-04-24T08:24:58.952 INFO:tasks.workunit.client.1.smithi149.stdout:4/699: symlink d2/d18/d1e/d3c/d48/d69/ldb 0 2024-04-24T08:24:58.955 INFO:tasks.workunit.client.0.smithi090.stdout:7/478: creat db/d17/d39/fa2 x:0 0 0 2024-04-24T08:24:58.955 INFO:tasks.workunit.client.1.smithi149.stdout:4/700: getdents d2/d18/da4 0 2024-04-24T08:24:58.957 INFO:tasks.workunit.client.1.smithi149.stdout:4/701: readlink d2/d24/d2e/l8b 0 2024-04-24T08:24:58.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/479: stat db/d17/d29/d32/d5a/d75/c64 0 2024-04-24T08:24:58.959 INFO:tasks.workunit.client.1.smithi149.stdout:4/702: creat d2/d24/d2e/d3f/d94/fdc x:0 0 0 2024-04-24T08:24:58.960 INFO:tasks.workunit.client.0.smithi090.stdout:7/480: creat db/d17/d29/d32/fa3 x:0 0 0 2024-04-24T08:24:58.961 INFO:tasks.workunit.client.1.smithi149.stdout:4/703: mkdir d2/d10/d1a/dac/ddd 0 2024-04-24T08:24:58.962 INFO:tasks.workunit.client.0.smithi090.stdout:7/481: chown db/d17/d29/l33 24 1 2024-04-24T08:24:58.963 INFO:tasks.workunit.client.1.smithi149.stdout:4/704: chown d2/d10/d1a/l1c 9 1 2024-04-24T08:24:58.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/482: creat db/d17/d29/d32/d5a/d75/fa4 x:0 0 0 2024-04-24T08:24:58.967 INFO:tasks.workunit.client.1.smithi149.stdout:3/806: sync 2024-04-24T08:24:58.967 INFO:tasks.workunit.client.1.smithi149.stdout:0/670: sync 2024-04-24T08:24:58.969 INFO:tasks.workunit.client.0.smithi090.stdout:1/454: dread d3/d5/f17 [0,4194304] 0 2024-04-24T08:24:58.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/671: fdatasync d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fcb 0 2024-04-24T08:24:58.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/807: dread - d0/d5/d1a/d70/da3/f112 zero size 2024-04-24T08:24:58.974 INFO:tasks.workunit.client.0.smithi090.stdout:1/455: fdatasync d3/d1d/d34/f3b 0 2024-04-24T08:24:58.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/456: chown d3/d5/da/dc/d80/f90 7 1 2024-04-24T08:24:58.982 INFO:tasks.workunit.client.1.smithi149.stdout:3/808: unlink d0/d5/d53/d34/d3a/d93/f105 0 2024-04-24T08:24:58.985 INFO:tasks.workunit.client.0.smithi090.stdout:1/457: creat d3/d5/d58/f9f x:0 0 0 2024-04-24T08:24:58.987 INFO:tasks.workunit.client.1.smithi149.stdout:3/809: mknod d0/d5/da/d19/d1b/c113 0 2024-04-24T08:24:58.989 INFO:tasks.workunit.client.0.smithi090.stdout:1/458: mknod d3/d5/ca0 0 2024-04-24T08:24:58.989 INFO:tasks.workunit.client.0.smithi090.stdout:7/483: dwrite db/d17/d29/d32/d5a/d75/f70 [4194304,4194304] 0 2024-04-24T08:24:58.991 INFO:tasks.workunit.client.0.smithi090.stdout:1/459: read d3/d5/f17 [1114285,83415] 0 2024-04-24T08:24:58.991 INFO:tasks.workunit.client.0.smithi090.stdout:7/484: chown db/d17/d39/l4b 79384 1 2024-04-24T08:24:58.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/460: symlink d3/d5/da/d94/la1 0 2024-04-24T08:24:58.995 INFO:tasks.workunit.client.0.smithi090.stdout:7/485: mknod db/d17/d29/d38/d3e/ca5 0 2024-04-24T08:24:58.996 INFO:tasks.workunit.client.0.smithi090.stdout:2/479: write d0/d14/f34 [1824557,97856] 0 2024-04-24T08:24:58.997 INFO:tasks.workunit.client.0.smithi090.stdout:1/461: mkdir d3/d25/d30/d9d/da2 0 2024-04-24T08:24:58.999 INFO:tasks.workunit.client.0.smithi090.stdout:7/486: fsync db/d17/d29/f42 0 2024-04-24T08:24:59.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/480: symlink d0/d14/d4d/d50/l9f 0 2024-04-24T08:24:59.002 INFO:tasks.workunit.client.0.smithi090.stdout:7/487: mknod db/d90/ca6 0 2024-04-24T08:24:59.010 INFO:tasks.workunit.client.1.smithi149.stdout:3/810: dwrite d0/d26/d3c/db1/fd0 [4194304,4194304] 0 2024-04-24T08:24:59.012 INFO:tasks.workunit.client.0.smithi090.stdout:7/488: symlink db/d17/d39/d86/la7 0 2024-04-24T08:24:59.016 INFO:tasks.workunit.client.0.smithi090.stdout:9/413: write d0/d19/f25 [5010904,96954] 0 2024-04-24T08:24:59.017 INFO:tasks.workunit.client.0.smithi090.stdout:5/498: write d8/f10 [497010,107833] 0 2024-04-24T08:24:59.019 INFO:tasks.workunit.client.0.smithi090.stdout:9/414: chown d0/d19/d33/d4d/d32 1 1 2024-04-24T08:24:59.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/499: write f7 [528087,26419] 0 2024-04-24T08:24:59.021 INFO:tasks.workunit.client.0.smithi090.stdout:9/415: chown d0/d19/f56 111 1 2024-04-24T08:24:59.023 INFO:tasks.workunit.client.1.smithi149.stdout:3/811: creat d0/d5/d53/d4c/dca/dfb/f114 x:0 0 0 2024-04-24T08:24:59.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/489: mknod db/d54/ca8 0 2024-04-24T08:24:59.026 INFO:tasks.workunit.client.0.smithi090.stdout:3/383: dwrite d2/df/d1a/d37/d53/f74 [0,4194304] 0 2024-04-24T08:24:59.026 INFO:tasks.workunit.client.0.smithi090.stdout:5/500: chown d8/d56/f5f 4891 1 2024-04-24T08:24:59.027 INFO:tasks.workunit.client.1.smithi149.stdout:3/812: symlink d0/d5/da/d59/d79/l115 0 2024-04-24T08:24:59.030 INFO:tasks.workunit.client.1.smithi149.stdout:3/813: chown d0/d5/d53/d34/d3a/d93/fed 4188 1 2024-04-24T08:24:59.031 INFO:tasks.workunit.client.0.smithi090.stdout:5/501: dread - d8/da/d14/d2f/d44/f73 zero size 2024-04-24T08:24:59.031 INFO:tasks.workunit.client.0.smithi090.stdout:9/416: mkdir d0/d5e/d72/d77 0 2024-04-24T08:24:59.036 INFO:tasks.workunit.client.1.smithi149.stdout:3/814: unlink d0/d5/d53/d34/d3a/l31 0 2024-04-24T08:24:59.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/384: creat d2/df/d1a/d37/d53/d60/f7f x:0 0 0 2024-04-24T08:24:59.039 INFO:tasks.workunit.client.0.smithi090.stdout:7/490: link l5 db/d54/d78/la9 0 2024-04-24T08:24:59.041 INFO:tasks.workunit.client.0.smithi090.stdout:5/502: mkdir d8/da/d14/d2f/d81/da6 0 2024-04-24T08:24:59.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/448: dwrite d8/dc/d19/f27 [0,4194304] 0 2024-04-24T08:24:59.043 INFO:tasks.workunit.client.1.smithi149.stdout:3/815: unlink d0/d26/d3c/db1/ffc 0 2024-04-24T08:24:59.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/503: truncate d8/da/d3a/d2d/d37/d93/d7b/f80 648799 0 2024-04-24T08:24:59.048 INFO:tasks.workunit.client.0.smithi090.stdout:3/385: creat d2/d25/d33/f80 x:0 0 0 2024-04-24T08:24:59.048 INFO:tasks.workunit.client.0.smithi090.stdout:5/504: stat d8/d99 0 2024-04-24T08:24:59.051 INFO:tasks.workunit.client.0.smithi090.stdout:6/449: creat d8/d1d/d1f/d3e/d44/f87 x:0 0 0 2024-04-24T08:24:59.053 INFO:tasks.workunit.client.1.smithi149.stdout:0/672: write d2/d9/df/d40/f88 [840571,107608] 0 2024-04-24T08:24:59.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/386: creat d2/d25/d33/f81 x:0 0 0 2024-04-24T08:24:59.057 INFO:tasks.workunit.client.0.smithi090.stdout:7/491: mknod db/d17/d39/caa 0 2024-04-24T08:24:59.059 INFO:tasks.workunit.client.1.smithi149.stdout:4/705: dwrite d2/d24/f39 [0,4194304] 0 2024-04-24T08:24:59.062 INFO:tasks.workunit.client.1.smithi149.stdout:0/673: read d2/d9/df/d56/f80 [2933918,125692] 0 2024-04-24T08:24:59.063 INFO:tasks.workunit.client.1.smithi149.stdout:4/706: fsync d2/d18/db9/dd8/f79 0 2024-04-24T08:24:59.063 INFO:tasks.workunit.client.0.smithi090.stdout:6/450: creat d8/d1d/d2f/f88 x:0 0 0 2024-04-24T08:24:59.066 INFO:tasks.workunit.client.1.smithi149.stdout:3/816: dwrite d0/d5/da/d5e/f30 [0,4194304] 0 2024-04-24T08:24:59.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/462: truncate d3/d68/f8f 1247544 0 2024-04-24T08:24:59.068 INFO:tasks.workunit.client.0.smithi090.stdout:5/505: dwrite d8/da/d3a/d2d/d37/d93/d7a/d50/f79 [0,4194304] 0 2024-04-24T08:24:59.070 INFO:tasks.workunit.client.0.smithi090.stdout:2/481: truncate d0/d14/d17/f62 2123086 0 2024-04-24T08:24:59.071 INFO:tasks.workunit.client.1.smithi149.stdout:3/817: dread - d0/d5/da/d19/d1b/d72/f10d zero size 2024-04-24T08:24:59.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/451: dread d8/f1b [0,4194304] 0 2024-04-24T08:24:59.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/482: write d0/d5e/f9a [840457,17120] 0 2024-04-24T08:24:59.085 INFO:tasks.workunit.client.1.smithi149.stdout:4/707: dread d2/d24/d2e/f5b [0,4194304] 0 2024-04-24T08:24:59.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/674: mkdir d2/d9/d1a/d36/d85/dcd/dda 0 2024-04-24T08:24:59.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/452: dread d8/d1d/d29/f6c [0,4194304] 0 2024-04-24T08:24:59.090 INFO:tasks.workunit.client.1.smithi149.stdout:3/818: symlink d0/d5/da/d19/d80/dac/l116 0 2024-04-24T08:24:59.096 INFO:tasks.workunit.client.0.smithi090.stdout:2/483: mknod d0/d14/d4d/ca0 0 2024-04-24T08:24:59.096 INFO:tasks.workunit.client.1.smithi149.stdout:3/819: rmdir d0/d5/da 39 2024-04-24T08:24:59.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/417: dwrite d0/fb [0,4194304] 0 2024-04-24T08:24:59.098 INFO:tasks.workunit.client.1.smithi149.stdout:4/708: rename d2/d18/d1e/d3c/d48/c89 to d2/d18/d1e/d3c/d48/d69/cde 0 2024-04-24T08:24:59.101 INFO:tasks.workunit.client.1.smithi149.stdout:0/675: mknod d2/d9/d1a/cdb 0 2024-04-24T08:24:59.110 INFO:tasks.workunit.client.1.smithi149.stdout:0/676: stat d2/d9/d1a/c97 0 2024-04-24T08:24:59.111 INFO:tasks.workunit.client.0.smithi090.stdout:2/484: creat d0/d2c/fa1 x:0 0 0 2024-04-24T08:24:59.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/492: write db/d17/d29/d32/f3d [3832153,43894] 0 2024-04-24T08:24:59.112 INFO:tasks.workunit.client.1.smithi149.stdout:0/677: write d2/d17/f24 [96919,99285] 0 2024-04-24T08:24:59.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/418: mkdir d0/d5e/d78 0 2024-04-24T08:24:59.117 INFO:tasks.workunit.client.1.smithi149.stdout:4/709: mknod d2/d18/dd6/cdf 0 2024-04-24T08:24:59.117 INFO:tasks.workunit.client.0.smithi090.stdout:7/493: stat db/d17/d29/d32/fa3 0 2024-04-24T08:24:59.118 INFO:tasks.workunit.client.1.smithi149.stdout:4/710: write d2/f6c [1956890,39971] 0 2024-04-24T08:24:59.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/387: link d2/d15/c29 d2/c82 0 2024-04-24T08:24:59.124 INFO:tasks.workunit.client.0.smithi090.stdout:1/463: dwrite d3/d1d/d54/f67 [0,4194304] 0 2024-04-24T08:24:59.126 INFO:tasks.workunit.client.0.smithi090.stdout:5/506: dread d8/da/d3a/d2d/d37/d93/d7a/d50/f5e [0,4194304] 0 2024-04-24T08:24:59.128 INFO:tasks.workunit.client.1.smithi149.stdout:4/711: creat d2/d18/db9/dd8/fe0 x:0 0 0 2024-04-24T08:24:59.131 INFO:tasks.workunit.client.1.smithi149.stdout:0/678: rename d2/d9/df/d2f/dc9 to d2/d17/d41/d68/ddc 0 2024-04-24T08:24:59.132 INFO:tasks.workunit.client.1.smithi149.stdout:3/820: write d0/d5/da/f14 [617110,69704] 0 2024-04-24T08:24:59.133 INFO:tasks.workunit.client.0.smithi090.stdout:7/494: chown db/d17/c2a 3875 1 2024-04-24T08:24:59.136 INFO:tasks.workunit.client.0.smithi090.stdout:9/419: mknod d0/dc/d24/d2d/d49/c79 0 2024-04-24T08:24:59.137 INFO:tasks.workunit.client.0.smithi090.stdout:1/464: unlink d3/d1d/d54/f72 0 2024-04-24T08:24:59.140 INFO:tasks.workunit.client.1.smithi149.stdout:4/712: dread d2/d24/da7/fd0 [4194304,4194304] 0 2024-04-24T08:24:59.143 INFO:tasks.workunit.client.0.smithi090.stdout:6/453: dwrite d8/d1d/d2f/d30/f52 [0,4194304] 0 2024-04-24T08:24:59.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/507: dwrite d8/da/d3a/d2d/f82 [0,4194304] 0 2024-04-24T08:24:59.146 INFO:tasks.workunit.client.0.smithi090.stdout:7/495: mkdir db/dab 0 2024-04-24T08:24:59.149 INFO:tasks.workunit.client.0.smithi090.stdout:1/465: mkdir d3/d5/d3e/da3 0 2024-04-24T08:24:59.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/485: getdents d0/d7/d67/d7f 0 2024-04-24T08:24:59.152 INFO:tasks.workunit.client.0.smithi090.stdout:7/496: symlink db/d17/d39/d59/lac 0 2024-04-24T08:24:59.152 INFO:tasks.workunit.client.1.smithi149.stdout:0/679: dwrite d2/d17/fd1 [0,4194304] 0 2024-04-24T08:24:59.159 INFO:tasks.workunit.client.0.smithi090.stdout:2/486: write d0/d14/d17/f1b [3289962,56362] 0 2024-04-24T08:24:59.159 INFO:tasks.workunit.client.0.smithi090.stdout:3/388: write d2/df/d21/f30 [221755,124436] 0 2024-04-24T08:24:59.159 INFO:tasks.workunit.client.0.smithi090.stdout:9/420: mknod d0/d5e/d78/c7a 0 2024-04-24T08:24:59.159 INFO:tasks.workunit.client.0.smithi090.stdout:3/389: write d2/df/d1a/f2b [2831881,6] 0 2024-04-24T08:24:59.165 INFO:tasks.workunit.client.0.smithi090.stdout:3/390: creat d2/d15/d36/f83 x:0 0 0 2024-04-24T08:24:59.165 INFO:tasks.workunit.client.0.smithi090.stdout:3/391: chown d2/d15/d36/d6f/d7a 355 1 2024-04-24T08:24:59.166 INFO:tasks.workunit.client.0.smithi090.stdout:3/392: chown d2/d15/l4d 1107185062 1 2024-04-24T08:24:59.166 INFO:tasks.workunit.client.1.smithi149.stdout:0/680: rename d2/d9/df/d2f/d6c/f75 to d2/d17/d41/d68/ddc/fdd 0 2024-04-24T08:24:59.174 INFO:tasks.workunit.client.1.smithi149.stdout:3/821: dread d0/d5/da/d59/d79/f9e [0,4194304] 0 2024-04-24T08:24:59.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/793: sync 2024-04-24T08:24:59.174 INFO:tasks.workunit.client.1.smithi149.stdout:1/839: sync 2024-04-24T08:24:59.176 INFO:tasks.workunit.client.1.smithi149.stdout:0/681: fsync d2/fa 0 2024-04-24T08:24:59.180 INFO:tasks.workunit.client.1.smithi149.stdout:9/794: unlink d1/d2/d15/d1c/f20 0 2024-04-24T08:24:59.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/795: chown d1/da/dd/d79/f56 209587367 1 2024-04-24T08:24:59.183 INFO:tasks.workunit.client.1.smithi149.stdout:1/840: unlink d0/d5/df/f94 0 2024-04-24T08:24:59.184 INFO:tasks.workunit.client.1.smithi149.stdout:9/796: read d1/d2/d15/f23 [142521,42208] 0 2024-04-24T08:24:59.186 INFO:tasks.workunit.client.1.smithi149.stdout:1/841: rmdir d0/d1b/db5/d108 39 2024-04-24T08:24:59.191 INFO:tasks.workunit.client.1.smithi149.stdout:1/842: rename d0/d5/db/f72 to d0/d1b/d2a/f115 0 2024-04-24T08:24:59.193 INFO:tasks.workunit.client.1.smithi149.stdout:1/843: write d0/ddf/da0/db8/dec/f100 [755540,42921] 0 2024-04-24T08:24:59.196 INFO:tasks.workunit.client.1.smithi149.stdout:1/844: creat d0/d5/db/d18/d4d/de8/f116 x:0 0 0 2024-04-24T08:24:59.199 INFO:tasks.workunit.client.1.smithi149.stdout:1/845: rmdir d0/ddf 39 2024-04-24T08:24:59.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/797: dread d1/de/f18 [0,4194304] 0 2024-04-24T08:24:59.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/798: stat d1/de/f18 0 2024-04-24T08:24:59.206 INFO:tasks.workunit.client.1.smithi149.stdout:9/799: fsync d1/da/d13/f1e 0 2024-04-24T08:24:59.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/800: readlink d1/da/d13/d74/lf8 0 2024-04-24T08:24:59.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/393: rmdir d2/df/d1a/d37/d59 39 2024-04-24T08:24:59.211 INFO:tasks.workunit.client.1.smithi149.stdout:9/801: mknod d1/da/d13/d74/dbb/dc0/c11f 0 2024-04-24T08:24:59.213 INFO:tasks.workunit.client.1.smithi149.stdout:9/802: creat d1/d2/d33/f120 x:0 0 0 2024-04-24T08:24:59.216 INFO:tasks.workunit.client.1.smithi149.stdout:9/803: truncate d1/da/d13/d74/dbb/fcd 1296279 0 2024-04-24T08:24:59.219 INFO:tasks.workunit.client.1.smithi149.stdout:4/713: dread d2/d10/d1a/da2/fa3 [0,4194304] 0 2024-04-24T08:24:59.221 INFO:tasks.workunit.client.0.smithi090.stdout:5/508: write d8/da/d18/f22 [6537391,7690] 0 2024-04-24T08:24:59.221 INFO:tasks.workunit.client.0.smithi090.stdout:3/394: dwrite d2/d15/d36/f83 [0,4194304] 0 2024-04-24T08:24:59.221 INFO:tasks.workunit.client.1.smithi149.stdout:4/714: mknod d2/d18/d1e/d3c/d48/ce1 0 2024-04-24T08:24:59.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/454: getdents d8/d1d/d1f/d3e/d4f 0 2024-04-24T08:24:59.226 INFO:tasks.workunit.client.1.smithi149.stdout:4/715: truncate d2/d10/f31 2618280 0 2024-04-24T08:24:59.229 INFO:tasks.workunit.client.0.smithi090.stdout:9/421: link d0/d5e/d78/c7a d0/d19/c7b 0 2024-04-24T08:24:59.229 INFO:tasks.workunit.client.0.smithi090.stdout:4/361: sync 2024-04-24T08:24:59.229 INFO:tasks.workunit.client.0.smithi090.stdout:0/468: sync 2024-04-24T08:24:59.229 INFO:tasks.workunit.client.0.smithi090.stdout:8/385: sync 2024-04-24T08:24:59.229 INFO:tasks.workunit.client.0.smithi090.stdout:5/509: mknod d8/d7c/ca7 0 2024-04-24T08:24:59.231 INFO:tasks.workunit.client.0.smithi090.stdout:0/469: fdatasync d1a/f1e 0 2024-04-24T08:24:59.232 INFO:tasks.workunit.client.0.smithi090.stdout:9/422: stat d0/dc/f58 0 2024-04-24T08:24:59.234 INFO:tasks.workunit.client.0.smithi090.stdout:6/455: creat d8/d1d/d1f/f89 x:0 0 0 2024-04-24T08:24:59.236 INFO:tasks.workunit.client.0.smithi090.stdout:3/395: mkdir d2/d84 0 2024-04-24T08:24:59.240 INFO:tasks.workunit.client.0.smithi090.stdout:0/470: chown l10 2563727 1 2024-04-24T08:24:59.240 INFO:tasks.workunit.client.0.smithi090.stdout:4/362: write d3/d38/f60 [1960036,19928] 0 2024-04-24T08:24:59.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/466: write d3/f2e [752806,32560] 0 2024-04-24T08:24:59.241 INFO:tasks.workunit.client.0.smithi090.stdout:6/456: read - d8/d1d/f7c zero size 2024-04-24T08:24:59.243 INFO:tasks.workunit.client.1.smithi149.stdout:3/822: dread d0/d5/da/d5a/d9c/fef [0,4194304] 0 2024-04-24T08:24:59.244 INFO:tasks.workunit.client.0.smithi090.stdout:8/386: dread d2/d62/f6e [0,4194304] 0 2024-04-24T08:24:59.251 INFO:tasks.workunit.client.1.smithi149.stdout:3/823: mkdir d0/d5/da/d19/d80/d117 0 2024-04-24T08:24:59.258 INFO:tasks.workunit.client.0.smithi090.stdout:4/363: dwrite d3/df/f68 [0,4194304] 0 2024-04-24T08:24:59.258 INFO:tasks.workunit.client.0.smithi090.stdout:8/387: chown d2/da/d13/l33 27067183 1 2024-04-24T08:24:59.260 INFO:tasks.workunit.client.0.smithi090.stdout:4/364: read - d3/d38/d49/f63 zero size 2024-04-24T08:24:59.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/467: truncate d3/d5/da/dc/f27 967146 0 2024-04-24T08:24:59.264 INFO:tasks.workunit.client.1.smithi149.stdout:3/824: symlink d0/d5/da/d19/d1b/d22/dc6/l118 0 2024-04-24T08:24:59.267 INFO:tasks.workunit.client.0.smithi090.stdout:3/396: creat d2/df/d1a/d37/f85 x:0 0 0 2024-04-24T08:24:59.270 INFO:tasks.workunit.client.1.smithi149.stdout:5/855: sync 2024-04-24T08:24:59.271 INFO:tasks.workunit.client.1.smithi149.stdout:9/804: write d1/d2/d33/fb2 [852724,123286] 0 2024-04-24T08:24:59.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/682: dread d2/fa [0,4194304] 0 2024-04-24T08:24:59.275 INFO:tasks.workunit.client.0.smithi090.stdout:8/388: chown d2/cb 0 1 2024-04-24T08:24:59.278 INFO:tasks.workunit.client.1.smithi149.stdout:3/825: rename d0/d5/d1a/fcc to d0/d5/d53/d34/d3a/d93/f119 0 2024-04-24T08:24:59.279 INFO:tasks.workunit.client.1.smithi149.stdout:1/846: dwrite d0/d1b/fef [0,4194304] 0 2024-04-24T08:24:59.283 INFO:tasks.workunit.client.1.smithi149.stdout:4/716: write d2/d18/db9/dd8/f42 [3803467,42137] 0 2024-04-24T08:24:59.288 INFO:tasks.workunit.client.0.smithi090.stdout:9/423: rename d0/d19/d33 to d0/dc/d24/d7c 0 2024-04-24T08:24:59.290 INFO:tasks.workunit.client.0.smithi090.stdout:0/471: getdents d1a 0 2024-04-24T08:24:59.290 INFO:tasks.workunit.client.1.smithi149.stdout:0/683: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fde x:0 0 0 2024-04-24T08:24:59.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/847: rmdir d0/d5/d4f 39 2024-04-24T08:24:59.294 INFO:tasks.workunit.client.0.smithi090.stdout:0/472: chown d1a/d4b/d53/d79/d35/f3e 418 1 2024-04-24T08:24:59.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/389: rmdir d2/da/d13 39 2024-04-24T08:24:59.294 INFO:tasks.workunit.client.0.smithi090.stdout:6/457: write d8/d1d/d1f/d3e/d4f/d7e/f80 [107329,114251] 0 2024-04-24T08:24:59.297 INFO:tasks.workunit.client.0.smithi090.stdout:6/458: stat d8/f16 0 2024-04-24T08:24:59.298 INFO:tasks.workunit.client.0.smithi090.stdout:7/497: dread db/d17/d39/f49 [0,4194304] 0 2024-04-24T08:24:59.302 INFO:tasks.workunit.client.1.smithi149.stdout:0/684: mknod d2/d9/df/d2f/d6c/d72/d91/cdf 0 2024-04-24T08:24:59.302 INFO:tasks.workunit.client.1.smithi149.stdout:4/717: mknod d2/d18/da4/ce2 0 2024-04-24T08:24:59.306 INFO:tasks.workunit.client.1.smithi149.stdout:5/856: dwrite db/d12/d2f/d95/fe5 [0,4194304] 0 2024-04-24T08:24:59.306 INFO:tasks.workunit.client.1.smithi149.stdout:1/848: dread d0/d5/db/d82/f9b [0,4194304] 0 2024-04-24T08:24:59.308 INFO:tasks.workunit.client.1.smithi149.stdout:5/857: chown db/d12/d2f/d95/fdc 43665 1 2024-04-24T08:24:59.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/365: dwrite d3/d18/d1d/d22/f62 [0,4194304] 0 2024-04-24T08:24:59.312 INFO:tasks.workunit.client.0.smithi090.stdout:4/366: chown d3/d24/f44 55 1 2024-04-24T08:24:59.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/367: fdatasync d3/d18/d1d/f4e 0 2024-04-24T08:24:59.316 INFO:tasks.workunit.client.1.smithi149.stdout:3/826: link d0/d5/da/d5e/c75 d0/d26/d3c/db1/deb/c11a 0 2024-04-24T08:24:59.321 INFO:tasks.workunit.client.1.smithi149.stdout:2/670: sync 2024-04-24T08:24:59.321 INFO:tasks.workunit.client.1.smithi149.stdout:7/744: sync 2024-04-24T08:24:59.321 INFO:tasks.workunit.client.1.smithi149.stdout:8/677: sync 2024-04-24T08:24:59.321 INFO:tasks.workunit.client.1.smithi149.stdout:6/665: sync 2024-04-24T08:24:59.322 INFO:tasks.workunit.client.0.smithi090.stdout:3/397: dwrite d2/df/d21/f32 [0,4194304] 0 2024-04-24T08:24:59.325 INFO:tasks.workunit.client.0.smithi090.stdout:3/398: dread - d2/df/d1a/d37/d53/f6b zero size 2024-04-24T08:24:59.326 INFO:tasks.workunit.client.1.smithi149.stdout:9/805: symlink d1/d2/d15/de7/l121 0 2024-04-24T08:24:59.326 INFO:tasks.workunit.client.0.smithi090.stdout:3/399: chown d2/f2f 12 1 2024-04-24T08:24:59.327 INFO:tasks.workunit.client.1.smithi149.stdout:8/678: dread - d1/d1f/d31/fbb zero size 2024-04-24T08:24:59.327 INFO:tasks.workunit.client.1.smithi149.stdout:4/718: dread d2/d18/f55 [0,4194304] 0 2024-04-24T08:24:59.329 INFO:tasks.workunit.client.1.smithi149.stdout:5/858: stat db/d12/l75 0 2024-04-24T08:24:59.331 INFO:tasks.workunit.client.1.smithi149.stdout:7/745: dread - d9/dc/d11/d83/da6/ff6 zero size 2024-04-24T08:24:59.335 INFO:tasks.workunit.client.1.smithi149.stdout:5/859: fdatasync db/d12/d1e/d65/de8/ff5 0 2024-04-24T08:24:59.339 INFO:tasks.workunit.client.1.smithi149.stdout:5/860: stat db/d12/f82 0 2024-04-24T08:24:59.341 INFO:tasks.workunit.client.1.smithi149.stdout:1/849: creat d0/d5/dce/f117 x:0 0 0 2024-04-24T08:24:59.342 INFO:tasks.workunit.client.1.smithi149.stdout:5/861: chown db/d23/cd5 2686768 1 2024-04-24T08:24:59.345 INFO:tasks.workunit.client.1.smithi149.stdout:0/685: dread d2/d9/d69/fa7 [0,4194304] 0 2024-04-24T08:24:59.352 INFO:tasks.workunit.client.1.smithi149.stdout:2/671: creat d0/d6/db/d27/d81/fc7 x:0 0 0 2024-04-24T08:24:59.358 INFO:tasks.workunit.client.1.smithi149.stdout:2/672: read d0/f2 [146581,102679] 0 2024-04-24T08:24:59.361 INFO:tasks.workunit.client.1.smithi149.stdout:4/719: symlink d2/le3 0 2024-04-24T08:24:59.361 INFO:tasks.workunit.client.1.smithi149.stdout:8/679: dwrite d1/de/d18/d32/d65/d8b/f7c [0,4194304] 0 2024-04-24T08:24:59.361 INFO:tasks.workunit.client.0.smithi090.stdout:0/473: mkdir d1a/d4b/d53/d79/d69/d78/d9b 0 2024-04-24T08:24:59.363 INFO:tasks.workunit.client.0.smithi090.stdout:0/474: fsync d1a/d4b/d53/d79/d69/d78/f92 0 2024-04-24T08:24:59.364 INFO:tasks.workunit.client.1.smithi149.stdout:7/746: mknod d9/d43/d4f/da5/cfb 0 2024-04-24T08:24:59.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/475: chown d1a/f3f 12112 1 2024-04-24T08:24:59.366 INFO:tasks.workunit.client.0.smithi090.stdout:4/368: dread d3/d24/f2b [0,4194304] 0 2024-04-24T08:24:59.371 INFO:tasks.workunit.client.0.smithi090.stdout:2/487: sync 2024-04-24T08:24:59.376 INFO:tasks.workunit.client.1.smithi149.stdout:8/680: dread - d1/de/d18/d91/fc5 zero size 2024-04-24T08:24:59.376 INFO:tasks.workunit.client.1.smithi149.stdout:7/747: chown d9/dc/d11/d84/da7 137 1 2024-04-24T08:24:59.378 INFO:tasks.workunit.client.1.smithi149.stdout:5/862: chown db/d12/d22/d45/l55 451 1 2024-04-24T08:24:59.381 INFO:tasks.workunit.client.0.smithi090.stdout:0/476: dwrite d1a/f9a [0,4194304] 0 2024-04-24T08:24:59.381 INFO:tasks.workunit.client.1.smithi149.stdout:3/827: dread d0/d5/da/d19/d1b/d22/d38/f50 [0,4194304] 0 2024-04-24T08:24:59.382 INFO:tasks.workunit.client.0.smithi090.stdout:3/400: write d2/d15/d36/f5f [297911,118019] 0 2024-04-24T08:24:59.385 INFO:tasks.workunit.client.1.smithi149.stdout:3/828: stat d0/d5/da/d5a/d63/d6a/dd9 0 2024-04-24T08:24:59.387 INFO:tasks.workunit.client.1.smithi149.stdout:1/850: dwrite d0/d1b/db5/d108/f10a [0,4194304] 0 2024-04-24T08:24:59.390 INFO:tasks.workunit.client.0.smithi090.stdout:6/459: mkdir d8/d1d/d2f/d30/d81/d8a 0 2024-04-24T08:24:59.393 INFO:tasks.workunit.client.0.smithi090.stdout:4/369: creat d3/d24/f6a x:0 0 0 2024-04-24T08:24:59.398 INFO:tasks.workunit.client.0.smithi090.stdout:2/488: creat d0/d14/d4d/d50/d96/fa2 x:0 0 0 2024-04-24T08:24:59.403 INFO:tasks.workunit.client.0.smithi090.stdout:1/468: link d3/d1d/d54/l62 d3/d1d/la4 0 2024-04-24T08:24:59.408 INFO:tasks.workunit.client.1.smithi149.stdout:0/686: dread d2/d9/f2d [0,4194304] 0 2024-04-24T08:24:59.408 INFO:tasks.workunit.client.0.smithi090.stdout:3/401: dread - d2/d25/f50 zero size 2024-04-24T08:24:59.408 INFO:tasks.workunit.client.0.smithi090.stdout:5/510: rename d8/da/d3a/d2d/c5a to d8/da/d3a/d2d/d37/d93/d7a/d50/ca8 0 2024-04-24T08:24:59.409 INFO:tasks.workunit.client.0.smithi090.stdout:1/469: write d3/d68/d7f/f9e [978517,108891] 0 2024-04-24T08:24:59.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/470: write d3/f8c [90546,72378] 0 2024-04-24T08:24:59.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/370: dwrite d3/df/f68 [4194304,4194304] 0 2024-04-24T08:24:59.419 INFO:tasks.workunit.client.0.smithi090.stdout:0/477: symlink d1a/d4b/l9c 0 2024-04-24T08:24:59.421 INFO:tasks.workunit.client.0.smithi090.stdout:5/511: mkdir d8/d7c/da9 0 2024-04-24T08:24:59.422 INFO:tasks.workunit.client.0.smithi090.stdout:5/512: chown d8/da/d18/fa0 3 1 2024-04-24T08:24:59.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/498: getdents db/d17/d39/d86 0 2024-04-24T08:24:59.432 INFO:tasks.workunit.client.0.smithi090.stdout:9/424: rename d0/dc/d24/d3d to d0/d69/d7d 0 2024-04-24T08:24:59.433 INFO:tasks.workunit.client.0.smithi090.stdout:8/390: rmdir d2/da/d13/d63 0 2024-04-24T08:24:59.435 INFO:tasks.workunit.client.0.smithi090.stdout:0/478: mkdir d1a/d4b/d53/d79/d82/d9d 0 2024-04-24T08:24:59.439 INFO:tasks.workunit.client.0.smithi090.stdout:3/402: dread d2/d15/f16 [0,4194304] 0 2024-04-24T08:24:59.440 INFO:tasks.workunit.client.0.smithi090.stdout:0/479: write d1a/d4b/d53/f8b [883017,16974] 0 2024-04-24T08:24:59.442 INFO:tasks.workunit.client.0.smithi090.stdout:5/513: symlink d8/d7c/da9/laa 0 2024-04-24T08:24:59.442 INFO:tasks.workunit.client.0.smithi090.stdout:3/403: readlink d2/d15/l4d 0 2024-04-24T08:24:59.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/471: creat d3/d5/fa5 x:0 0 0 2024-04-24T08:24:59.443 INFO:tasks.workunit.client.1.smithi149.stdout:6/666: rename d0/d1a/d28/d31/d33/c86 to d0/d1a/d28/d31/d4c/d4e/cdf 0 2024-04-24T08:24:59.445 INFO:tasks.workunit.client.0.smithi090.stdout:1/472: dread - d3/d5/f8d zero size 2024-04-24T08:24:59.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/460: dwrite d8/dc/f72 [0,4194304] 0 2024-04-24T08:24:59.446 INFO:tasks.workunit.client.1.smithi149.stdout:2/673: fdatasync d0/d6/db/d39/d3d/d44/d75/f79 0 2024-04-24T08:24:59.449 INFO:tasks.workunit.client.0.smithi090.stdout:2/489: link d0/d7/d28/l69 d0/d5e/la3 0 2024-04-24T08:24:59.453 INFO:tasks.workunit.client.0.smithi090.stdout:4/371: rename d3/d18/d1d/d4b to d3/d38/d6b 0 2024-04-24T08:24:59.454 INFO:tasks.workunit.client.0.smithi090.stdout:9/425: unlink d0/dc/d24/l63 0 2024-04-24T08:24:59.456 INFO:tasks.workunit.client.0.smithi090.stdout:9/426: truncate d0/f59 744621 0 2024-04-24T08:24:59.457 INFO:tasks.workunit.client.0.smithi090.stdout:8/391: dwrite d2/da/d13/d64/f16 [0,4194304] 0 2024-04-24T08:24:59.460 INFO:tasks.workunit.client.0.smithi090.stdout:9/427: write d0/d19/f25 [4412849,108894] 0 2024-04-24T08:24:59.462 INFO:tasks.workunit.client.0.smithi090.stdout:9/428: write d0/dc/fd [3647354,17352] 0 2024-04-24T08:24:59.471 INFO:tasks.workunit.client.0.smithi090.stdout:1/473: mkdir d3/d5/da/d94/da6 0 2024-04-24T08:24:59.471 INFO:tasks.workunit.client.0.smithi090.stdout:1/474: readlink d3/d5/l1c 0 2024-04-24T08:24:59.471 INFO:tasks.workunit.client.0.smithi090.stdout:1/475: dread - d3/d5/da/dc/f6c zero size 2024-04-24T08:24:59.473 INFO:tasks.workunit.client.1.smithi149.stdout:0/687: dread - d2/d9/d1a/d36/d85/fa4 zero size 2024-04-24T08:24:59.474 INFO:tasks.workunit.client.0.smithi090.stdout:8/392: dread d2/da/d13/d64/f16 [0,4194304] 0 2024-04-24T08:24:59.477 INFO:tasks.workunit.client.1.smithi149.stdout:5/863: dwrite db/da4/fa8 [0,4194304] 0 2024-04-24T08:24:59.477 INFO:tasks.workunit.client.0.smithi090.stdout:5/514: dread d8/f31 [0,4194304] 0 2024-04-24T08:24:59.477 INFO:tasks.workunit.client.0.smithi090.stdout:2/490: creat d0/d7/d67/d7f/d31/d46/d60/fa4 x:0 0 0 2024-04-24T08:24:59.478 INFO:tasks.workunit.client.1.smithi149.stdout:9/806: symlink d1/d2/d15/d1c/l122 0 2024-04-24T08:24:59.485 INFO:tasks.workunit.client.0.smithi090.stdout:0/480: dwrite d1a/d4b/d53/d79/d35/f42 [0,4194304] 0 2024-04-24T08:24:59.485 INFO:tasks.workunit.client.0.smithi090.stdout:3/404: fdatasync d2/fa 0 2024-04-24T08:24:59.485 INFO:tasks.workunit.client.1.smithi149.stdout:7/748: dread d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:24:59.487 INFO:tasks.workunit.client.1.smithi149.stdout:8/681: mkdir d1/de/d18/d32/d65/de5 0 2024-04-24T08:24:59.488 INFO:tasks.workunit.client.0.smithi090.stdout:7/499: write db/d17/d29/d32/f40 [636713,99963] 0 2024-04-24T08:24:59.489 INFO:tasks.workunit.client.1.smithi149.stdout:4/720: symlink d2/d18/d1e/d3c/d48/le4 0 2024-04-24T08:24:59.492 INFO:tasks.workunit.client.0.smithi090.stdout:1/476: dwrite d3/d1d/d34/d5a/f5e [4194304,4194304] 0 2024-04-24T08:24:59.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/429: rmdir d0/d69/d7d 39 2024-04-24T08:24:59.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/461: dwrite d8/d1d/d1f/f58 [4194304,4194304] 0 2024-04-24T08:24:59.498 INFO:tasks.workunit.client.0.smithi090.stdout:9/430: chown d0/c60 0 1 2024-04-24T08:24:59.500 INFO:tasks.workunit.client.0.smithi090.stdout:6/462: chown d8/d1d/d29/f61 2100043903 1 2024-04-24T08:24:59.503 INFO:tasks.workunit.client.0.smithi090.stdout:2/491: mknod d0/d2e/ca5 0 2024-04-24T08:24:59.510 INFO:tasks.workunit.client.1.smithi149.stdout:2/674: symlink d0/d6/db/d39/d3d/lc8 0 2024-04-24T08:24:59.512 INFO:tasks.workunit.client.0.smithi090.stdout:0/481: unlink d1a/l22 0 2024-04-24T08:24:59.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/721: creat d2/d18/d1e/d3c/fe5 x:0 0 0 2024-04-24T08:24:59.523 INFO:tasks.workunit.client.1.smithi149.stdout:3/829: write d0/d5/d53/f3f [5219805,107765] 0 2024-04-24T08:24:59.528 INFO:tasks.workunit.client.0.smithi090.stdout:6/463: dread d8/fa [4194304,4194304] 0 2024-04-24T08:24:59.529 INFO:tasks.workunit.client.1.smithi149.stdout:5/864: dread db/d12/d60/db2/f115 [0,4194304] 0 2024-04-24T08:24:59.529 INFO:tasks.workunit.client.0.smithi090.stdout:7/500: fdatasync db/f21 0 2024-04-24T08:24:59.536 INFO:tasks.workunit.client.0.smithi090.stdout:5/515: dread d8/da/d14/d2f/d44/f5b [0,4194304] 0 2024-04-24T08:24:59.537 INFO:tasks.workunit.client.1.smithi149.stdout:5/865: write db/d12/d1e/d85/fa5 [4135481,28668] 0 2024-04-24T08:24:59.537 INFO:tasks.workunit.client.0.smithi090.stdout:5/516: chown d8/d25/f3d 21012948 1 2024-04-24T08:24:59.539 INFO:tasks.workunit.client.1.smithi149.stdout:7/749: dread d9/fd [0,4194304] 0 2024-04-24T08:24:59.540 INFO:tasks.workunit.client.0.smithi090.stdout:7/501: dread f7 [4194304,4194304] 0 2024-04-24T08:24:59.550 INFO:tasks.workunit.client.1.smithi149.stdout:8/682: dread d1/de/d18/f7e [0,4194304] 0 2024-04-24T08:24:59.552 INFO:tasks.workunit.client.1.smithi149.stdout:4/722: mkdir d2/d18/dd6/de6 0 2024-04-24T08:24:59.552 INFO:tasks.workunit.client.1.smithi149.stdout:8/683: chown d1/de/d18/d32/d65/de5 2 1 2024-04-24T08:24:59.556 INFO:tasks.workunit.client.1.smithi149.stdout:1/851: link d0/d5/db/d18/d4d/d50/fd8 d0/d5/db/d18/d4d/d50/f118 0 2024-04-24T08:24:59.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/517: dwrite d8/da/d3a/d2d/d37/d93/d7a/d50/f71 [4194304,4194304] 0 2024-04-24T08:24:59.560 INFO:tasks.workunit.client.0.smithi090.stdout:3/405: unlink d2/d15/c29 0 2024-04-24T08:24:59.561 INFO:tasks.workunit.client.0.smithi090.stdout:5/518: dread - d8/da/d14/d2f/d81/f98 zero size 2024-04-24T08:24:59.562 INFO:tasks.workunit.client.0.smithi090.stdout:5/519: chown f7 769373 1 2024-04-24T08:24:59.563 INFO:tasks.workunit.client.1.smithi149.stdout:5/866: mknod db/d12/d60/c11f 0 2024-04-24T08:24:59.574 INFO:tasks.workunit.client.1.smithi149.stdout:5/867: read db/d12/d1e/fc1 [775381,83139] 0 2024-04-24T08:24:59.576 INFO:tasks.workunit.client.1.smithi149.stdout:9/807: link d1/da/f3e d1/d2/f123 0 2024-04-24T08:24:59.579 INFO:tasks.workunit.client.0.smithi090.stdout:6/464: read - d8/d1d/d2f/d30/f54 zero size 2024-04-24T08:24:59.580 INFO:tasks.workunit.client.1.smithi149.stdout:5/868: stat db/d12/d60/d8f/ce0 0 2024-04-24T08:24:59.580 INFO:tasks.workunit.client.1.smithi149.stdout:0/688: dwrite d2/d9/f13 [4194304,4194304] 0 2024-04-24T08:24:59.580 INFO:tasks.workunit.client.1.smithi149.stdout:6/667: link d0/d1a/d28/d31/d4c/d4e/lbb d0/le0 0 2024-04-24T08:24:59.581 INFO:tasks.workunit.client.1.smithi149.stdout:2/675: mkdir d0/d6/db/d27/d64/d66/db1/dc9 0 2024-04-24T08:24:59.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/372: getdents d3/d18/d1d 0 2024-04-24T08:24:59.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/373: chown d3/d18/d1d/d22/c28 221 1 2024-04-24T08:24:59.582 INFO:tasks.workunit.client.0.smithi090.stdout:2/492: dwrite d0/d7/d67/d7f/d31/f41 [4194304,4194304] 0 2024-04-24T08:24:59.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/374: fsync d3/d38/d6b/f64 0 2024-04-24T08:24:59.584 INFO:tasks.workunit.client.0.smithi090.stdout:7/502: creat db/d17/d29/d38/d99/fad x:0 0 0 2024-04-24T08:24:59.585 INFO:tasks.workunit.client.1.smithi149.stdout:8/684: mknod d1/de/d18/d32/d63/d9e/ce6 0 2024-04-24T08:24:59.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/503: dread - db/d17/d39/d86/f92 zero size 2024-04-24T08:24:59.587 INFO:tasks.workunit.client.0.smithi090.stdout:9/431: mknod d0/c7e 0 2024-04-24T08:24:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:8/685: chown d1/de/d18/d91/fc5 1566555 1 2024-04-24T08:24:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:4/723: symlink d2/d18/d1e/d3c/d48/le7 0 2024-04-24T08:24:59.588 INFO:tasks.workunit.client.0.smithi090.stdout:7/504: chown db/d54/d83/f8b 3734539 1 2024-04-24T08:24:59.589 INFO:tasks.workunit.client.1.smithi149.stdout:1/852: creat d0/d1b/d5d/dd6/f119 x:0 0 0 2024-04-24T08:24:59.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/393: creat d2/da/d13/f84 x:0 0 0 2024-04-24T08:24:59.597 INFO:tasks.workunit.client.0.smithi090.stdout:0/482: chown d1a/d4b/d53/d79/d35/d68/l77 37927898 1 2024-04-24T08:24:59.597 INFO:tasks.workunit.client.1.smithi149.stdout:9/808: chown d1/d2/d15/f2f 7329 1 2024-04-24T08:24:59.597 INFO:tasks.workunit.client.1.smithi149.stdout:5/869: mkdir db/da4/d120 0 2024-04-24T08:24:59.601 INFO:tasks.workunit.client.1.smithi149.stdout:7/750: dwrite d9/d43/d64/fc5 [0,4194304] 0 2024-04-24T08:24:59.604 INFO:tasks.workunit.client.0.smithi090.stdout:5/520: creat d8/da/d18/fab x:0 0 0 2024-04-24T08:24:59.604 INFO:tasks.workunit.client.1.smithi149.stdout:6/668: creat d0/d1a/d28/d31/d4c/d76/fe1 x:0 0 0 2024-04-24T08:24:59.606 INFO:tasks.workunit.client.0.smithi090.stdout:5/521: stat d8/da/d18 0 2024-04-24T08:24:59.610 INFO:tasks.workunit.client.0.smithi090.stdout:1/477: dwrite d3/d5/da/dc/f27 [0,4194304] 0 2024-04-24T08:24:59.613 INFO:tasks.workunit.client.0.smithi090.stdout:2/493: symlink d0/d14/la6 0 2024-04-24T08:24:59.626 INFO:tasks.workunit.client.1.smithi149.stdout:1/853: creat d0/d5/db/d18/f11a x:0 0 0 2024-04-24T08:24:59.627 INFO:tasks.workunit.client.0.smithi090.stdout:4/375: mknod d3/d38/d6b/c6c 0 2024-04-24T08:24:59.629 INFO:tasks.workunit.client.1.smithi149.stdout:5/870: mkdir db/d12/d1e/d85/d92/da6/d121 0 2024-04-24T08:24:59.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/505: mkdir db/d17/d39/d86/dae 0 2024-04-24T08:24:59.632 INFO:tasks.workunit.client.1.smithi149.stdout:3/830: getdents d0/d5/da/d5e 0 2024-04-24T08:24:59.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/394: rename d2/da/d13/c73 to d2/d62/d6b/c85 0 2024-04-24T08:24:59.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/669: stat d0/d1a/f32 0 2024-04-24T08:24:59.633 INFO:tasks.workunit.client.0.smithi090.stdout:0/483: dread - d1a/d4b/f61 zero size 2024-04-24T08:24:59.633 INFO:tasks.workunit.client.1.smithi149.stdout:7/751: truncate d9/dc/d11/f6c 1297118 0 2024-04-24T08:24:59.634 INFO:tasks.workunit.client.1.smithi149.stdout:3/831: read d0/d5/da/f14 [146849,13203] 0 2024-04-24T08:24:59.635 INFO:tasks.workunit.client.0.smithi090.stdout:5/522: mknod d8/da/d14/d2f/d44/cac 0 2024-04-24T08:24:59.636 INFO:tasks.workunit.client.1.smithi149.stdout:7/752: write d9/dc/d11/fef [577794,30664] 0 2024-04-24T08:24:59.637 INFO:tasks.workunit.client.1.smithi149.stdout:8/686: rename d1/f4f to d1/de/d18/d32/d65/d8b/dc9/fe7 0 2024-04-24T08:24:59.637 INFO:tasks.workunit.client.1.smithi149.stdout:3/832: readlink d0/d5/da/d19/d1b/d22/d38/l4a 0 2024-04-24T08:24:59.641 INFO:tasks.workunit.client.0.smithi090.stdout:6/465: symlink d8/dc/l8b 0 2024-04-24T08:24:59.641 INFO:tasks.workunit.client.1.smithi149.stdout:0/689: write d2/d9/df/d2f/d6c/d72/fb1 [6313262,103684] 0 2024-04-24T08:24:59.645 INFO:tasks.workunit.client.0.smithi090.stdout:4/376: write d3/d18/d5c/f3d [2977509,67783] 0 2024-04-24T08:24:59.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/406: rename d2/df/d21/f32 to d2/df/d57/d4f/f86 0 2024-04-24T08:24:59.649 INFO:tasks.workunit.client.1.smithi149.stdout:4/724: dwrite d2/d10/d91/fc4 [0,4194304] 0 2024-04-24T08:24:59.649 INFO:tasks.workunit.client.0.smithi090.stdout:8/395: fsync d2/da/d13/d34/f3d 0 2024-04-24T08:24:59.651 INFO:tasks.workunit.client.1.smithi149.stdout:7/753: mkdir d9/dc/d7b/dfc 0 2024-04-24T08:24:59.652 INFO:tasks.workunit.client.0.smithi090.stdout:0/484: fsync d1a/f56 0 2024-04-24T08:24:59.652 INFO:tasks.workunit.client.1.smithi149.stdout:6/670: read d0/f26 [3208807,2863] 0 2024-04-24T08:24:59.653 INFO:tasks.workunit.client.0.smithi090.stdout:0/485: stat d1a/d4b/d53/d79/f32 0 2024-04-24T08:24:59.655 INFO:tasks.workunit.client.0.smithi090.stdout:5/523: unlink d8/da/d14/d2f/d44/cac 0 2024-04-24T08:24:59.660 INFO:tasks.workunit.client.1.smithi149.stdout:1/854: creat d0/ddf/d53/d61/f11b x:0 0 0 2024-04-24T08:24:59.661 INFO:tasks.workunit.client.0.smithi090.stdout:6/466: mkdir d8/d1d/d1f/d8c 0 2024-04-24T08:24:59.662 INFO:tasks.workunit.client.0.smithi090.stdout:6/467: readlink d8/d1d/l37 0 2024-04-24T08:24:59.664 INFO:tasks.workunit.client.0.smithi090.stdout:6/468: dread - d8/d1d/d1f/f89 zero size 2024-04-24T08:24:59.666 INFO:tasks.workunit.client.0.smithi090.stdout:2/494: mkdir d0/da7 0 2024-04-24T08:24:59.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/469: truncate d8/dc/d19/f3d 4780118 0 2024-04-24T08:24:59.667 INFO:tasks.workunit.client.1.smithi149.stdout:6/671: dread d0/d1a/d28/d31/d33/f9b [0,4194304] 0 2024-04-24T08:24:59.676 INFO:tasks.workunit.client.1.smithi149.stdout:3/833: write d0/d5/d53/d34/d3a/d93/f77 [810712,12776] 0 2024-04-24T08:24:59.676 INFO:tasks.workunit.client.0.smithi090.stdout:7/506: rename db/f34 to db/d90/faf 0 2024-04-24T08:24:59.680 INFO:tasks.workunit.client.0.smithi090.stdout:0/486: readlink d1a/l2d 0 2024-04-24T08:24:59.682 INFO:tasks.workunit.client.0.smithi090.stdout:1/478: creat d3/d1d/fa7 x:0 0 0 2024-04-24T08:24:59.682 INFO:tasks.workunit.client.1.smithi149.stdout:9/809: link d1/da/dd/d55/d72/f7b d1/da/dd/d79/db6/d115/f124 0 2024-04-24T08:24:59.683 INFO:tasks.workunit.client.1.smithi149.stdout:7/754: dread d9/dc/d11/d83/da6/d67/f9e [0,4194304] 0 2024-04-24T08:24:59.685 INFO:tasks.workunit.client.0.smithi090.stdout:2/495: chown d0/d14/l20 1494398259 1 2024-04-24T08:24:59.687 INFO:tasks.workunit.client.0.smithi090.stdout:6/470: chown d8/dc/l2b 7 1 2024-04-24T08:24:59.688 INFO:tasks.workunit.client.0.smithi090.stdout:9/432: link d0/dc/d24/d7c/d4d/c67 d0/c7f 0 2024-04-24T08:24:59.690 INFO:tasks.workunit.client.0.smithi090.stdout:6/471: write d8/d1d/d1f/d3e/d44/f87 [424738,100784] 0 2024-04-24T08:24:59.690 INFO:tasks.workunit.client.1.smithi149.stdout:0/690: creat d2/d9/d1a/d36/d85/dcd/dda/fe0 x:0 0 0 2024-04-24T08:24:59.692 INFO:tasks.workunit.client.0.smithi090.stdout:4/377: link d3/f9 d3/d38/d6b/f6d 0 2024-04-24T08:24:59.692 INFO:tasks.workunit.client.0.smithi090.stdout:7/507: fsync db/d17/d39/f97 0 2024-04-24T08:24:59.694 INFO:tasks.workunit.client.1.smithi149.stdout:4/725: creat d2/d24/d2e/d3f/d94/fe8 x:0 0 0 2024-04-24T08:24:59.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/407: mkdir d2/df/d1a/d37/d59/d87 0 2024-04-24T08:24:59.696 INFO:tasks.workunit.client.1.smithi149.stdout:1/855: dwrite d0/d1b/d5d/de5/f111 [4194304,4194304] 0 2024-04-24T08:24:59.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/524: mkdir d8/d76/dad 0 2024-04-24T08:24:59.697 INFO:tasks.workunit.client.0.smithi090.stdout:3/408: write d2/df/d1a/d37/d59/f5e [644668,9554] 0 2024-04-24T08:24:59.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/479: creat d3/d25/d30/d9d/fa8 x:0 0 0 2024-04-24T08:24:59.700 INFO:tasks.workunit.client.0.smithi090.stdout:1/480: stat d3/d25/c8a 0 2024-04-24T08:24:59.702 INFO:tasks.workunit.client.0.smithi090.stdout:9/433: creat d0/dc/d24/d2d/f80 x:0 0 0 2024-04-24T08:24:59.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/434: chown d0/l9 234295510 1 2024-04-24T08:24:59.703 INFO:tasks.workunit.client.0.smithi090.stdout:6/472: creat d8/d1d/d29/f8d x:0 0 0 2024-04-24T08:24:59.704 INFO:tasks.workunit.client.0.smithi090.stdout:1/481: read d3/d5/da/dc/f40 [3172526,5467] 0 2024-04-24T08:24:59.710 INFO:tasks.workunit.client.0.smithi090.stdout:3/409: symlink d2/d15/l88 0 2024-04-24T08:24:59.713 INFO:tasks.workunit.client.0.smithi090.stdout:2/496: fdatasync d0/d7/f95 0 2024-04-24T08:24:59.716 INFO:tasks.workunit.client.0.smithi090.stdout:7/508: creat db/d17/d39/d86/dae/fb0 x:0 0 0 2024-04-24T08:24:59.719 INFO:tasks.workunit.client.0.smithi090.stdout:5/525: mkdir d8/da/d3a/d2d/d37/d93/d7a/da1/dae 0 2024-04-24T08:24:59.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/410: symlink d2/d25/d33/l89 0 2024-04-24T08:24:59.721 INFO:tasks.workunit.client.0.smithi090.stdout:4/378: link d3/c5e d3/d18/d1d/d22/c6e 0 2024-04-24T08:24:59.721 INFO:tasks.workunit.client.0.smithi090.stdout:2/497: rmdir d0/d7/d67/d7f 39 2024-04-24T08:24:59.722 INFO:tasks.workunit.client.0.smithi090.stdout:0/487: link d1a/c31 d1a/d4b/d53/d79/d35/d62/c9e 0 2024-04-24T08:24:59.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/435: dwrite d0/fb [4194304,4194304] 0 2024-04-24T08:24:59.724 INFO:tasks.workunit.client.0.smithi090.stdout:3/411: chown d2/df/d1a/d37/d59/f7b 9003413 1 2024-04-24T08:24:59.726 INFO:tasks.workunit.client.0.smithi090.stdout:2/498: chown d0/d7/f11 141593118 1 2024-04-24T08:24:59.727 INFO:tasks.workunit.client.0.smithi090.stdout:1/482: write d3/d5/f33 [1024650,80485] 0 2024-04-24T08:24:59.727 INFO:tasks.workunit.client.0.smithi090.stdout:6/473: dwrite d8/d1d/d2f/d30/f52 [4194304,4194304] 0 2024-04-24T08:24:59.728 INFO:tasks.workunit.client.0.smithi090.stdout:1/483: stat d3/l96 0 2024-04-24T08:24:59.729 INFO:tasks.workunit.client.0.smithi090.stdout:9/436: stat d0/d69/d7d/f5c 0 2024-04-24T08:24:59.732 INFO:tasks.workunit.client.0.smithi090.stdout:7/509: mkdir db/d54/db1 0 2024-04-24T08:24:59.732 INFO:tasks.workunit.client.0.smithi090.stdout:2/499: rmdir d0/d7/d57 39 2024-04-24T08:24:59.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/437: chown d0/c6e 13264 1 2024-04-24T08:24:59.737 INFO:tasks.workunit.client.0.smithi090.stdout:7/510: readlink db/d17/d29/d32/l82 0 2024-04-24T08:24:59.737 INFO:tasks.workunit.client.0.smithi090.stdout:5/526: creat d8/faf x:0 0 0 2024-04-24T08:24:59.740 INFO:tasks.workunit.client.0.smithi090.stdout:0/488: rename d1a/l1f to d1a/d4b/d53/d79/d35/d8e/l9f 0 2024-04-24T08:24:59.742 INFO:tasks.workunit.client.0.smithi090.stdout:4/379: dwrite d3/d38/f4d [0,4194304] 0 2024-04-24T08:24:59.743 INFO:tasks.workunit.client.0.smithi090.stdout:1/484: truncate d3/d68/f8f 535507 0 2024-04-24T08:24:59.744 INFO:tasks.workunit.client.0.smithi090.stdout:4/380: read d3/d38/d6b/f64 [765302,80090] 0 2024-04-24T08:24:59.745 INFO:tasks.workunit.client.0.smithi090.stdout:9/438: mkdir d0/dc/d24/d2d/d49/d81 0 2024-04-24T08:24:59.760 INFO:tasks.workunit.client.0.smithi090.stdout:2/500: rename d0/d7/l47 to d0/d7/d67/d7f/d31/la8 0 2024-04-24T08:24:59.761 INFO:tasks.workunit.client.0.smithi090.stdout:7/511: getdents db/dab 0 2024-04-24T08:24:59.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/485: creat d3/d5/da/d97/d9c/fa9 x:0 0 0 2024-04-24T08:24:59.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/486: mknod d3/d1d/d54/caa 0 2024-04-24T08:24:59.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/487: stat d3/d5/da/dc/f27 0 2024-04-24T08:24:59.761 INFO:tasks.workunit.client.0.smithi090.stdout:4/381: dread d3/d18/d1d/d22/f62 [0,4194304] 0 2024-04-24T08:24:59.767 INFO:tasks.workunit.client.0.smithi090.stdout:1/488: rename d3/d5/f1a to d3/d5/da/dc/fab 0 2024-04-24T08:24:59.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/755: fdatasync d9/dc/d11/d83/da6/f71 0 2024-04-24T08:24:59.772 INFO:tasks.workunit.client.0.smithi090.stdout:3/412: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:24:59.772 INFO:tasks.workunit.client.1.smithi149.stdout:4/726: mkdir d2/d18/d1e/d3c/d48/d69/d7d/de9 0 2024-04-24T08:24:59.772 INFO:tasks.workunit.client.1.smithi149.stdout:8/687: link d1/d1f/d22/l5c d1/d1f/d5e/le8 0 2024-04-24T08:24:59.773 INFO:tasks.workunit.client.0.smithi090.stdout:6/474: dwrite d8/dc/f11 [0,4194304] 0 2024-04-24T08:24:59.773 INFO:tasks.workunit.client.1.smithi149.stdout:3/834: truncate d0/d5/d53/d34/f35 3284894 0 2024-04-24T08:24:59.774 INFO:tasks.workunit.client.1.smithi149.stdout:1/856: chown d0/d5/db/d18/d4d/d50/c52 606783 1 2024-04-24T08:24:59.775 INFO:tasks.workunit.client.0.smithi090.stdout:3/413: truncate d2/df/d57/d4f/f72 13585 0 2024-04-24T08:24:59.775 INFO:tasks.workunit.client.0.smithi090.stdout:0/489: read d1a/d4b/d53/d79/d35/d68/f76 [31024,33181] 0 2024-04-24T08:24:59.775 INFO:tasks.workunit.client.0.smithi090.stdout:4/382: rename d3/d18/d1d/d22 to d3/d38/d49/d6f 0 2024-04-24T08:24:59.776 INFO:tasks.workunit.client.1.smithi149.stdout:5/871: link db/d23/daa/cf9 db/d12/d22/c122 0 2024-04-24T08:24:59.776 INFO:tasks.workunit.client.0.smithi090.stdout:1/489: mknod d3/cac 0 2024-04-24T08:24:59.791 INFO:tasks.workunit.client.1.smithi149.stdout:3/835: chown d0/d5/da/d5a/d9c/cce 25 1 2024-04-24T08:24:59.792 INFO:tasks.workunit.client.0.smithi090.stdout:7/512: write db/d17/d39/f65 [3519809,111306] 0 2024-04-24T08:24:59.793 INFO:tasks.workunit.client.1.smithi149.stdout:7/756: rmdir d9/d43/d4f 39 2024-04-24T08:24:59.793 INFO:tasks.workunit.client.0.smithi090.stdout:9/439: write d0/dc/d24/d7c/d4d/d32/f3c [714261,105133] 0 2024-04-24T08:24:59.793 INFO:tasks.workunit.client.0.smithi090.stdout:2/501: truncate d0/d2e/f37 691835 0 2024-04-24T08:24:59.794 INFO:tasks.workunit.client.0.smithi090.stdout:6/475: dread - d8/d1d/d1f/f5d zero size 2024-04-24T08:24:59.794 INFO:tasks.workunit.client.0.smithi090.stdout:0/490: dread d1a/d4b/d53/f8b [0,4194304] 0 2024-04-24T08:24:59.794 INFO:tasks.workunit.client.1.smithi149.stdout:4/727: symlink d2/d10/d91/lea 0 2024-04-24T08:24:59.794 INFO:tasks.workunit.client.1.smithi149.stdout:7/757: dread - d9/dc/d11/d83/da6/d67/d69/dad/fc6 zero size 2024-04-24T08:24:59.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:59 smithi090 ceph-mon[25016]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:24:59.795 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:24:59 smithi090 ceph-mon[25016]: pgmap v22: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 72 MiB/s rd, 259 MiB/s wr, 429 op/s 2024-04-24T08:24:59.799 INFO:tasks.workunit.client.0.smithi090.stdout:7/513: rmdir db/d17/d29/d32 39 2024-04-24T08:24:59.800 INFO:tasks.workunit.client.1.smithi149.stdout:1/857: mknod d0/d5/db/c11c 0 2024-04-24T08:24:59.800 INFO:tasks.workunit.client.0.smithi090.stdout:7/514: stat db/d17/d39/caa 0 2024-04-24T08:24:59.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/872: mkdir db/d35/d123 0 2024-04-24T08:24:59.804 INFO:tasks.workunit.client.0.smithi090.stdout:2/502: rename d0/d7/d6f/l5b to d0/d14/d17/la9 0 2024-04-24T08:24:59.806 INFO:tasks.workunit.client.1.smithi149.stdout:3/836: mknod d0/d5/da/d19/c11b 0 2024-04-24T08:24:59.821 INFO:tasks.workunit.client.1.smithi149.stdout:7/758: dwrite d9/dc/d11/d83/da6/d67/d69/fcf [0,4194304] 0 2024-04-24T08:24:59.821 INFO:tasks.workunit.client.1.smithi149.stdout:8/688: mkdir d1/dce/de9 0 2024-04-24T08:24:59.821 INFO:tasks.workunit.client.0.smithi090.stdout:1/490: creat d3/d25/d30/d9d/da2/fad x:0 0 0 2024-04-24T08:24:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:5/873: symlink db/d35/l124 0 2024-04-24T08:24:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/691: dread d2/d9/df/d40/fa2 [0,4194304] 0 2024-04-24T08:24:59.826 INFO:tasks.workunit.client.1.smithi149.stdout:5/874: stat db/d12/d1e/d85/d92/da6/fcf 0 2024-04-24T08:24:59.827 INFO:tasks.workunit.client.1.smithi149.stdout:1/858: creat d0/d1b/d5d/de5/d101/f11d x:0 0 0 2024-04-24T08:24:59.828 INFO:tasks.workunit.client.1.smithi149.stdout:6/672: link d0/d1a/d28/d31/d4c/d80/c78 d0/d1a/d28/d31/d4c/d4e/ce2 0 2024-04-24T08:24:59.830 INFO:tasks.workunit.client.0.smithi090.stdout:2/503: rmdir d0/d7/d67/d7f/d31/d46 39 2024-04-24T08:24:59.839 INFO:tasks.workunit.client.1.smithi149.stdout:8/689: rmdir d1/de/d18/d32 39 2024-04-24T08:24:59.841 INFO:tasks.workunit.client.0.smithi090.stdout:3/414: creat d2/df/d1a/d37/f8a x:0 0 0 2024-04-24T08:24:59.841 INFO:tasks.workunit.client.1.smithi149.stdout:4/728: dwrite d2/d10/fa6 [0,4194304] 0 2024-04-24T08:24:59.846 INFO:tasks.workunit.client.0.smithi090.stdout:7/515: mknod db/d17/d29/d38/cb2 0 2024-04-24T08:24:59.850 INFO:tasks.workunit.client.0.smithi090.stdout:1/491: symlink d3/d5/da/dc/d80/lae 0 2024-04-24T08:24:59.850 INFO:tasks.workunit.client.1.smithi149.stdout:1/859: write d0/ddf/d53/d61/f70 [4335118,128051] 0 2024-04-24T08:24:59.853 INFO:tasks.workunit.client.1.smithi149.stdout:3/837: dread d0/d5/d53/d39/f8a [0,4194304] 0 2024-04-24T08:24:59.854 INFO:tasks.workunit.client.0.smithi090.stdout:2/504: unlink d0/d14/d4d/d50/l9f 0 2024-04-24T08:24:59.855 INFO:tasks.workunit.client.0.smithi090.stdout:0/491: creat d1a/d4b/fa0 x:0 0 0 2024-04-24T08:24:59.856 INFO:tasks.workunit.client.1.smithi149.stdout:0/692: symlink d2/d9/d1a/d36/dcc/le1 0 2024-04-24T08:24:59.857 INFO:tasks.workunit.client.1.smithi149.stdout:7/759: dwrite d9/dc/d11/d83/da6/d67/f9e [0,4194304] 0 2024-04-24T08:24:59.861 INFO:tasks.workunit.client.0.smithi090.stdout:3/415: rmdir d2/d15/d36 39 2024-04-24T08:24:59.861 INFO:tasks.workunit.client.1.smithi149.stdout:7/760: chown d9/dc/d11/d38/fb4 1 1 2024-04-24T08:24:59.861 INFO:tasks.workunit.client.1.smithi149.stdout:8/690: dwrite d1/f9 [4194304,4194304] 0 2024-04-24T08:24:59.861 INFO:tasks.workunit.client.0.smithi090.stdout:3/416: dread - d2/df/d1a/d37/f8a zero size 2024-04-24T08:24:59.863 INFO:tasks.workunit.client.0.smithi090.stdout:7/516: write db/d17/d29/d32/d5a/d75/f70 [656591,116107] 0 2024-04-24T08:24:59.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/492: dwrite d3/d1d/d54/f67 [0,4194304] 0 2024-04-24T08:24:59.872 INFO:tasks.workunit.client.1.smithi149.stdout:1/860: fdatasync d0/d5/db/f92 0 2024-04-24T08:24:59.878 INFO:tasks.workunit.client.1.smithi149.stdout:0/693: unlink d2/d9/lb8 0 2024-04-24T08:24:59.879 INFO:tasks.workunit.client.1.smithi149.stdout:3/838: mkdir d0/d5/da/d5a/d9c/d11c 0 2024-04-24T08:24:59.883 INFO:tasks.workunit.client.1.smithi149.stdout:6/673: link d0/d1a/d28/d31/d4c/d8a/fb5 d0/d1a/d28/d31/d4c/d80/d6f/d97/fe3 0 2024-04-24T08:24:59.888 INFO:tasks.workunit.client.1.smithi149.stdout:4/729: write d2/d10/f17 [1472486,69789] 0 2024-04-24T08:24:59.889 INFO:tasks.workunit.client.1.smithi149.stdout:0/694: creat d2/d9/d1a/d36/d85/fe2 x:0 0 0 2024-04-24T08:24:59.896 INFO:tasks.workunit.client.1.smithi149.stdout:5/875: rename db/d12/d2f/d74/dc6/cf7 to db/d12/d22/d45/c125 0 2024-04-24T08:24:59.897 INFO:tasks.workunit.client.1.smithi149.stdout:6/674: rmdir d0/d1a/d28/d31/d4c/d80/d6f/dc2 39 2024-04-24T08:24:59.901 INFO:tasks.workunit.client.1.smithi149.stdout:0/695: fdatasync d2/d17/fb9 0 2024-04-24T08:24:59.904 INFO:tasks.workunit.client.1.smithi149.stdout:8/691: dread d1/d1f/f2c [0,4194304] 0 2024-04-24T08:24:59.907 INFO:tasks.workunit.client.1.smithi149.stdout:3/839: write d0/d5/d53/d4c/fdb [914148,128495] 0 2024-04-24T08:24:59.911 INFO:tasks.workunit.client.1.smithi149.stdout:1/861: dread d0/d5/df/fbf [0,4194304] 0 2024-04-24T08:24:59.914 INFO:tasks.workunit.client.1.smithi149.stdout:4/730: dwrite d2/d18/d1e/d3c/fe5 [0,4194304] 0 2024-04-24T08:24:59.920 INFO:tasks.workunit.client.1.smithi149.stdout:4/731: dread - d2/d18/d1e/fcd zero size 2024-04-24T08:24:59.923 INFO:tasks.workunit.client.1.smithi149.stdout:3/840: creat d0/d5/da/d5a/d63/d6a/f11d x:0 0 0 2024-04-24T08:24:59.927 INFO:tasks.workunit.client.1.smithi149.stdout:7/761: dwrite d9/dc/f18 [0,4194304] 0 2024-04-24T08:24:59.934 INFO:tasks.workunit.client.1.smithi149.stdout:9/810: sync 2024-04-24T08:24:59.934 INFO:tasks.workunit.client.1.smithi149.stdout:2/676: sync 2024-04-24T08:24:59.937 INFO:tasks.workunit.client.1.smithi149.stdout:0/696: dwrite d2/d9/df/d2f/d6c/f9a [0,4194304] 0 2024-04-24T08:24:59.943 INFO:tasks.workunit.client.1.smithi149.stdout:5/876: dwrite db/d12/d22/d45/d5d/f63 [0,4194304] 0 2024-04-24T08:24:59.943 INFO:tasks.workunit.client.0.smithi090.stdout:6/476: rename d8/d1d/d2f/d30 to d8/dc/d19/d8e 0 2024-04-24T08:24:59.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/417: write d2/df/d1a/d37/d59/f61 [1627420,53170] 0 2024-04-24T08:24:59.945 INFO:tasks.workunit.client.0.smithi090.stdout:2/505: mkdir d0/d7/d28/daa 0 2024-04-24T08:24:59.950 INFO:tasks.workunit.client.0.smithi090.stdout:1/493: chown c2 2 1 2024-04-24T08:24:59.954 INFO:tasks.workunit.client.0.smithi090.stdout:4/383: dread d3/d18/d1d/f27 [0,4194304] 0 2024-04-24T08:24:59.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/517: dwrite db/fc [0,4194304] 0 2024-04-24T08:24:59.971 INFO:tasks.workunit.client.0.smithi090.stdout:0/492: dwrite d1a/f30 [8388608,4194304] 0 2024-04-24T08:24:59.973 INFO:tasks.workunit.client.1.smithi149.stdout:7/762: dread d9/dc/de/d44/fab [0,4194304] 0 2024-04-24T08:24:59.975 INFO:tasks.workunit.client.0.smithi090.stdout:0/493: chown d1a/d4b/d53/d79/d35/l50 152729 1 2024-04-24T08:24:59.975 INFO:tasks.workunit.client.0.smithi090.stdout:7/518: dwrite db/d17/d39/d86/f92 [0,4194304] 0 2024-04-24T08:24:59.977 INFO:tasks.workunit.client.0.smithi090.stdout:0/494: read - d1a/d4b/d53/d79/d58/f93 zero size 2024-04-24T08:24:59.978 INFO:tasks.workunit.client.0.smithi090.stdout:0/495: chown d1a/d4b/d53/d57 528333 1 2024-04-24T08:25:00.003 INFO:tasks.workunit.client.0.smithi090.stdout:1/494: readlink d3/d1d/la4 0 2024-04-24T08:25:00.004 INFO:tasks.workunit.client.0.smithi090.stdout:1/495: chown d3/d5/da/c3c 1063557156 1 2024-04-24T08:25:00.006 INFO:tasks.workunit.client.1.smithi149.stdout:1/862: write d0/fbd [1125619,80296] 0 2024-04-24T08:25:00.006 INFO:tasks.workunit.client.1.smithi149.stdout:3/841: creat d0/d5/da/d19/d76/f11e x:0 0 0 2024-04-24T08:25:00.008 INFO:tasks.workunit.client.0.smithi090.stdout:4/384: dread d3/df/f30 [0,4194304] 0 2024-04-24T08:25:00.010 INFO:tasks.workunit.client.1.smithi149.stdout:0/697: unlink d2/d9/df/f16 0 2024-04-24T08:25:00.013 INFO:tasks.workunit.client.1.smithi149.stdout:9/811: symlink d1/da/d13/d74/d11a/l125 0 2024-04-24T08:25:00.016 INFO:tasks.workunit.client.0.smithi090.stdout:6/477: mkdir d8/d1d/d1f/d3e/d8f 0 2024-04-24T08:25:00.016 INFO:tasks.workunit.client.1.smithi149.stdout:5/877: truncate db/d12/d1e/d85/d92/d107/feb 458694 0 2024-04-24T08:25:00.017 INFO:tasks.workunit.client.0.smithi090.stdout:3/418: write d2/df/f19 [128479,65925] 0 2024-04-24T08:25:00.017 INFO:tasks.workunit.client.0.smithi090.stdout:2/506: symlink d0/d7/d67/d7f/d31/d46/d60/lab 0 2024-04-24T08:25:00.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/763: chown d9/dc/d11/d38/f4e 324 1 2024-04-24T08:25:00.020 INFO:tasks.workunit.client.1.smithi149.stdout:7/764: chown d9 51545180 1 2024-04-24T08:25:00.020 INFO:tasks.workunit.client.0.smithi090.stdout:2/507: read d0/d7/d67/d7f/d31/f56 [486839,72395] 0 2024-04-24T08:25:00.021 INFO:tasks.workunit.client.0.smithi090.stdout:2/508: readlink d0/la 0 2024-04-24T08:25:00.022 INFO:tasks.workunit.client.0.smithi090.stdout:1/496: creat d3/d25/faf x:0 0 0 2024-04-24T08:25:00.023 INFO:tasks.workunit.client.1.smithi149.stdout:1/863: mknod d0/d1b/d5d/c11e 0 2024-04-24T08:25:00.025 INFO:tasks.workunit.client.1.smithi149.stdout:3/842: fdatasync d0/d5/da/d19/d1b/d72/fbb 0 2024-04-24T08:25:00.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/519: creat db/dab/fb3 x:0 0 0 2024-04-24T08:25:00.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:59 smithi149 ceph-mon[31282]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:00.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:24:59 smithi149 ceph-mon[31282]: pgmap v22: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 72 MiB/s rd, 259 MiB/s wr, 429 op/s 2024-04-24T08:25:00.028 INFO:tasks.workunit.client.1.smithi149.stdout:1/864: truncate d0/ddf/da0/db8/dc5/f113 194729 0 2024-04-24T08:25:00.028 INFO:tasks.workunit.client.1.smithi149.stdout:4/732: write d2/d24/d2e/d38/f7a [3385993,76986] 0 2024-04-24T08:25:00.030 INFO:tasks.workunit.client.1.smithi149.stdout:2/677: mknod d0/d6/db/d39/d3d/cca 0 2024-04-24T08:25:00.031 INFO:tasks.workunit.client.0.smithi090.stdout:6/478: mkdir d8/d1d/d1f/d3e/d44/d90 0 2024-04-24T08:25:00.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/419: rename d2/d25/c35 to d2/df/d21/c8b 0 2024-04-24T08:25:00.037 INFO:tasks.workunit.client.0.smithi090.stdout:2/509: symlink d0/d5e/lac 0 2024-04-24T08:25:00.038 INFO:tasks.workunit.client.1.smithi149.stdout:9/812: rename d1/da/dd/d55 to d1/d2/d33/dea/d126 0 2024-04-24T08:25:00.039 INFO:tasks.workunit.client.1.smithi149.stdout:7/765: rename d9/dc/d11/d38/fb4 to d9/dc/d11/d38/ffd 0 2024-04-24T08:25:00.041 INFO:tasks.workunit.client.1.smithi149.stdout:3/843: chown d0/d5/da/d19/d1b/d22/d38/lff 113743715 1 2024-04-24T08:25:00.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/766: fsync d9/dc/d11/d83/f93 0 2024-04-24T08:25:00.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/479: creat d8/d1d/d1f/d3e/d44/f91 x:0 0 0 2024-04-24T08:25:00.044 INFO:tasks.workunit.client.1.smithi149.stdout:0/698: dread d2/d9/d1a/fbc [0,4194304] 0 2024-04-24T08:25:00.046 INFO:tasks.workunit.client.0.smithi090.stdout:3/420: creat d2/d25/f8c x:0 0 0 2024-04-24T08:25:00.047 INFO:tasks.workunit.client.1.smithi149.stdout:0/699: write d2/d9/df/d2f/d6c/dc1/fcf [937811,75963] 0 2024-04-24T08:25:00.047 INFO:tasks.workunit.client.1.smithi149.stdout:7/767: dread d9/dc/de/f3a [0,4194304] 0 2024-04-24T08:25:00.048 INFO:tasks.workunit.client.1.smithi149.stdout:1/865: rmdir d0/d5/db/d18/d2f 39 2024-04-24T08:25:00.052 INFO:tasks.workunit.client.1.smithi149.stdout:2/678: symlink d0/d6/db/d39/d78/lcb 0 2024-04-24T08:25:00.053 INFO:tasks.workunit.client.0.smithi090.stdout:7/520: write db/d17/d29/d38/f44 [863870,82379] 0 2024-04-24T08:25:00.053 INFO:tasks.workunit.client.0.smithi090.stdout:4/385: creat d3/d18/f70 x:0 0 0 2024-04-24T08:25:00.054 INFO:tasks.workunit.client.1.smithi149.stdout:5/878: mkdir db/d126 0 2024-04-24T08:25:00.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/521: readlink db/d17/d29/d38/d3e/la1 0 2024-04-24T08:25:00.056 INFO:tasks.workunit.client.1.smithi149.stdout:9/813: symlink d1/d2/dc8/dc9/l127 0 2024-04-24T08:25:00.058 INFO:tasks.workunit.client.0.smithi090.stdout:0/496: getdents d1a/d4b/d53/d79/d35/d8e 0 2024-04-24T08:25:00.058 INFO:tasks.workunit.client.1.smithi149.stdout:4/733: dwrite d2/f6c [4194304,4194304] 0 2024-04-24T08:25:00.060 INFO:tasks.workunit.client.0.smithi090.stdout:6/480: unlink d8/d1d/d29/f2c 0 2024-04-24T08:25:00.061 INFO:tasks.workunit.client.1.smithi149.stdout:4/734: write d2/d10/f17 [364867,96597] 0 2024-04-24T08:25:00.062 INFO:tasks.workunit.client.0.smithi090.stdout:0/497: truncate d1a/d4b/d53/d79/d35/d62/f7a 206664 0 2024-04-24T08:25:00.067 INFO:tasks.workunit.client.1.smithi149.stdout:0/700: unlink d2/d17/d41/d68/l87 0 2024-04-24T08:25:00.067 INFO:tasks.workunit.client.1.smithi149.stdout:1/866: chown d0/d5/d4f/d60/fb6 115 1 2024-04-24T08:25:00.067 INFO:tasks.workunit.client.0.smithi090.stdout:0/498: read d1a/d4b/d53/d79/d35/d68/f76 [67005,107451] 0 2024-04-24T08:25:00.069 INFO:tasks.workunit.client.0.smithi090.stdout:1/497: link d3/d25/d30/l3d d3/d5/da/d94/da6/lb0 0 2024-04-24T08:25:00.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/522: fdatasync db/fd 0 2024-04-24T08:25:00.072 INFO:tasks.workunit.client.1.smithi149.stdout:5/879: creat db/d12/d22/d45/d5d/f127 x:0 0 0 2024-04-24T08:25:00.073 INFO:tasks.workunit.client.0.smithi090.stdout:7/523: fsync db/d90/f91 0 2024-04-24T08:25:00.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/499: truncate d1a/d4b/d53/f7f 570357 0 2024-04-24T08:25:00.078 INFO:tasks.workunit.client.1.smithi149.stdout:9/814: mkdir d1/d2/d15/d1c/dca/d128 0 2024-04-24T08:25:00.080 INFO:tasks.workunit.client.0.smithi090.stdout:4/386: dread d3/d24/f2b [0,4194304] 0 2024-04-24T08:25:00.081 INFO:tasks.workunit.client.0.smithi090.stdout:2/510: creat d0/fad x:0 0 0 2024-04-24T08:25:00.086 INFO:tasks.workunit.client.1.smithi149.stdout:4/735: dwrite d2/d18/d1e/d3c/d59/fb1 [0,4194304] 0 2024-04-24T08:25:00.087 INFO:tasks.workunit.client.0.smithi090.stdout:2/511: write d0/d7/d67/d7f/d31/d90/f9d [574803,101552] 0 2024-04-24T08:25:00.088 INFO:tasks.workunit.client.1.smithi149.stdout:7/768: dwrite d9/dc/d11/d83/da6/d67/d69/fe9 [0,4194304] 0 2024-04-24T08:25:00.088 INFO:tasks.workunit.client.0.smithi090.stdout:6/481: dwrite d8/f46 [0,4194304] 0 2024-04-24T08:25:00.094 INFO:tasks.workunit.client.0.smithi090.stdout:3/421: dwrite d2/df/d1a/d37/d53/f70 [0,4194304] 0 2024-04-24T08:25:00.094 INFO:tasks.workunit.client.1.smithi149.stdout:7/769: write d9/dc/d72/fa1 [1933318,42834] 0 2024-04-24T08:25:00.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/498: creat d3/d68/d7f/fb1 x:0 0 0 2024-04-24T08:25:00.095 INFO:tasks.workunit.client.1.smithi149.stdout:5/880: rename db/d12/d1e/d85/d92/d107/f5c to db/da4/f128 0 2024-04-24T08:25:00.096 INFO:tasks.workunit.client.1.smithi149.stdout:3/844: dread d0/fe [0,4194304] 0 2024-04-24T08:25:00.099 INFO:tasks.workunit.client.1.smithi149.stdout:9/815: creat d1/da/d4a/dad/db7/f129 x:0 0 0 2024-04-24T08:25:00.101 INFO:tasks.workunit.client.1.smithi149.stdout:4/736: rmdir d2/d18 39 2024-04-24T08:25:00.107 INFO:tasks.workunit.client.0.smithi090.stdout:2/512: creat d0/d2c/fae x:0 0 0 2024-04-24T08:25:00.112 INFO:tasks.workunit.client.1.smithi149.stdout:7/770: rmdir d9/dc/d2e/d3c 39 2024-04-24T08:25:00.113 INFO:tasks.workunit.client.0.smithi090.stdout:6/482: unlink d8/dc/l8b 0 2024-04-24T08:25:00.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/396: sync 2024-04-24T08:25:00.115 INFO:tasks.workunit.client.1.smithi149.stdout:6/675: sync 2024-04-24T08:25:00.115 INFO:tasks.workunit.client.1.smithi149.stdout:3/845: unlink d0/d5/fc7 0 2024-04-24T08:25:00.118 INFO:tasks.workunit.client.1.smithi149.stdout:8/692: sync 2024-04-24T08:25:00.119 INFO:tasks.workunit.client.1.smithi149.stdout:0/701: write d2/d9/df/d48/f4f [1863294,82296] 0 2024-04-24T08:25:00.121 INFO:tasks.workunit.client.0.smithi090.stdout:7/524: unlink db/d17/l6a 0 2024-04-24T08:25:00.123 INFO:tasks.workunit.client.0.smithi090.stdout:4/387: mknod d3/df/c71 0 2024-04-24T08:25:00.125 INFO:tasks.workunit.client.1.smithi149.stdout:2/679: getdents d0/d6/db/d27/d64/d66 0 2024-04-24T08:25:00.128 INFO:tasks.workunit.client.0.smithi090.stdout:0/500: dwrite d1a/f24 [0,4194304] 0 2024-04-24T08:25:00.129 INFO:tasks.workunit.client.1.smithi149.stdout:9/816: rename d1/da/d13/d74/ca7 to d1/da/d13/d67/c12a 0 2024-04-24T08:25:00.129 INFO:tasks.workunit.client.1.smithi149.stdout:2/680: stat d0/d6/db/d39/d3d/fb5 0 2024-04-24T08:25:00.130 INFO:tasks.workunit.client.0.smithi090.stdout:0/501: write d1a/d4b/f6f [1541209,78380] 0 2024-04-24T08:25:00.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/502: write d1a/d4b/d53/f6a [2882960,8786] 0 2024-04-24T08:25:00.141 INFO:tasks.workunit.client.1.smithi149.stdout:1/867: dwrite d0/da/f36 [0,4194304] 0 2024-04-24T08:25:00.144 INFO:tasks.workunit.client.0.smithi090.stdout:2/513: write d0/d7/d67/d7f/d31/f56 [741535,40856] 0 2024-04-24T08:25:00.144 INFO:tasks.workunit.client.0.smithi090.stdout:3/422: dread d2/d15/d36/f83 [0,4194304] 0 2024-04-24T08:25:00.148 INFO:tasks.workunit.client.1.smithi149.stdout:6/676: fdatasync d0/d1a/d28/d31/f37 0 2024-04-24T08:25:00.148 INFO:tasks.workunit.client.1.smithi149.stdout:7/771: dwrite d9/dc/d11/d83/fd9 [0,4194304] 0 2024-04-24T08:25:00.149 INFO:tasks.workunit.client.1.smithi149.stdout:5/881: write db/d12/d2f/d74/df6/fb6 [872380,127184] 0 2024-04-24T08:25:00.153 INFO:tasks.workunit.client.0.smithi090.stdout:7/525: truncate db/d17/d39/f49 3937351 0 2024-04-24T08:25:00.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/499: creat d3/d5/d3e/da3/fb2 x:0 0 0 2024-04-24T08:25:00.160 INFO:tasks.workunit.client.1.smithi149.stdout:0/702: truncate d2/d9/df/f1d 3596552 0 2024-04-24T08:25:00.162 INFO:tasks.workunit.client.1.smithi149.stdout:5/882: chown db/d12/d60/db2/d83 2557 1 2024-04-24T08:25:00.163 INFO:tasks.workunit.client.1.smithi149.stdout:2/681: dwrite d0/d6/db/f10 [4194304,4194304] 0 2024-04-24T08:25:00.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/514: symlink d0/d7/d6f/laf 0 2024-04-24T08:25:00.165 INFO:tasks.workunit.client.0.smithi090.stdout:2/515: write d0/f3b [1602990,5337] 0 2024-04-24T08:25:00.170 INFO:tasks.workunit.client.0.smithi090.stdout:4/388: link d3/d38/f39 d3/d38/d49/d6f/f72 0 2024-04-24T08:25:00.172 INFO:tasks.workunit.client.1.smithi149.stdout:4/737: creat d2/d10/d1a/d34/feb x:0 0 0 2024-04-24T08:25:00.172 INFO:tasks.workunit.client.0.smithi090.stdout:4/389: chown d3/d38/d49/f4c 72744 1 2024-04-24T08:25:00.174 INFO:tasks.workunit.client.0.smithi090.stdout:1/500: creat d3/d5/d58/fb3 x:0 0 0 2024-04-24T08:25:00.175 INFO:tasks.workunit.client.1.smithi149.stdout:8/693: mknod d1/de/d18/d32/d65/d8b/d7d/cea 0 2024-04-24T08:25:00.176 INFO:tasks.workunit.client.1.smithi149.stdout:0/703: readlink d2/d9/df/d40/lc8 0 2024-04-24T08:25:00.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/694: stat d1/d1f/d25/l4d 0 2024-04-24T08:25:00.178 INFO:tasks.workunit.client.0.smithi090.stdout:3/423: creat d2/d84/f8d x:0 0 0 2024-04-24T08:25:00.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/501: write d3/d5/da/dc/f87 [157311,29357] 0 2024-04-24T08:25:00.181 INFO:tasks.workunit.client.1.smithi149.stdout:2/682: rename d0/f4d to d0/d6/db/d28/d7f/d8f/fcc 0 2024-04-24T08:25:00.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/397: creat d2/da/d13/d64/f86 x:0 0 0 2024-04-24T08:25:00.184 INFO:tasks.workunit.client.0.smithi090.stdout:7/526: creat db/d54/db1/fb4 x:0 0 0 2024-04-24T08:25:00.186 INFO:tasks.workunit.client.0.smithi090.stdout:1/502: chown d3/d25/d30/d9d/da2 208440254 1 2024-04-24T08:25:00.190 INFO:tasks.workunit.client.0.smithi090.stdout:6/483: dwrite d8/d1d/d1f/d3e/d44/f87 [0,4194304] 0 2024-04-24T08:25:00.191 INFO:tasks.workunit.client.0.smithi090.stdout:1/503: fdatasync d3/d5/da/f64 0 2024-04-24T08:25:00.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/516: dwrite d0/d14/d4d/d50/d96/fa2 [0,4194304] 0 2024-04-24T08:25:00.195 INFO:tasks.workunit.client.0.smithi090.stdout:2/517: dread - d0/d2c/f76 zero size 2024-04-24T08:25:00.195 INFO:tasks.workunit.client.0.smithi090.stdout:1/504: fdatasync d3/d5/d3e/f18 0 2024-04-24T08:25:00.198 INFO:tasks.workunit.client.1.smithi149.stdout:3/846: dwrite d0/d5/d1a/f5d [0,4194304] 0 2024-04-24T08:25:00.198 INFO:tasks.workunit.client.1.smithi149.stdout:6/677: creat d0/d1a/d28/d31/d4c/fe4 x:0 0 0 2024-04-24T08:25:00.201 INFO:tasks.workunit.client.0.smithi090.stdout:8/398: dwrite d2/f3 [0,4194304] 0 2024-04-24T08:25:00.210 INFO:tasks.workunit.client.0.smithi090.stdout:0/503: write d1a/d4b/d53/f8b [503521,73632] 0 2024-04-24T08:25:00.212 INFO:tasks.workunit.client.1.smithi149.stdout:2/683: dread - d0/f77 zero size 2024-04-24T08:25:00.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/504: stat d1a/d4b/d53/d79/d35/d62/f7a 0 2024-04-24T08:25:00.214 INFO:tasks.workunit.client.1.smithi149.stdout:8/695: write d1/de/d18/d32/d65/d8b/d7d/f96 [4690496,118399] 0 2024-04-24T08:25:00.215 INFO:tasks.workunit.client.1.smithi149.stdout:1/868: write d0/fd [945861,94471] 0 2024-04-24T08:25:00.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/817: getdents d1/da/dd/d79 0 2024-04-24T08:25:00.224 INFO:tasks.workunit.client.0.smithi090.stdout:7/527: symlink db/d17/lb5 0 2024-04-24T08:25:00.225 INFO:tasks.workunit.client.1.smithi149.stdout:6/678: dwrite d0/f3f [0,4194304] 0 2024-04-24T08:25:00.232 INFO:tasks.workunit.client.1.smithi149.stdout:4/738: unlink d2/d18/d1e/d3c/d48/le7 0 2024-04-24T08:25:00.247 INFO:tasks.workunit.client.1.smithi149.stdout:3/847: unlink d0/d5/da/d5a/d63/d6a/fe9 0 2024-04-24T08:25:00.251 INFO:tasks.workunit.client.1.smithi149.stdout:7/772: truncate d9/dc/d2e/d51/fc0 753827 0 2024-04-24T08:25:00.252 INFO:tasks.workunit.client.0.smithi090.stdout:8/399: dread d2/f3 [4194304,4194304] 0 2024-04-24T08:25:00.258 INFO:tasks.workunit.client.1.smithi149.stdout:1/869: readlink d0/ddf/d53/l10b 0 2024-04-24T08:25:00.258 INFO:tasks.workunit.client.0.smithi090.stdout:0/505: mknod d1a/ca1 0 2024-04-24T08:25:00.260 INFO:tasks.workunit.client.1.smithi149.stdout:9/818: readlink d1/da/l9a 0 2024-04-24T08:25:00.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/505: creat d3/fb4 x:0 0 0 2024-04-24T08:25:00.262 INFO:tasks.workunit.client.0.smithi090.stdout:4/390: dwrite d3/df/f36 [0,4194304] 0 2024-04-24T08:25:00.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/679: creat d0/d1a/d28/d31/d4c/d8a/fe5 x:0 0 0 2024-04-24T08:25:00.267 INFO:tasks.workunit.client.0.smithi090.stdout:2/518: link d0/d2c/fae d0/fb0 0 2024-04-24T08:25:00.268 INFO:tasks.workunit.client.1.smithi149.stdout:5/883: dwrite db/d12/d22/d45/f21 [8388608,4194304] 0 2024-04-24T08:25:00.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/440: sync 2024-04-24T08:25:00.269 INFO:tasks.workunit.client.0.smithi090.stdout:5/527: sync 2024-04-24T08:25:00.270 INFO:tasks.workunit.client.1.smithi149.stdout:3/848: symlink d0/d5/d53/d4c/dca/l11f 0 2024-04-24T08:25:00.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/484: truncate d8/dc/f72 973083 0 2024-04-24T08:25:00.277 INFO:tasks.workunit.client.0.smithi090.stdout:1/506: creat d3/d25/d30/d9d/da2/fb5 x:0 0 0 2024-04-24T08:25:00.278 INFO:tasks.workunit.client.1.smithi149.stdout:7/773: dwrite d9/dc/d11/d38/fe8 [0,4194304] 0 2024-04-24T08:25:00.280 INFO:tasks.workunit.client.0.smithi090.stdout:1/507: chown d3/d1d/d54/l62 282 1 2024-04-24T08:25:00.282 INFO:tasks.workunit.client.1.smithi149.stdout:0/704: rename d2/d3c/c53 to d2/d9/df/d2f/d6c/dc1/d4e/ce3 0 2024-04-24T08:25:00.283 INFO:tasks.workunit.client.1.smithi149.stdout:3/849: truncate d0/d5/da/d5e/f101 38739 0 2024-04-24T08:25:00.291 INFO:tasks.workunit.client.0.smithi090.stdout:4/391: mknod d3/d38/d49/c73 0 2024-04-24T08:25:00.291 INFO:tasks.workunit.client.0.smithi090.stdout:5/528: symlink d8/d76/lb0 0 2024-04-24T08:25:00.292 INFO:tasks.workunit.client.0.smithi090.stdout:5/529: stat d8/da/d3a/d2d 0 2024-04-24T08:25:00.292 INFO:tasks.workunit.client.0.smithi090.stdout:9/441: creat d0/dc/d24/d7c/f82 x:0 0 0 2024-04-24T08:25:00.292 INFO:tasks.workunit.client.1.smithi149.stdout:8/696: mknod d1/d1f/d5e/d92/ceb 0 2024-04-24T08:25:00.294 INFO:tasks.workunit.client.0.smithi090.stdout:7/528: dread db/d17/d29/d32/f36 [0,4194304] 0 2024-04-24T08:25:00.296 INFO:tasks.workunit.client.0.smithi090.stdout:8/400: rename d2/f2c to d2/da/f87 0 2024-04-24T08:25:00.300 INFO:tasks.workunit.client.0.smithi090.stdout:6/485: dwrite d8/d1d/d1f/f89 [0,4194304] 0 2024-04-24T08:25:00.307 INFO:tasks.workunit.client.1.smithi149.stdout:3/850: mkdir d0/d5/d1a/d70/da0/d120 0 2024-04-24T08:25:00.312 INFO:tasks.workunit.client.0.smithi090.stdout:5/530: dwrite d8/da/d3a/d2d/d37/d93/d7a/d50/f71 [4194304,4194304] 0 2024-04-24T08:25:00.314 INFO:tasks.workunit.client.0.smithi090.stdout:5/531: fsync d8/da/d3a/d2d/d37/d93/d7a/d50/f8e 0 2024-04-24T08:25:00.315 INFO:tasks.workunit.client.1.smithi149.stdout:1/870: fsync d0/d1b/d2a/f115 0 2024-04-24T08:25:00.315 INFO:tasks.workunit.client.0.smithi090.stdout:5/532: stat d8/da/d14/d2f 0 2024-04-24T08:25:00.317 INFO:tasks.workunit.client.1.smithi149.stdout:2/684: dwrite d0/d6/db/f42 [0,4194304] 0 2024-04-24T08:25:00.321 INFO:tasks.workunit.client.1.smithi149.stdout:4/739: truncate d2/d18/d1e/d3c/d48/d69/f70 3031280 0 2024-04-24T08:25:00.321 INFO:tasks.workunit.client.1.smithi149.stdout:9/819: mkdir d1/d2/d12b 0 2024-04-24T08:25:00.321 INFO:tasks.workunit.client.1.smithi149.stdout:2/685: read d0/d6/db/d28/d7f/d8f/fcc [2481670,36303] 0 2024-04-24T08:25:00.326 INFO:tasks.workunit.client.1.smithi149.stdout:8/697: mkdir d1/de/d18/d32/d63/d9e/dec 0 2024-04-24T08:25:00.332 INFO:tasks.workunit.client.1.smithi149.stdout:0/705: write d2/d9/df/d40/f67 [1585211,123089] 0 2024-04-24T08:25:00.338 INFO:tasks.workunit.client.1.smithi149.stdout:1/871: rmdir d0/d5 39 2024-04-24T08:25:00.339 INFO:tasks.workunit.client.1.smithi149.stdout:4/740: rmdir d2/d18/dd6 39 2024-04-24T08:25:00.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/508: unlink d3/l13 0 2024-04-24T08:25:00.346 INFO:tasks.workunit.client.0.smithi090.stdout:2/519: mkdir d0/d7/d28/daa/db1 0 2024-04-24T08:25:00.348 INFO:tasks.workunit.client.1.smithi149.stdout:0/706: dwrite d2/d9/fd8 [0,4194304] 0 2024-04-24T08:25:00.348 INFO:tasks.workunit.client.1.smithi149.stdout:5/884: rmdir db/d126 0 2024-04-24T08:25:00.349 INFO:tasks.workunit.client.0.smithi090.stdout:4/392: creat d3/d18/d5c/f74 x:0 0 0 2024-04-24T08:25:00.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/393: stat d3/df/c71 0 2024-04-24T08:25:00.351 INFO:tasks.workunit.client.1.smithi149.stdout:6/680: rename d0/d1a/d7e/lcd to d0/d1a/le6 0 2024-04-24T08:25:00.355 INFO:tasks.workunit.client.1.smithi149.stdout:5/885: read db/d12/d1e/fc1 [2520465,130051] 0 2024-04-24T08:25:00.355 INFO:tasks.workunit.client.1.smithi149.stdout:0/707: write d2/d9/f89 [4413275,113848] 0 2024-04-24T08:25:00.363 INFO:tasks.workunit.client.1.smithi149.stdout:0/708: write d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fcb [286786,69998] 0 2024-04-24T08:25:00.364 INFO:tasks.workunit.client.0.smithi090.stdout:8/401: unlink d2/da/d13/d60/l7b 0 2024-04-24T08:25:00.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/424: sync 2024-04-24T08:25:00.371 INFO:tasks.workunit.client.0.smithi090.stdout:2/520: symlink d0/d14/d4d/d50/lb2 0 2024-04-24T08:25:00.371 INFO:tasks.workunit.client.0.smithi090.stdout:3/425: readlink d2/d25/d2c/l42 0 2024-04-24T08:25:00.373 INFO:tasks.workunit.client.0.smithi090.stdout:4/394: readlink d3/d18/d1d/l54 0 2024-04-24T08:25:00.375 INFO:tasks.workunit.client.1.smithi149.stdout:7/774: rename d9/dc/de/f31 to d9/d43/d4f/d66/d96/ffe 0 2024-04-24T08:25:00.375 INFO:tasks.workunit.client.1.smithi149.stdout:3/851: link d0/d5/da/fa7 d0/d5/da/d5a/d63/d6a/df5/f121 0 2024-04-24T08:25:00.380 INFO:tasks.workunit.client.0.smithi090.stdout:0/506: rename d1a/f64 to d1a/d4b/d53/d79/d69/d8a/fa2 0 2024-04-24T08:25:00.383 INFO:tasks.workunit.client.1.smithi149.stdout:1/872: creat d0/d5/d4f/f11f x:0 0 0 2024-04-24T08:25:00.385 INFO:tasks.workunit.client.0.smithi090.stdout:8/402: symlink d2/da/d13/d60/l88 0 2024-04-24T08:25:00.385 INFO:tasks.workunit.client.1.smithi149.stdout:1/873: chown d0/d5/lb0 0 1 2024-04-24T08:25:00.386 INFO:tasks.workunit.client.1.smithi149.stdout:2/686: creat d0/d6/db/fcd x:0 0 0 2024-04-24T08:25:00.388 INFO:tasks.workunit.client.1.smithi149.stdout:4/741: creat d2/d24/d2e/d38/fec x:0 0 0 2024-04-24T08:25:00.388 INFO:tasks.workunit.client.0.smithi090.stdout:2/521: dwrite d0/d5e/f63 [0,4194304] 0 2024-04-24T08:25:00.390 INFO:tasks.workunit.client.1.smithi149.stdout:9/820: write d1/d2/d34/f110 [228024,116382] 0 2024-04-24T08:25:00.394 INFO:tasks.workunit.client.0.smithi090.stdout:4/395: write d3/d38/d49/d6f/f62 [895683,10042] 0 2024-04-24T08:25:00.394 INFO:tasks.workunit.client.0.smithi090.stdout:2/522: read d0/d7/d67/d7f/f5d [1617882,46490] 0 2024-04-24T08:25:00.395 INFO:tasks.workunit.client.1.smithi149.stdout:8/698: write d1/d1f/d5e/f73 [176924,122367] 0 2024-04-24T08:25:00.398 INFO:tasks.workunit.client.1.smithi149.stdout:8/699: write d1/d1f/f69 [1527016,68462] 0 2024-04-24T08:25:00.398 INFO:tasks.workunit.client.0.smithi090.stdout:9/442: dwrite d0/dc/d24/d2d/f65 [4194304,4194304] 0 2024-04-24T08:25:00.400 INFO:tasks.workunit.client.0.smithi090.stdout:2/523: truncate d0/d14/f34 2297721 0 2024-04-24T08:25:00.401 INFO:tasks.workunit.client.0.smithi090.stdout:7/529: rename db/d17/d29/d32/c48 to db/d54/db1/cb6 0 2024-04-24T08:25:00.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/486: truncate d8/dc/f5f 1789897 0 2024-04-24T08:25:00.403 INFO:tasks.workunit.client.0.smithi090.stdout:7/530: chown db/c89 26253 1 2024-04-24T08:25:00.406 INFO:tasks.workunit.client.0.smithi090.stdout:8/403: mknod d2/da/d13/d64/d24/c89 0 2024-04-24T08:25:00.408 INFO:tasks.workunit.client.0.smithi090.stdout:8/404: read d2/da/d13/d64/d50/f5d [5380782,59930] 0 2024-04-24T08:25:00.411 INFO:tasks.workunit.client.0.smithi090.stdout:8/405: write d2/da/f82 [1231197,126627] 0 2024-04-24T08:25:00.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/396: mknod d3/d18/d5c/c75 0 2024-04-24T08:25:00.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/397: write d3/d38/d49/f65 [315464,71042] 0 2024-04-24T08:25:00.421 INFO:tasks.workunit.client.0.smithi090.stdout:9/443: symlink d0/dc/d24/d7c/d4d/d32/l83 0 2024-04-24T08:25:00.423 INFO:tasks.workunit.client.0.smithi090.stdout:5/533: link d8/d59/l9b d8/da/d3a/d2d/d37/d93/d7a/d50/lb1 0 2024-04-24T08:25:00.424 INFO:tasks.workunit.client.0.smithi090.stdout:0/507: dread d1a/f2c [0,4194304] 0 2024-04-24T08:25:00.424 INFO:tasks.workunit.client.0.smithi090.stdout:6/487: chown d8/d1d/d1f/d3e/d4f 905 1 2024-04-24T08:25:00.450 INFO:tasks.workunit.client.0.smithi090.stdout:1/509: rename d3/d5/c31 to d3/d25/d30/d9d/da2/cb6 0 2024-04-24T08:25:00.452 INFO:tasks.workunit.client.0.smithi090.stdout:6/488: read - d8/dc/d19/d8e/f7b zero size 2024-04-24T08:25:00.453 INFO:tasks.workunit.client.0.smithi090.stdout:0/508: fdatasync d1a/d4b/d53/d79/d3c/f54 0 2024-04-24T08:25:00.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/426: rename d2/df/d21/f3e to d2/df/d5b/f8e 0 2024-04-24T08:25:00.456 INFO:tasks.workunit.client.0.smithi090.stdout:1/510: write d3/d68/d7f/f9e [207107,37904] 0 2024-04-24T08:25:00.461 INFO:tasks.workunit.client.0.smithi090.stdout:3/427: chown d2/df/d21/l58 8140 1 2024-04-24T08:25:00.463 INFO:tasks.workunit.client.0.smithi090.stdout:3/428: chown d2/d25/d2c 2942035 1 2024-04-24T08:25:00.464 INFO:tasks.workunit.client.0.smithi090.stdout:2/524: dwrite d0/d2e/f84 [0,4194304] 0 2024-04-24T08:25:00.468 INFO:tasks.workunit.client.0.smithi090.stdout:6/489: rmdir d8/dc/d19/d8e/d38 39 2024-04-24T08:25:00.470 INFO:tasks.workunit.client.0.smithi090.stdout:7/531: rename db/d17/d29/d38/d99/f9b to db/d17/d29/d38/d3e/fb7 0 2024-04-24T08:25:00.472 INFO:tasks.workunit.client.1.smithi149.stdout:1/874: symlink d0/ddf/d76/l120 0 2024-04-24T08:25:00.473 INFO:tasks.workunit.client.0.smithi090.stdout:3/429: creat d2/d15/d36/d6f/d7a/f8f x:0 0 0 2024-04-24T08:25:00.477 INFO:tasks.workunit.client.0.smithi090.stdout:5/534: dread d8/da/d3a/d2d/d37/d93/d7a/d50/f79 [0,4194304] 0 2024-04-24T08:25:00.478 INFO:tasks.workunit.client.0.smithi090.stdout:0/509: dwrite d1a/d4b/d53/d79/d58/f93 [0,4194304] 0 2024-04-24T08:25:00.479 INFO:tasks.workunit.client.0.smithi090.stdout:4/398: dread d3/d38/d49/f4c [4194304,4194304] 0 2024-04-24T08:25:00.481 INFO:tasks.workunit.client.0.smithi090.stdout:5/535: read d8/da/d14/d2f/f66 [149635,64294] 0 2024-04-24T08:25:00.483 INFO:tasks.workunit.client.0.smithi090.stdout:1/511: symlink d3/d5/da/d97/lb7 0 2024-04-24T08:25:00.484 INFO:tasks.workunit.client.1.smithi149.stdout:8/700: chown d1/lbd 257553061 1 2024-04-24T08:25:00.487 INFO:tasks.workunit.client.0.smithi090.stdout:7/532: creat db/d17/d29/d38/d3e/fb8 x:0 0 0 2024-04-24T08:25:00.488 INFO:tasks.workunit.client.1.smithi149.stdout:3/852: fsync d0/d5/da/d5a/d63/d6a/df5/f121 0 2024-04-24T08:25:00.495 INFO:tasks.workunit.client.0.smithi090.stdout:9/444: dread d0/fb [0,4194304] 0 2024-04-24T08:25:00.495 INFO:tasks.workunit.client.0.smithi090.stdout:2/525: mknod d0/d2c/d99/cb3 0 2024-04-24T08:25:00.495 INFO:tasks.workunit.client.0.smithi090.stdout:4/399: fsync d3/d18/d1d/f40 0 2024-04-24T08:25:00.499 INFO:tasks.workunit.client.0.smithi090.stdout:5/536: dwrite f7 [0,4194304] 0 2024-04-24T08:25:00.499 INFO:tasks.workunit.client.0.smithi090.stdout:1/512: unlink d3/d1d/fa7 0 2024-04-24T08:25:00.503 INFO:tasks.workunit.client.0.smithi090.stdout:3/430: mkdir d2/d90 0 2024-04-24T08:25:00.503 INFO:tasks.workunit.client.0.smithi090.stdout:4/400: dread d3/d38/d49/f53 [0,4194304] 0 2024-04-24T08:25:00.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/533: symlink db/d17/d29/d32/d5a/d75/lb9 0 2024-04-24T08:25:00.504 INFO:tasks.workunit.client.0.smithi090.stdout:4/401: write d3/d38/d49/f65 [788747,33211] 0 2024-04-24T08:25:00.506 INFO:tasks.workunit.client.0.smithi090.stdout:9/445: unlink d0/d19/f1c 0 2024-04-24T08:25:00.507 INFO:tasks.workunit.client.0.smithi090.stdout:9/446: readlink d0/d19/l31 0 2024-04-24T08:25:00.508 INFO:tasks.workunit.client.0.smithi090.stdout:9/447: chown d0/dc/f3e 94 1 2024-04-24T08:25:00.509 INFO:tasks.workunit.client.1.smithi149.stdout:5/886: write db/d12/d1e/f3c [4243042,109926] 0 2024-04-24T08:25:00.509 INFO:tasks.workunit.client.0.smithi090.stdout:0/510: mkdir d1a/d4b/d53/d79/da3 0 2024-04-24T08:25:00.511 INFO:tasks.workunit.client.1.smithi149.stdout:2/687: write d0/d6/db/d39/d78/d7c/f82 [27161,14066] 0 2024-04-24T08:25:00.513 INFO:tasks.workunit.client.0.smithi090.stdout:2/526: mkdir d0/d14/d4d/db4 0 2024-04-24T08:25:00.514 INFO:tasks.workunit.client.1.smithi149.stdout:4/742: write d2/d10/f81 [5884342,113348] 0 2024-04-24T08:25:00.514 INFO:tasks.workunit.client.0.smithi090.stdout:2/527: stat d0/d2e/f49 0 2024-04-24T08:25:00.514 INFO:tasks.workunit.client.1.smithi149.stdout:0/709: write d2/d17/f5e [703526,3397] 0 2024-04-24T08:25:00.523 INFO:tasks.workunit.client.1.smithi149.stdout:9/821: rmdir d1/de/d9d/dfd/d100 0 2024-04-24T08:25:00.524 INFO:tasks.workunit.client.1.smithi149.stdout:6/681: link d0/d1a/d28/d31/d4c/d4e/cdf d0/ce7 0 2024-04-24T08:25:00.524 INFO:tasks.workunit.client.1.smithi149.stdout:7/775: dwrite d9/dc/d11/d83/da6/f71 [4194304,4194304] 0 2024-04-24T08:25:00.525 INFO:tasks.workunit.client.0.smithi090.stdout:6/490: write d8/dc/f5f [1888013,124873] 0 2024-04-24T08:25:00.526 INFO:tasks.workunit.client.0.smithi090.stdout:1/513: creat d3/d68/fb8 x:0 0 0 2024-04-24T08:25:00.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/537: fsync d8/d56/f5f 0 2024-04-24T08:25:00.528 INFO:tasks.workunit.client.0.smithi090.stdout:2/528: dwrite d0/d7/d67/d7f/d31/f56 [0,4194304] 0 2024-04-24T08:25:00.529 INFO:tasks.workunit.client.0.smithi090.stdout:1/514: dread - d3/d25/d30/d9d/fa8 zero size 2024-04-24T08:25:00.530 INFO:tasks.workunit.client.1.smithi149.stdout:5/887: mknod db/d12/d60/c129 0 2024-04-24T08:25:00.532 INFO:tasks.workunit.client.1.smithi149.stdout:7/776: dread d9/dc/de/f3a [0,4194304] 0 2024-04-24T08:25:00.532 INFO:tasks.workunit.client.0.smithi090.stdout:3/431: unlink d2/df/f6e 0 2024-04-24T08:25:00.535 INFO:tasks.workunit.client.1.smithi149.stdout:4/743: chown d2/l45 93408088 1 2024-04-24T08:25:00.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/402: write d3/df/f30 [1637773,121406] 0 2024-04-24T08:25:00.541 INFO:tasks.workunit.client.1.smithi149.stdout:8/701: truncate d1/de/d18/d32/d65/d8b/dc9/fe7 61544 0 2024-04-24T08:25:00.545 INFO:tasks.workunit.client.1.smithi149.stdout:8/702: dread - d1/de/d18/d32/d65/d8b/d7d/ddc/fe0 zero size 2024-04-24T08:25:00.545 INFO:tasks.workunit.client.1.smithi149.stdout:0/710: dwrite d2/d17/fd1 [0,4194304] 0 2024-04-24T08:25:00.546 INFO:tasks.workunit.client.0.smithi090.stdout:5/538: dwrite d8/da/d18/d21/f41 [0,4194304] 0 2024-04-24T08:25:00.550 INFO:tasks.workunit.client.0.smithi090.stdout:9/448: creat d0/dc/d24/d7c/d4d/f84 x:0 0 0 2024-04-24T08:25:00.550 INFO:tasks.workunit.client.1.smithi149.stdout:9/822: unlink d1/da/d13/d74/l112 0 2024-04-24T08:25:00.557 INFO:tasks.workunit.client.1.smithi149.stdout:6/682: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fe8 x:0 0 0 2024-04-24T08:25:00.557 INFO:tasks.workunit.client.0.smithi090.stdout:6/491: mknod d8/d1d/d2f/c92 0 2024-04-24T08:25:00.565 INFO:tasks.workunit.client.1.smithi149.stdout:7/777: creat d9/d43/d4f/da5/fff x:0 0 0 2024-04-24T08:25:00.569 INFO:tasks.workunit.client.1.smithi149.stdout:4/744: symlink d2/d18/db9/led 0 2024-04-24T08:25:00.572 INFO:tasks.workunit.client.1.smithi149.stdout:1/875: getdents d0/d1b/d5d/de5/d101 0 2024-04-24T08:25:00.575 INFO:tasks.workunit.client.1.smithi149.stdout:1/876: write d0/ddf/da0/db8/dec/f103 [1025216,54089] 0 2024-04-24T08:25:00.576 INFO:tasks.workunit.client.1.smithi149.stdout:0/711: dwrite d2/d9/df/d2f/d6c/f9a [0,4194304] 0 2024-04-24T08:25:00.578 INFO:tasks.workunit.client.1.smithi149.stdout:1/877: dread - d0/d5/d4f/f11f zero size 2024-04-24T08:25:00.580 INFO:tasks.workunit.client.1.smithi149.stdout:7/778: dread f1 [4194304,4194304] 0 2024-04-24T08:25:00.582 INFO:tasks.workunit.client.1.smithi149.stdout:3/853: getdents d0/d5/d53/d34/d3a/d93 0 2024-04-24T08:25:00.591 INFO:tasks.workunit.client.0.smithi090.stdout:5/539: creat d8/d76/fb2 x:0 0 0 2024-04-24T08:25:00.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/511: dread f18 [0,4194304] 0 2024-04-24T08:25:00.595 INFO:tasks.workunit.client.0.smithi090.stdout:9/449: creat d0/d69/d7d/f85 x:0 0 0 2024-04-24T08:25:00.595 INFO:tasks.workunit.client.0.smithi090.stdout:5/540: truncate d8/da/d18/fa0 1020211 0 2024-04-24T08:25:00.596 INFO:tasks.workunit.client.0.smithi090.stdout:8/406: sync 2024-04-24T08:25:00.597 INFO:tasks.workunit.client.0.smithi090.stdout:5/541: dread - d8/d76/fb2 zero size 2024-04-24T08:25:00.598 INFO:tasks.workunit.client.1.smithi149.stdout:4/745: truncate d2/d18/f1b 2835246 0 2024-04-24T08:25:00.598 INFO:tasks.workunit.client.0.smithi090.stdout:5/542: stat d8/da/d18/d21/l29 0 2024-04-24T08:25:00.599 INFO:tasks.workunit.client.1.smithi149.stdout:2/688: dwrite d0/f70 [4194304,4194304] 0 2024-04-24T08:25:00.599 INFO:tasks.workunit.client.0.smithi090.stdout:2/529: mknod d0/d7/d67/d7f/d31/cb5 0 2024-04-24T08:25:00.602 INFO:tasks.workunit.client.1.smithi149.stdout:0/712: readlink d2/l3 0 2024-04-24T08:25:00.602 INFO:tasks.workunit.client.0.smithi090.stdout:1/515: creat d3/d5/da/d94/da6/fb9 x:0 0 0 2024-04-24T08:25:00.607 INFO:tasks.workunit.client.1.smithi149.stdout:7/779: rmdir d9/dc/d11/d84/da7 39 2024-04-24T08:25:00.612 INFO:tasks.workunit.client.0.smithi090.stdout:6/492: mknod d8/dc/c93 0 2024-04-24T08:25:00.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/450: rmdir d0/d2a 39 2024-04-24T08:25:00.614 INFO:tasks.workunit.client.0.smithi090.stdout:6/493: stat d8/dc/d19/f64 0 2024-04-24T08:25:00.616 INFO:tasks.workunit.client.0.smithi090.stdout:6/494: chown d8/dc/c93 192 1 2024-04-24T08:25:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:5/888: creat db/d12/d22/f12a x:0 0 0 2024-04-24T08:25:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:1/878: dread d0/d1b/d5d/f6c [0,4194304] 0 2024-04-24T08:25:00.616 INFO:tasks.workunit.client.0.smithi090.stdout:9/451: truncate d0/d19/f28 9093282 0 2024-04-24T08:25:00.618 INFO:tasks.workunit.client.0.smithi090.stdout:7/534: dwrite db/d90/faf [0,4194304] 0 2024-04-24T08:25:00.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/407: rmdir d2/da 39 2024-04-24T08:25:00.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/889: chown db/d12/d60/c129 7556628 1 2024-04-24T08:25:00.621 INFO:tasks.workunit.client.0.smithi090.stdout:0/512: creat d1a/d4b/d53/d57/fa4 x:0 0 0 2024-04-24T08:25:00.622 INFO:tasks.workunit.client.1.smithi149.stdout:9/823: dwrite d1/de/fdb [0,4194304] 0 2024-04-24T08:25:00.622 INFO:tasks.workunit.client.1.smithi149.stdout:5/890: write db/d12/d1e/d65/fe1 [525096,121057] 0 2024-04-24T08:25:00.626 INFO:tasks.workunit.client.1.smithi149.stdout:4/746: mknod d2/d10/d1a/dac/cee 0 2024-04-24T08:25:00.626 INFO:tasks.workunit.client.0.smithi090.stdout:5/543: symlink d8/da/d3a/d2d/d37/lb3 0 2024-04-24T08:25:00.627 INFO:tasks.workunit.client.1.smithi149.stdout:5/891: chown db/d12/d60/c129 366806 1 2024-04-24T08:25:00.632 INFO:tasks.workunit.client.1.smithi149.stdout:1/879: dread d0/ddd/fe9 [0,4194304] 0 2024-04-24T08:25:00.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/403: dwrite d3/f32 [0,4194304] 0 2024-04-24T08:25:00.638 INFO:tasks.workunit.client.1.smithi149.stdout:7/780: symlink d9/dc/d11/d26/l100 0 2024-04-24T08:25:00.638 INFO:tasks.workunit.client.1.smithi149.stdout:8/703: truncate d1/f8e 946490 0 2024-04-24T08:25:00.638 INFO:tasks.workunit.client.1.smithi149.stdout:7/781: fsync d9/dc/d11/d38/fe8 0 2024-04-24T08:25:00.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/683: creat d0/d1a/fe9 x:0 0 0 2024-04-24T08:25:00.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/747: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/def 0 2024-04-24T08:25:00.659 INFO:tasks.workunit.client.1.smithi149.stdout:3/854: rename d0/d5/d53/d4c/dca/l11f to d0/d5/da/d5a/d63/l122 0 2024-04-24T08:25:00.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/713: dwrite d2/d9/df/d2f/d6c/d72/f99 [0,4194304] 0 2024-04-24T08:25:00.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/704: rename d1/de/d18/d32/d65/d8b/f7c to d1/dce/fed 0 2024-04-24T08:25:00.672 INFO:tasks.workunit.client.1.smithi149.stdout:4/748: dread d2/d18/db9/dd8/fa0 [0,4194304] 0 2024-04-24T08:25:00.676 INFO:tasks.workunit.client.1.smithi149.stdout:2/689: link d0/d6/db/d27/c69 d0/d6/db/d39/d3d/d58/cce 0 2024-04-24T08:25:00.677 INFO:tasks.workunit.client.1.smithi149.stdout:3/855: dread d0/d5/d53/d34/d3a/d93/f77 [0,4194304] 0 2024-04-24T08:25:00.682 INFO:tasks.workunit.client.1.smithi149.stdout:1/880: dread d0/d5/db/d63/f86 [0,4194304] 0 2024-04-24T08:25:00.684 INFO:tasks.workunit.client.1.smithi149.stdout:7/782: getdents d9/dc/de/d44 0 2024-04-24T08:25:00.687 INFO:tasks.workunit.client.1.smithi149.stdout:4/749: mkdir d2/d18/d1e/d3c/df0 0 2024-04-24T08:25:00.688 INFO:tasks.workunit.client.1.smithi149.stdout:2/690: mkdir d0/d6/db/d27/d64/d66/db1/dcf 0 2024-04-24T08:25:00.690 INFO:tasks.workunit.client.1.smithi149.stdout:3/856: rmdir d0/d5/d1a/d70/da0 39 2024-04-24T08:25:00.690 INFO:tasks.workunit.client.1.smithi149.stdout:8/705: dread d1/de/d18/f7e [0,4194304] 0 2024-04-24T08:25:00.704 INFO:tasks.workunit.client.1.smithi149.stdout:4/750: fsync d2/fc 0 2024-04-24T08:25:00.704 INFO:tasks.workunit.client.1.smithi149.stdout:5/892: truncate db/f94 922314 0 2024-04-24T08:25:00.707 INFO:tasks.workunit.client.1.smithi149.stdout:7/783: dread d9/d43/d4f/d66/f6e [0,4194304] 0 2024-04-24T08:25:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:9/824: dwrite d1/de/f2e [0,4194304] 0 2024-04-24T08:25:00.714 INFO:tasks.workunit.client.1.smithi149.stdout:8/706: getdents d1/d1f/d5e/d92 0 2024-04-24T08:25:00.714 INFO:tasks.workunit.client.1.smithi149.stdout:7/784: mkdir d9/d43/d4f/d101 0 2024-04-24T08:25:00.717 INFO:tasks.workunit.client.1.smithi149.stdout:4/751: fdatasync d2/fc2 0 2024-04-24T08:25:00.719 INFO:tasks.workunit.client.1.smithi149.stdout:6/684: dwrite d0/d1a/d28/d31/d38/f3a [4194304,4194304] 0 2024-04-24T08:25:00.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/857: dwrite d0/d5/d53/dd6/f110 [0,4194304] 0 2024-04-24T08:25:00.726 INFO:tasks.workunit.client.1.smithi149.stdout:5/893: truncate db/d12/d60/fe7 3076628 0 2024-04-24T08:25:00.728 INFO:tasks.workunit.client.1.smithi149.stdout:1/881: write d0/d1b/d5d/f6c [2063672,128851] 0 2024-04-24T08:25:00.732 INFO:tasks.workunit.client.0.smithi090.stdout:6/495: fdatasync d8/d1d/d2f/f3c 0 2024-04-24T08:25:00.732 INFO:tasks.workunit.client.1.smithi149.stdout:0/714: dwrite d2/d9/df/d2f/d6c/dc1/fa9 [0,4194304] 0 2024-04-24T08:25:00.733 INFO:tasks.workunit.client.0.smithi090.stdout:7/535: rename db/d54/d83/l9d to db/d90/lba 0 2024-04-24T08:25:00.734 INFO:tasks.workunit.client.0.smithi090.stdout:1/516: write d3/d1d/f55 [1098178,77191] 0 2024-04-24T08:25:00.735 INFO:tasks.workunit.client.0.smithi090.stdout:5/544: truncate d8/d25/f26 1372290 0 2024-04-24T08:25:00.736 INFO:tasks.workunit.client.0.smithi090.stdout:0/513: fsync d1a/f1e 0 2024-04-24T08:25:00.736 INFO:tasks.workunit.client.1.smithi149.stdout:0/715: readlink d2/d3c/d57/l58 0 2024-04-24T08:25:00.737 INFO:tasks.workunit.client.1.smithi149.stdout:5/894: write db/d12/d22/f12a [510588,59529] 0 2024-04-24T08:25:00.739 INFO:tasks.workunit.client.0.smithi090.stdout:3/432: getdents d2/d25/d2c 0 2024-04-24T08:25:00.740 INFO:tasks.workunit.client.0.smithi090.stdout:4/404: mknod d3/d38/c76 0 2024-04-24T08:25:00.740 INFO:tasks.workunit.client.1.smithi149.stdout:1/882: dread - d0/d1b/db5/d108/f10c zero size 2024-04-24T08:25:00.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/408: readlink d2/da/d13/d64/d24/l56 0 2024-04-24T08:25:00.742 INFO:tasks.workunit.client.1.smithi149.stdout:4/752: fdatasync d2/d10/d1a/fda 0 2024-04-24T08:25:00.745 INFO:tasks.workunit.client.0.smithi090.stdout:2/530: rename d0/d7/d57/d79/c89 to d0/d14/d4d/d50/d96/cb6 0 2024-04-24T08:25:00.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/545: symlink d8/d76/lb4 0 2024-04-24T08:25:00.750 INFO:tasks.workunit.client.1.smithi149.stdout:6/685: fsync d0/d1a/d28/d31/d33/f9e 0 2024-04-24T08:25:00.750 INFO:tasks.workunit.client.1.smithi149.stdout:2/691: dwrite d0/d6/db/d27/d64/d66/f93 [0,4194304] 0 2024-04-24T08:25:00.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/496: fsync d8/d1d/d2f/f3c 0 2024-04-24T08:25:00.757 INFO:tasks.workunit.client.1.smithi149.stdout:3/858: mkdir d0/d5/d53/dd6/d123 0 2024-04-24T08:25:00.757 INFO:tasks.workunit.client.0.smithi090.stdout:6/497: truncate d8/f3f 632864 0 2024-04-24T08:25:00.759 INFO:tasks.workunit.client.0.smithi090.stdout:6/498: stat d8/d1d/d2f/f3c 0 2024-04-24T08:25:00.760 INFO:tasks.workunit.client.0.smithi090.stdout:2/531: dwrite d0/d7/d57/f66 [0,4194304] 0 2024-04-24T08:25:00.761 INFO:tasks.workunit.client.1.smithi149.stdout:8/707: write d1/d1f/d25/f42 [552760,120976] 0 2024-04-24T08:25:00.802 INFO:tasks.workunit.client.1.smithi149.stdout:4/753: dread d2/d24/f39 [0,4194304] 0 2024-04-24T08:25:00.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/517: fsync d3/d5/da/dc/fab 0 2024-04-24T08:25:00.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/452: sync 2024-04-24T08:25:00.863 INFO:tasks.workunit.client.1.smithi149.stdout:9/825: write d1/d2/d33/dea/d126/d72/f7b [473478,43005] 0 2024-04-24T08:25:00.866 INFO:tasks.workunit.client.1.smithi149.stdout:7/785: write d9/dc/de/d44/fd5 [102164,37259] 0 2024-04-24T08:25:00.874 INFO:tasks.workunit.client.0.smithi090.stdout:8/409: truncate d2/da/d13/f3a 2655235 0 2024-04-24T08:25:00.879 INFO:tasks.workunit.client.0.smithi090.stdout:5/546: mknod d8/da/d18/d21/d4c/cb5 0 2024-04-24T08:25:00.887 INFO:tasks.workunit.client.0.smithi090.stdout:0/514: fsync d1a/f1d 0 2024-04-24T08:25:00.898 INFO:tasks.workunit.client.0.smithi090.stdout:1/518: creat d3/d5/da/dc/d80/fba x:0 0 0 2024-04-24T08:25:00.900 INFO:tasks.workunit.client.1.smithi149.stdout:7/786: fdatasync d9/dc/de/d44/f91 0 2024-04-24T08:25:00.903 INFO:tasks.workunit.client.1.smithi149.stdout:5/895: mknod db/d12/d1e/d65/de8/dfa/c12b 0 2024-04-24T08:25:00.913 INFO:tasks.workunit.client.0.smithi090.stdout:8/410: mknod d2/da/d13/d34/c8a 0 2024-04-24T08:25:00.920 INFO:tasks.workunit.client.0.smithi090.stdout:5/547: write f6 [5316199,111127] 0 2024-04-24T08:25:00.928 INFO:tasks.workunit.client.0.smithi090.stdout:7/536: creat db/d17/d39/fbb x:0 0 0 2024-04-24T08:25:00.933 INFO:tasks.workunit.client.1.smithi149.stdout:9/826: creat d1/d2/d34/d11d/f12c x:0 0 0 2024-04-24T08:25:00.937 INFO:tasks.workunit.client.1.smithi149.stdout:7/787: creat d9/dc/d7b/f102 x:0 0 0 2024-04-24T08:25:00.937 INFO:tasks.workunit.client.0.smithi090.stdout:3/433: link d2/df/d57/d4f/l66 d2/df/d1a/d37/d59/d87/l91 0 2024-04-24T08:25:00.939 INFO:tasks.workunit.client.1.smithi149.stdout:7/788: read d9/dc/d11/d38/fe8 [1285011,28053] 0 2024-04-24T08:25:00.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/411: dread d2/da/d13/d34/f3d [0,4194304] 0 2024-04-24T08:25:00.944 INFO:tasks.workunit.client.1.smithi149.stdout:7/789: readlink d9/dc/d11/d83/da6/d67/dba/led 0 2024-04-24T08:25:00.944 INFO:tasks.workunit.client.0.smithi090.stdout:6/499: rmdir d8/dc/d19/d8e/d81/d8a 0 2024-04-24T08:25:00.947 INFO:tasks.workunit.client.1.smithi149.stdout:0/716: mknod d2/d9/d1a/d36/ce4 0 2024-04-24T08:25:00.950 INFO:tasks.workunit.client.0.smithi090.stdout:3/434: dread d2/df/d1a/d37/d53/f70 [0,4194304] 0 2024-04-24T08:25:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:7/790: fsync f1 0 2024-04-24T08:25:00.957 INFO:tasks.workunit.client.1.smithi149.stdout:4/754: getdents d2/d10 0 2024-04-24T08:25:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/717: chown d2/d9/df/l20 4308 1 2024-04-24T08:25:00.962 INFO:tasks.workunit.client.0.smithi090.stdout:4/405: getdents d3 0 2024-04-24T08:25:00.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/515: rmdir d1a/d4b/d53/d79/da3 0 2024-04-24T08:25:00.970 INFO:tasks.workunit.client.1.smithi149.stdout:7/791: dwrite d9/fe5 [4194304,4194304] 0 2024-04-24T08:25:00.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/718: rename d2/d9/df/d40/fb4 to d2/d9/d1a/d36/d85/fe5 0 2024-04-24T08:25:00.973 INFO:tasks.workunit.client.0.smithi090.stdout:3/435: mkdir d2/d15/d36/d6f/d92 0 2024-04-24T08:25:00.978 INFO:tasks.workunit.client.0.smithi090.stdout:2/532: getdents d0/d2c 0 2024-04-24T08:25:00.982 INFO:tasks.workunit.client.0.smithi090.stdout:1/519: link d3/d1d/d34/f3b d3/d1d/d54/fbb 0 2024-04-24T08:25:00.992 INFO:tasks.workunit.client.1.smithi149.stdout:7/792: dwrite d9/dc/d11/d83/da6/d67/d69/fcf [0,4194304] 0 2024-04-24T08:25:00.994 INFO:tasks.workunit.client.0.smithi090.stdout:3/436: unlink d2/df/d1a/c54 0 2024-04-24T08:25:01.002 INFO:tasks.workunit.client.1.smithi149.stdout:7/793: mkdir d9/dc/d11/d38/dac/d103 0 2024-04-24T08:25:01.004 INFO:tasks.workunit.client.0.smithi090.stdout:1/520: creat d3/d5/da/dc/d80/fbc x:0 0 0 2024-04-24T08:25:01.004 INFO:tasks.workunit.client.0.smithi090.stdout:8/412: symlink d2/l8b 0 2024-04-24T08:25:01.006 INFO:tasks.workunit.client.0.smithi090.stdout:8/413: readlink d2/da/l68 0 2024-04-24T08:25:01.009 INFO:tasks.workunit.client.0.smithi090.stdout:3/437: fdatasync d2/df/d1a/d37/d53/f70 0 2024-04-24T08:25:01.020 INFO:tasks.workunit.client.0.smithi090.stdout:6/500: link d8/d1d/d1f/d3e/d4f/c76 d8/d1d/c94 0 2024-04-24T08:25:01.020 INFO:tasks.workunit.client.0.smithi090.stdout:3/438: stat d2/ld 0 2024-04-24T08:25:01.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/439: mkdir d2/df/d1a/d37/d53/d60/d93 0 2024-04-24T08:25:01.025 INFO:tasks.workunit.client.0.smithi090.stdout:9/453: creat d0/dc/d24/d7c/f86 x:0 0 0 2024-04-24T08:25:01.027 INFO:tasks.workunit.client.0.smithi090.stdout:3/440: chown d2/d15/d36/d6f/d92 0 1 2024-04-24T08:25:01.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/454: mkdir d0/dc/d24/d2d/d49/d81/d87 0 2024-04-24T08:25:01.037 INFO:tasks.workunit.client.0.smithi090.stdout:8/414: dwrite d2/da/f87 [0,4194304] 0 2024-04-24T08:25:01.038 INFO:tasks.workunit.client.1.smithi149.stdout:6/686: write d0/f26 [1275879,37587] 0 2024-04-24T08:25:01.040 INFO:tasks.workunit.client.0.smithi090.stdout:6/501: dwrite d8/dc/d19/f3d [0,4194304] 0 2024-04-24T08:25:01.044 INFO:tasks.workunit.client.0.smithi090.stdout:9/455: mknod d0/dc/d24/d2d/d49/d81/d87/c88 0 2024-04-24T08:25:01.044 INFO:tasks.workunit.client.1.smithi149.stdout:6/687: read d0/d1a/f2b [148801,26068] 0 2024-04-24T08:25:01.045 INFO:tasks.workunit.client.0.smithi090.stdout:6/502: write d8/f16 [10048707,82502] 0 2024-04-24T08:25:01.049 INFO:tasks.workunit.client.1.smithi149.stdout:2/692: dwrite d0/f77 [0,4194304] 0 2024-04-24T08:25:01.055 INFO:tasks.workunit.client.0.smithi090.stdout:8/415: creat d2/da/d13/d64/f8c x:0 0 0 2024-04-24T08:25:01.055 INFO:tasks.workunit.client.0.smithi090.stdout:8/416: chown d2/d62/d83 91 1 2024-04-24T08:25:01.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/693: symlink d0/d6/db/d28/db0/ld0 0 2024-04-24T08:25:01.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/688: rename d0/d1a/d28/d31/d4c/d4e/d62/d66/dc9 to d0/d1a/d28/d31/d4c/d4e/db8/dea 0 2024-04-24T08:25:01.065 INFO:tasks.workunit.client.0.smithi090.stdout:8/417: mknod d2/da/d13/d64/d24/c8d 0 2024-04-24T08:25:01.071 INFO:tasks.workunit.client.1.smithi149.stdout:2/694: getdents d0/d6/db/d28/d7f 0 2024-04-24T08:25:01.072 INFO:tasks.workunit.client.0.smithi090.stdout:8/418: write d2/da/d13/f5f [1492008,25913] 0 2024-04-24T08:25:01.074 INFO:tasks.workunit.client.0.smithi090.stdout:8/419: chown d2/l11 1151388615 1 2024-04-24T08:25:01.075 INFO:tasks.workunit.client.1.smithi149.stdout:2/695: symlink d0/d6/db/d27/d81/d9f/ld1 0 2024-04-24T08:25:01.081 INFO:tasks.workunit.client.0.smithi090.stdout:8/420: creat d2/da/d13/d64/f8e x:0 0 0 2024-04-24T08:25:01.088 INFO:tasks.workunit.client.1.smithi149.stdout:1/883: dwrite d0/d5/db/d18/d4d/d50/f118 [0,4194304] 0 2024-04-24T08:25:01.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/421: creat d2/da/d13/d64/f8f x:0 0 0 2024-04-24T08:25:01.090 INFO:tasks.workunit.client.1.smithi149.stdout:1/884: chown d0/d5/db/d63/f95 50 1 2024-04-24T08:25:01.093 INFO:tasks.workunit.client.1.smithi149.stdout:1/885: chown d0/d5/df/c20 61 1 2024-04-24T08:25:01.096 INFO:tasks.workunit.client.0.smithi090.stdout:8/422: dread d2/da/d13/d34/f3c [0,4194304] 0 2024-04-24T08:25:01.097 INFO:tasks.workunit.client.0.smithi090.stdout:5/548: write d8/da/d3a/d2d/d37/d93/d7a/d50/f8a [155009,86785] 0 2024-04-24T08:25:01.100 INFO:tasks.workunit.client.0.smithi090.stdout:8/423: creat d2/d62/d6b/f90 x:0 0 0 2024-04-24T08:25:01.104 INFO:tasks.workunit.client.0.smithi090.stdout:5/549: mkdir d8/da/d3a/d2d/d37/d93/d7a/db6 0 2024-04-24T08:25:01.108 INFO:tasks.workunit.client.1.smithi149.stdout:3/859: dwrite d0/d5/da/d19/d1b/d22/dd4/fe1 [0,4194304] 0 2024-04-24T08:25:01.108 INFO:tasks.workunit.client.0.smithi090.stdout:5/550: mknod d8/da/d3a/d2d/d37/cb7 0 2024-04-24T08:25:01.111 INFO:tasks.workunit.client.1.smithi149.stdout:5/896: dwrite db/f14 [0,4194304] 0 2024-04-24T08:25:01.112 INFO:tasks.workunit.client.0.smithi090.stdout:5/551: write d8/da/d3a/d2d/d37/d93/d7a/d50/f71 [2396530,95757] 0 2024-04-24T08:25:01.114 INFO:tasks.workunit.client.1.smithi149.stdout:5/897: fsync db/d12/d1e/d85/d92/da6/fce 0 2024-04-24T08:25:01.115 INFO:tasks.workunit.client.1.smithi149.stdout:3/860: symlink d0/d5/d53/d39/l124 0 2024-04-24T08:25:01.117 INFO:tasks.workunit.client.1.smithi149.stdout:4/755: truncate d2/d18/d1e/d3c/d48/d69/fb6 3385389 0 2024-04-24T08:25:01.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/898: fdatasync db/d12/d22/f12a 0 2024-04-24T08:25:01.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/756: chown d2/d10/d91 174938 1 2024-04-24T08:25:01.124 INFO:tasks.workunit.client.0.smithi090.stdout:2/533: rename d0/d7/d28/f91 to d0/d7/d67/d7f/d31/d46/fb7 0 2024-04-24T08:25:01.125 INFO:tasks.workunit.client.1.smithi149.stdout:3/861: mknod d0/d26/d3c/c125 0 2024-04-24T08:25:01.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/441: rename d2/d15/d36 to d2/d15/d36/d6f/d7a/d94 22 2024-04-24T08:25:01.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/424: dwrite d2/da/d13/d64/d24/f5b [0,4194304] 0 2024-04-24T08:25:01.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/899: creat db/d12/d1e/d85/f12c x:0 0 0 2024-04-24T08:25:01.134 INFO:tasks.workunit.client.0.smithi090.stdout:7/537: dwrite db/f10 [0,4194304] 0 2024-04-24T08:25:01.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/886: dread d0/d5/df/fca [0,4194304] 0 2024-04-24T08:25:01.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/534: symlink d0/d14/d17/lb8 0 2024-04-24T08:25:01.139 INFO:tasks.workunit.client.0.smithi090.stdout:2/535: chown d0/d7/d67/f72 8328974 1 2024-04-24T08:25:01.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/552: dwrite d8/da/d14/d2f/d44/f49 [0,4194304] 0 2024-04-24T08:25:01.143 INFO:tasks.workunit.client.1.smithi149.stdout:5/900: mkdir db/d12/d1e/d12d 0 2024-04-24T08:25:01.146 INFO:tasks.workunit.client.0.smithi090.stdout:7/538: symlink db/d54/d78/lbc 0 2024-04-24T08:25:01.153 INFO:tasks.workunit.client.0.smithi090.stdout:4/406: truncate d3/d38/d49/d6f/f72 6581904 0 2024-04-24T08:25:01.154 INFO:tasks.workunit.client.0.smithi090.stdout:8/425: rmdir d2/da/d13/d64/d24/d81 0 2024-04-24T08:25:01.154 INFO:tasks.workunit.client.1.smithi149.stdout:0/719: mkdir d2/d9/de6 0 2024-04-24T08:25:01.155 INFO:tasks.workunit.client.1.smithi149.stdout:5/901: mkdir db/d12/d1e/d12e 0 2024-04-24T08:25:01.156 INFO:tasks.workunit.client.0.smithi090.stdout:0/516: write d1a/d4b/d53/d79/d69/d8a/fa2 [566789,66879] 0 2024-04-24T08:25:01.157 INFO:tasks.workunit.client.0.smithi090.stdout:3/442: truncate d2/df/d1a/d37/f47 1590460 0 2024-04-24T08:25:01.157 INFO:tasks.workunit.client.1.smithi149.stdout:0/720: symlink d2/d9/df/d2f/d6c/dc1/d4e/d6f/le7 0 2024-04-24T08:25:01.159 INFO:tasks.workunit.client.0.smithi090.stdout:7/539: unlink db/d17/d39/d59/fa0 0 2024-04-24T08:25:01.160 INFO:tasks.workunit.client.0.smithi090.stdout:6/503: getdents d8/d1d 0 2024-04-24T08:25:01.162 INFO:tasks.workunit.client.1.smithi149.stdout:5/902: creat db/d12/d1e/d85/d92/f12f x:0 0 0 2024-04-24T08:25:01.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/794: truncate d9/dc/d11/d83/da6/d67/d69/fe9 2536503 0 2024-04-24T08:25:01.167 INFO:tasks.workunit.client.1.smithi149.stdout:7/795: chown d9/dc/d11/f42 23 1 2024-04-24T08:25:01.168 INFO:tasks.workunit.client.0.smithi090.stdout:1/521: symlink d3/d1d/d34/lbd 0 2024-04-24T08:25:01.168 INFO:tasks.workunit.client.0.smithi090.stdout:5/553: dread d8/da/d14/d2f/d44/f5b [0,4194304] 0 2024-04-24T08:25:01.169 INFO:tasks.workunit.client.0.smithi090.stdout:4/407: rmdir d3/df 39 2024-04-24T08:25:01.170 INFO:tasks.workunit.client.0.smithi090.stdout:2/536: dwrite d0/d7/d67/d7f/f6e [0,4194304] 0 2024-04-24T08:25:01.173 INFO:tasks.workunit.client.0.smithi090.stdout:0/517: creat d1a/d4b/d53/d79/d58/fa5 x:0 0 0 2024-04-24T08:25:01.177 INFO:tasks.workunit.client.1.smithi149.stdout:0/721: creat d2/d9/df/fe8 x:0 0 0 2024-04-24T08:25:01.182 INFO:tasks.workunit.client.1.smithi149.stdout:7/796: creat d9/d43/d4f/d101/f104 x:0 0 0 2024-04-24T08:25:01.184 INFO:tasks.workunit.client.0.smithi090.stdout:1/522: readlink d3/d25/d30/l3d 0 2024-04-24T08:25:01.189 INFO:tasks.workunit.client.1.smithi149.stdout:5/903: dwrite db/d12/d1e/d65/fe1 [0,4194304] 0 2024-04-24T08:25:01.191 INFO:tasks.workunit.client.1.smithi149.stdout:4/757: dread d2/d10/d1a/f3e [0,4194304] 0 2024-04-24T08:25:01.192 INFO:tasks.workunit.client.0.smithi090.stdout:3/443: link d2/df/d1a/f2b d2/d25/f95 0 2024-04-24T08:25:01.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/504: unlink d8/d1d/d1f/d3e/f60 0 2024-04-24T08:25:01.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/518: creat d1a/d4b/d53/d79/d82/fa6 x:0 0 0 2024-04-24T08:25:01.203 INFO:tasks.workunit.client.0.smithi090.stdout:5/554: dwrite d8/da/d18/d21/d4c/f9a [0,4194304] 0 2024-04-24T08:25:01.207 INFO:tasks.workunit.client.0.smithi090.stdout:4/408: write d3/d18/f70 [1023463,69994] 0 2024-04-24T08:25:01.208 INFO:tasks.workunit.client.0.smithi090.stdout:1/523: symlink d3/d5/da/d94/lbe 0 2024-04-24T08:25:01.212 INFO:tasks.workunit.client.1.smithi149.stdout:0/722: dread d2/d9/df/d56/f80 [0,4194304] 0 2024-04-24T08:25:01.213 INFO:tasks.workunit.client.1.smithi149.stdout:4/758: symlink d2/d18/d1e/d3c/d48/d69/d7d/de9/lf1 0 2024-04-24T08:25:01.215 INFO:tasks.workunit.client.1.smithi149.stdout:2/696: write d0/d6/db/d39/d3d/d58/d98/fa5 [4710462,127664] 0 2024-04-24T08:25:01.218 INFO:tasks.workunit.client.1.smithi149.stdout:0/723: write d2/d9/df/d2f/f8f [1984108,92186] 0 2024-04-24T08:25:01.220 INFO:tasks.workunit.client.1.smithi149.stdout:5/904: rename db/d12/d2f/d74/dc6/d113/cef to db/d12/d1e/d65/c130 0 2024-04-24T08:25:01.220 INFO:tasks.workunit.client.1.smithi149.stdout:2/697: dread - d0/d6/db/d27/d81/fc7 zero size 2024-04-24T08:25:01.223 INFO:tasks.workunit.client.1.smithi149.stdout:5/905: write db/d12/d1e/d85/d92/da6/fcf [4300610,58444] 0 2024-04-24T08:25:01.223 INFO:tasks.workunit.client.0.smithi090.stdout:1/524: creat d3/d5/da/d97/fbf x:0 0 0 2024-04-24T08:25:01.225 INFO:tasks.workunit.client.1.smithi149.stdout:7/797: link d9/dc/d2e/l30 d9/dc/d11/d38/dac/l105 0 2024-04-24T08:25:01.228 INFO:tasks.workunit.client.0.smithi090.stdout:0/519: rename ca to d1a/d4b/d53/d79/d35/d68/ca7 0 2024-04-24T08:25:01.229 INFO:tasks.workunit.client.0.smithi090.stdout:4/409: unlink d3/d18/c2d 0 2024-04-24T08:25:01.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/525: rmdir d3/d5/da/dc 39 2024-04-24T08:25:01.235 INFO:tasks.workunit.client.0.smithi090.stdout:5/555: rename d8/da/d18/fab to d8/da/d14/d2f/d81/fb8 0 2024-04-24T08:25:01.238 INFO:tasks.workunit.client.0.smithi090.stdout:5/556: readlink d8/da/d3a/d2d/d37/d93/d7a/d50/lb1 0 2024-04-24T08:25:01.238 INFO:tasks.workunit.client.0.smithi090.stdout:0/520: mknod d1a/d4b/d53/d79/d69/ca8 0 2024-04-24T08:25:01.240 INFO:tasks.workunit.client.1.smithi149.stdout:0/724: symlink d2/d3c/dc7/le9 0 2024-04-24T08:25:01.241 INFO:tasks.workunit.client.0.smithi090.stdout:4/410: readlink d3/d25/l48 0 2024-04-24T08:25:01.243 INFO:tasks.workunit.client.0.smithi090.stdout:5/557: truncate d8/da/d3a/d2d/d37/d93/d7a/d50/f68 769017 0 2024-04-24T08:25:01.243 INFO:tasks.workunit.client.0.smithi090.stdout:4/411: chown d3/d18/d5c/c75 557794817 1 2024-04-24T08:25:01.243 INFO:tasks.workunit.client.1.smithi149.stdout:4/759: dread d2/d10/f81 [0,4194304] 0 2024-04-24T08:25:01.244 INFO:tasks.workunit.client.0.smithi090.stdout:1/526: fsync d3/d5/da/dc/fab 0 2024-04-24T08:25:01.245 INFO:tasks.workunit.client.0.smithi090.stdout:5/558: readlink d8/da/d3a/d2d/d37/lb3 0 2024-04-24T08:25:01.245 INFO:tasks.workunit.client.1.smithi149.stdout:4/760: dread - d2/d10/d1a/d34/f62 zero size 2024-04-24T08:25:01.247 INFO:tasks.workunit.client.1.smithi149.stdout:0/725: creat d2/fea x:0 0 0 2024-04-24T08:25:01.248 INFO:tasks.workunit.client.0.smithi090.stdout:1/527: stat d3/d5/c15 0 2024-04-24T08:25:01.249 INFO:tasks.workunit.client.0.smithi090.stdout:1/528: fsync d3/d5/da/dc/d80/fba 0 2024-04-24T08:25:01.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/559: symlink d8/da/d3a/d2d/d37/d93/d7a/d50/lb9 0 2024-04-24T08:25:01.257 INFO:tasks.workunit.client.0.smithi090.stdout:5/560: creat d8/da/d3a/d2d/d37/d93/d7a/d50/d9f/fba x:0 0 0 2024-04-24T08:25:01.259 INFO:tasks.workunit.client.0.smithi090.stdout:1/529: getdents d3/d1d/d34 0 2024-04-24T08:25:01.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/561: fsync d8/d25/f3d 0 2024-04-24T08:25:01.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/537: write d0/fb0 [881176,3930] 0 2024-04-24T08:25:01.265 INFO:tasks.workunit.client.0.smithi090.stdout:2/538: mkdir d0/d14/d36/db9 0 2024-04-24T08:25:01.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/505: write d8/d1d/d1f/f5d [185249,72346] 0 2024-04-24T08:25:01.268 INFO:tasks.workunit.client.0.smithi090.stdout:9/456: sync 2024-04-24T08:25:01.269 INFO:tasks.workunit.client.0.smithi090.stdout:7/540: dwrite db/d17/d39/f3c [0,4194304] 0 2024-04-24T08:25:01.274 INFO:tasks.workunit.client.0.smithi090.stdout:2/539: creat d0/d7/d67/fba x:0 0 0 2024-04-24T08:25:01.279 INFO:tasks.workunit.client.0.smithi090.stdout:7/541: write db/d17/d29/d38/d3e/f58 [3970686,31614] 0 2024-04-24T08:25:01.280 INFO:tasks.workunit.client.0.smithi090.stdout:1/530: dwrite d3/d5/da/dc/f87 [0,4194304] 0 2024-04-24T08:25:01.283 INFO:tasks.workunit.client.0.smithi090.stdout:2/540: link d0/lc d0/d7/d57/d79/lbb 0 2024-04-24T08:25:01.287 INFO:tasks.workunit.client.0.smithi090.stdout:6/506: dwrite d8/dc/f5f [0,4194304] 0 2024-04-24T08:25:01.287 INFO:tasks.workunit.client.0.smithi090.stdout:7/542: unlink db/d17/d29/f87 0 2024-04-24T08:25:01.289 INFO:tasks.workunit.client.0.smithi090.stdout:2/541: symlink d0/d7/d57/lbc 0 2024-04-24T08:25:01.307 INFO:tasks.workunit.client.0.smithi090.stdout:6/507: symlink d8/dc/d19/l95 0 2024-04-24T08:25:01.307 INFO:tasks.workunit.client.0.smithi090.stdout:2/542: rename d0/d7/d67/d7f/c3d to d0/d2e/cbd 0 2024-04-24T08:25:01.310 INFO:tasks.workunit.client.0.smithi090.stdout:6/508: mkdir d8/dc/d19/d8e/d81/d96 0 2024-04-24T08:25:01.314 INFO:tasks.workunit.client.0.smithi090.stdout:6/509: mkdir d8/dc/d19/d8e/d81/d96/d97 0 2024-04-24T08:25:01.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/510: fsync d8/d1d/d1f/f2e 0 2024-04-24T08:25:01.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/511: chown d8/dc/d19/d8e/f52 9157 1 2024-04-24T08:25:01.319 INFO:tasks.workunit.client.0.smithi090.stdout:1/531: dwrite d3/d5/d58/f60 [0,4194304] 0 2024-04-24T08:25:01.338 INFO:tasks.workunit.client.0.smithi090.stdout:2/543: dread d0/d7/d67/d7f/d31/f41 [0,4194304] 0 2024-04-24T08:25:01.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/512: dread d8/dc/d19/f27 [0,4194304] 0 2024-04-24T08:25:01.348 INFO:tasks.workunit.client.0.smithi090.stdout:2/544: creat d0/d7/d67/d7f/d31/d46/d60/d65/d86/fbe x:0 0 0 2024-04-24T08:25:01.354 INFO:tasks.workunit.client.0.smithi090.stdout:6/513: creat d8/d1d/d1f/d3e/d4f/f98 x:0 0 0 2024-04-24T08:25:01.355 INFO:tasks.workunit.client.0.smithi090.stdout:2/545: creat d0/d7/d28/fbf x:0 0 0 2024-04-24T08:25:01.358 INFO:tasks.workunit.client.0.smithi090.stdout:2/546: creat d0/d7/d67/d7f/d31/d46/d60/d65/fc0 x:0 0 0 2024-04-24T08:25:01.360 INFO:tasks.workunit.client.0.smithi090.stdout:6/514: rename d8/d1d/d29/f61 to d8/d1d/d1f/d3e/d4d/f99 0 2024-04-24T08:25:01.367 INFO:tasks.workunit.client.1.smithi149.stdout:3/862: write d0/d5/d53/d34/f35 [1157566,42540] 0 2024-04-24T08:25:01.367 INFO:tasks.workunit.client.1.smithi149.stdout:5/906: rmdir db/d12/d1e/d85 39 2024-04-24T08:25:01.369 INFO:tasks.workunit.client.1.smithi149.stdout:7/798: write d9/d43/d64/f73 [1350058,79795] 0 2024-04-24T08:25:01.372 INFO:tasks.workunit.client.1.smithi149.stdout:4/761: rename d2/d24/lbb to d2/d18/d1e/d3c/d59/lf2 0 2024-04-24T08:25:01.374 INFO:tasks.workunit.client.1.smithi149.stdout:3/863: readlink d0/d5/da/d5a/d63/d6a/df5/l10a 0 2024-04-24T08:25:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:5/907: read db/d12/d1e/d85/d92/f9f [1773439,35309] 0 2024-04-24T08:25:01.383 INFO:tasks.workunit.client.1.smithi149.stdout:3/864: chown d0/d5/da/d19/d1b/d22/d38 41 1 2024-04-24T08:25:01.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/908: truncate db/d12/d1e/d65/de8/ff5 99426 0 2024-04-24T08:25:01.391 INFO:tasks.workunit.client.0.smithi090.stdout:9/457: truncate d0/dc/d24/d7c/d4d/f52 1977055 0 2024-04-24T08:25:01.392 INFO:tasks.workunit.client.0.smithi090.stdout:9/458: stat d0/dc/f29 0 2024-04-24T08:25:01.393 INFO:tasks.workunit.client.0.smithi090.stdout:6/515: dread d8/d1d/d1f/d3e/f5c [0,4194304] 0 2024-04-24T08:25:01.394 INFO:tasks.workunit.client.0.smithi090.stdout:9/459: fdatasync d0/d19/f25 0 2024-04-24T08:25:01.396 INFO:tasks.workunit.client.0.smithi090.stdout:9/460: write d0/d19/f28 [6262154,15170] 0 2024-04-24T08:25:01.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/461: read - d0/dc/d24/d7c/d4d/f5f zero size 2024-04-24T08:25:01.401 INFO:tasks.workunit.client.0.smithi090.stdout:9/462: chown d0/dc/d24/d7c/f40 1 1 2024-04-24T08:25:01.421 INFO:tasks.workunit.client.1.smithi149.stdout:7/799: truncate d9/dc/d7b/fa2 4186679 0 2024-04-24T08:25:01.425 INFO:tasks.workunit.client.1.smithi149.stdout:5/909: mknod db/d23/c131 0 2024-04-24T08:25:01.432 INFO:tasks.workunit.client.1.smithi149.stdout:5/910: mknod db/d12/d22/c132 0 2024-04-24T08:25:01.437 INFO:tasks.workunit.client.1.smithi149.stdout:5/911: creat db/d12/d2f/d95/f133 x:0 0 0 2024-04-24T08:25:01.441 INFO:tasks.workunit.client.1.smithi149.stdout:7/800: dwrite d9/d43/d64/fc5 [4194304,4194304] 0 2024-04-24T08:25:01.443 INFO:tasks.workunit.client.1.smithi149.stdout:8/708: sync 2024-04-24T08:25:01.446 INFO:tasks.workunit.client.1.smithi149.stdout:5/912: unlink db/d12/d2f/d74/dc6/d113/l112 0 2024-04-24T08:25:01.446 INFO:tasks.workunit.client.1.smithi149.stdout:7/801: read - d9/dc/d11/d83/da6/d67/ff9 zero size 2024-04-24T08:25:01.446 INFO:tasks.workunit.client.1.smithi149.stdout:4/762: dwrite d2/d10/f27 [0,4194304] 0 2024-04-24T08:25:01.448 INFO:tasks.workunit.client.1.smithi149.stdout:8/709: creat d1/d1f/d5e/d92/fee x:0 0 0 2024-04-24T08:25:01.451 INFO:tasks.workunit.client.1.smithi149.stdout:8/710: stat d1/d1f/d5e/d9c 0 2024-04-24T08:25:01.452 INFO:tasks.workunit.client.1.smithi149.stdout:4/763: creat d2/d18/db9/dd8/ff3 x:0 0 0 2024-04-24T08:25:01.454 INFO:tasks.workunit.client.1.smithi149.stdout:7/802: symlink d9/dc/d2e/d3c/db8/l106 0 2024-04-24T08:25:01.455 INFO:tasks.workunit.client.1.smithi149.stdout:7/803: read d9/d43/f7e [1934596,17047] 0 2024-04-24T08:25:01.461 INFO:tasks.workunit.client.1.smithi149.stdout:0/726: truncate d2/d9/df/f5d 4169528 0 2024-04-24T08:25:01.465 INFO:tasks.workunit.client.1.smithi149.stdout:3/865: write d0/d5/da/d5a/d63/d6a/f7c [358930,104948] 0 2024-04-24T08:25:01.469 INFO:tasks.workunit.client.0.smithi090.stdout:9/463: rmdir d0/dc/d24/d2d/d49/d81 39 2024-04-24T08:25:01.474 INFO:tasks.workunit.client.1.smithi149.stdout:5/913: dwrite db/d12/d1e/f41 [0,4194304] 0 2024-04-24T08:25:01.474 INFO:tasks.workunit.client.0.smithi090.stdout:6/516: dwrite d8/d1d/d2f/f84 [0,4194304] 0 2024-04-24T08:25:01.477 INFO:tasks.workunit.client.1.smithi149.stdout:4/764: link d2/d18/db9/dd8/ff3 d2/d18/ff4 0 2024-04-24T08:25:01.478 INFO:tasks.workunit.client.1.smithi149.stdout:3/866: rmdir d0/d5/d53/d6d 39 2024-04-24T08:25:01.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/543: dwrite db/d17/f3a [0,4194304] 0 2024-04-24T08:25:01.480 INFO:tasks.workunit.client.0.smithi090.stdout:2/547: dwrite d0/f21 [0,4194304] 0 2024-04-24T08:25:01.485 INFO:tasks.workunit.client.0.smithi090.stdout:2/548: write d0/d7/d6f/f29 [3756949,16208] 0 2024-04-24T08:25:01.488 INFO:tasks.workunit.client.1.smithi149.stdout:5/914: chown db/d12/d22/d45/l55 91 1 2024-04-24T08:25:01.490 INFO:tasks.workunit.client.1.smithi149.stdout:4/765: creat d2/d10/d1a/da2/ff5 x:0 0 0 2024-04-24T08:25:01.495 INFO:tasks.workunit.client.0.smithi090.stdout:6/517: unlink d8/dc/d19/d8e/c32 0 2024-04-24T08:25:01.495 INFO:tasks.workunit.client.0.smithi090.stdout:7/544: mkdir db/dbd 0 2024-04-24T08:25:01.498 INFO:tasks.workunit.client.1.smithi149.stdout:7/804: dwrite d9/dc/d11/d83/da6/d67/f9e [0,4194304] 0 2024-04-24T08:25:01.503 INFO:tasks.workunit.client.0.smithi090.stdout:9/464: dwrite d0/f53 [0,4194304] 0 2024-04-24T08:25:01.509 INFO:tasks.workunit.client.1.smithi149.stdout:5/915: mknod db/d35/d123/c134 0 2024-04-24T08:25:01.510 INFO:tasks.workunit.client.0.smithi090.stdout:2/549: dwrite d0/d2c/f76 [0,4194304] 0 2024-04-24T08:25:01.510 INFO:tasks.workunit.client.0.smithi090.stdout:9/465: truncate d0/d69/d7d/f5c 1350517 0 2024-04-24T08:25:01.518 INFO:tasks.workunit.client.0.smithi090.stdout:7/545: creat db/d17/d29/d32/d5a/d75/fbe x:0 0 0 2024-04-24T08:25:01.519 INFO:tasks.workunit.client.1.smithi149.stdout:8/711: dread d1/de/d18/d91/d9f/fc6 [0,4194304] 0 2024-04-24T08:25:01.520 INFO:tasks.workunit.client.0.smithi090.stdout:6/518: rename d8/f16 to d8/d1d/d1f/d3e/d7a/f9a 0 2024-04-24T08:25:01.522 INFO:tasks.workunit.client.1.smithi149.stdout:7/805: unlink d9/dc/d11/c4d 0 2024-04-24T08:25:01.525 INFO:tasks.workunit.client.0.smithi090.stdout:9/466: read d0/dc/d24/d2d/f3f [100501,11666] 0 2024-04-24T08:25:01.527 INFO:tasks.workunit.client.1.smithi149.stdout:5/916: mkdir db/d12/d60/db2/d83/d135 0 2024-04-24T08:25:01.528 INFO:tasks.workunit.client.1.smithi149.stdout:7/806: write d9/dc/d11/fef [19254,107737] 0 2024-04-24T08:25:01.532 INFO:tasks.workunit.client.1.smithi149.stdout:7/807: fsync d9/dc/d2e/d3c/ffa 0 2024-04-24T08:25:01.534 INFO:tasks.workunit.client.0.smithi090.stdout:2/550: dwrite d0/d2e/f84 [4194304,4194304] 0 2024-04-24T08:25:01.536 INFO:tasks.workunit.client.0.smithi090.stdout:2/551: fsync d0/d7/d67/d7f/d31/d90/f9d 0 2024-04-24T08:25:01.540 INFO:tasks.workunit.client.0.smithi090.stdout:3/444: sync 2024-04-24T08:25:01.540 INFO:tasks.workunit.client.0.smithi090.stdout:8/426: sync 2024-04-24T08:25:01.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/427: readlink l1 0 2024-04-24T08:25:01.545 INFO:tasks.workunit.client.0.smithi090.stdout:8/428: readlink d2/da/d13/d64/d24/l3b 0 2024-04-24T08:25:01.545 INFO:tasks.workunit.client.0.smithi090.stdout:8/429: stat d2/da/d13 0 2024-04-24T08:25:01.546 INFO:tasks.workunit.client.1.smithi149.stdout:5/917: dread db/d12/d1e/d85/d92/fcc [0,4194304] 0 2024-04-24T08:25:01.549 INFO:tasks.workunit.client.1.smithi149.stdout:0/727: dread d2/d9/d1a/d36/f38 [0,4194304] 0 2024-04-24T08:25:01.550 INFO:tasks.workunit.client.0.smithi090.stdout:6/519: creat d8/dc/d19/d8e/d81/d96/d97/f9b x:0 0 0 2024-04-24T08:25:01.551 INFO:tasks.workunit.client.1.smithi149.stdout:7/808: mknod d9/d43/d4f/dc8/dea/c107 0 2024-04-24T08:25:01.554 INFO:tasks.workunit.client.1.smithi149.stdout:7/809: chown d9/dc/d11/d38/dac/lbc 12 1 2024-04-24T08:25:01.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/467: mknod d0/d5e/d72/d77/c89 0 2024-04-24T08:25:01.559 INFO:tasks.workunit.client.1.smithi149.stdout:4/766: dread d2/d24/d2e/d38/f5c [0,4194304] 0 2024-04-24T08:25:01.560 INFO:tasks.workunit.client.1.smithi149.stdout:5/918: creat db/d12/d60/db2/de2/f136 x:0 0 0 2024-04-24T08:25:01.562 INFO:tasks.workunit.client.0.smithi090.stdout:2/552: rmdir d0/d7/d67/d7f 39 2024-04-24T08:25:01.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/553: chown d0/d7/d28/l35 7238102 1 2024-04-24T08:25:01.565 INFO:tasks.workunit.client.0.smithi090.stdout:3/445: rename d2/df/d1a/l34 to d2/d25/d33/l96 0 2024-04-24T08:25:01.571 INFO:tasks.workunit.client.0.smithi090.stdout:6/520: creat d8/d1d/d1f/f9c x:0 0 0 2024-04-24T08:25:01.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/546: write db/d17/d29/d32/d5a/d75/f81 [1558719,83558] 0 2024-04-24T08:25:01.573 INFO:tasks.workunit.client.0.smithi090.stdout:6/521: chown d8/d1d/d1f/f2e 0 1 2024-04-24T08:25:01.573 INFO:tasks.workunit.client.0.smithi090.stdout:9/468: dread - d0/dc/f58 zero size 2024-04-24T08:25:01.574 INFO:tasks.workunit.client.1.smithi149.stdout:5/919: creat db/d12/d2f/d95/f137 x:0 0 0 2024-04-24T08:25:01.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/469: chown d0/dc/f64 309 1 2024-04-24T08:25:01.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/767: creat d2/d18/d1e/d3c/d48/d69/d7d/de9/ff6 x:0 0 0 2024-04-24T08:25:01.576 INFO:tasks.workunit.client.0.smithi090.stdout:2/554: creat d0/d2c/d99/fc1 x:0 0 0 2024-04-24T08:25:01.577 INFO:tasks.workunit.client.1.smithi149.stdout:7/810: symlink d9/dc/d11/d83/da6/l108 0 2024-04-24T08:25:01.580 INFO:tasks.workunit.client.1.smithi149.stdout:4/768: read d2/d18/d1e/d3c/f43 [3993100,56575] 0 2024-04-24T08:25:01.582 INFO:tasks.workunit.client.1.smithi149.stdout:5/920: symlink db/d12/d2f/d95/l138 0 2024-04-24T08:25:01.583 INFO:tasks.workunit.client.0.smithi090.stdout:8/430: dread d2/da/d13/d34/f4d [0,4194304] 0 2024-04-24T08:25:01.585 INFO:tasks.workunit.client.0.smithi090.stdout:6/522: truncate d8/f15 3638849 0 2024-04-24T08:25:01.585 INFO:tasks.workunit.client.1.smithi149.stdout:8/712: dwrite d1/d1f/d31/f84 [0,4194304] 0 2024-04-24T08:25:01.587 INFO:tasks.workunit.client.0.smithi090.stdout:6/523: fdatasync d8/dc/d19/d8e/f52 0 2024-04-24T08:25:01.587 INFO:tasks.workunit.client.1.smithi149.stdout:7/811: dread - d9/dc/d2e/d3c/db8/fdb zero size 2024-04-24T08:25:01.587 INFO:tasks.workunit.client.0.smithi090.stdout:3/446: symlink d2/d15/d36/d6f/d92/l97 0 2024-04-24T08:25:01.589 INFO:tasks.workunit.client.0.smithi090.stdout:2/555: fsync d0/d7/d67/d7f/d31/f56 0 2024-04-24T08:25:01.590 INFO:tasks.workunit.client.1.smithi149.stdout:8/713: stat d1/d1f/d5e 0 2024-04-24T08:25:01.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/921: chown db/d12/d1e/l5e 8 1 2024-04-24T08:25:01.594 INFO:tasks.workunit.client.1.smithi149.stdout:7/812: dread d9/dc/de/d44/f91 [0,4194304] 0 2024-04-24T08:25:01.595 INFO:tasks.workunit.client.0.smithi090.stdout:7/547: getdents db/dbd 0 2024-04-24T08:25:01.596 INFO:tasks.workunit.client.0.smithi090.stdout:7/548: write db/fc [3982286,41642] 0 2024-04-24T08:25:01.597 INFO:tasks.workunit.client.0.smithi090.stdout:7/549: chown db/d17/d39/d59 4 1 2024-04-24T08:25:01.600 INFO:tasks.workunit.client.0.smithi090.stdout:7/550: read - db/d17/d29/d38/d3e/fb8 zero size 2024-04-24T08:25:01.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/551: chown db/d17/d39/f97 6 1 2024-04-24T08:25:01.603 INFO:tasks.workunit.client.0.smithi090.stdout:1/532: sync 2024-04-24T08:25:01.603 INFO:tasks.workunit.client.0.smithi090.stdout:0/521: sync 2024-04-24T08:25:01.603 INFO:tasks.workunit.client.0.smithi090.stdout:4/412: sync 2024-04-24T08:25:01.603 INFO:tasks.workunit.client.0.smithi090.stdout:5/562: sync 2024-04-24T08:25:01.604 INFO:tasks.workunit.client.0.smithi090.stdout:9/470: creat d0/dc/d24/d2d/d49/d81/d87/f8a x:0 0 0 2024-04-24T08:25:01.605 INFO:tasks.workunit.client.1.smithi149.stdout:5/922: creat db/d12/d1e/d65/de8/dfa/f139 x:0 0 0 2024-04-24T08:25:01.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/728: dread d2/d9/df/d2f/d6c/dc1/d4e/d6f/f76 [0,4194304] 0 2024-04-24T08:25:01.607 INFO:tasks.workunit.client.0.smithi090.stdout:9/471: write d0/d69/f6f [272774,92291] 0 2024-04-24T08:25:01.614 INFO:tasks.workunit.client.0.smithi090.stdout:3/447: mkdir d2/df/d1a/d37/d59/d98 0 2024-04-24T08:25:01.615 INFO:tasks.workunit.client.1.smithi149.stdout:8/714: rename d1/f8e to d1/de/d18/d91/d9f/fef 0 2024-04-24T08:25:01.617 INFO:tasks.workunit.client.1.smithi149.stdout:5/923: symlink db/d12/d2f/d74/df6/l13a 0 2024-04-24T08:25:01.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/924: stat db/d12/d2f/d95/l138 0 2024-04-24T08:25:01.619 INFO:tasks.workunit.client.0.smithi090.stdout:2/556: mkdir d0/d14/d17/dc2 0 2024-04-24T08:25:01.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/715: readlink d1/de/l37 0 2024-04-24T08:25:01.623 INFO:tasks.workunit.client.1.smithi149.stdout:8/716: chown d1/d1f/d31/fc0 23029 1 2024-04-24T08:25:01.623 INFO:tasks.workunit.client.1.smithi149.stdout:4/769: dread d2/f6c [0,4194304] 0 2024-04-24T08:25:01.625 INFO:tasks.workunit.client.1.smithi149.stdout:7/813: rename d9/dc/d7b/f102 to d9/d43/d4f/d66/f109 0 2024-04-24T08:25:01.625 INFO:tasks.workunit.client.0.smithi090.stdout:7/552: fsync f4 0 2024-04-24T08:25:01.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/533: creat d3/d5/da/d97/d9c/fc0 x:0 0 0 2024-04-24T08:25:01.626 INFO:tasks.workunit.client.1.smithi149.stdout:4/770: chown d2/d10/d1a/dac/ddd 0 1 2024-04-24T08:25:01.627 INFO:tasks.workunit.client.1.smithi149.stdout:7/814: fdatasync d9/dc/d11/d83/f93 0 2024-04-24T08:25:01.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/553: dread - db/d17/d29/d38/d3e/fb7 zero size 2024-04-24T08:25:01.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/431: dread d2/f59 [0,4194304] 0 2024-04-24T08:25:01.632 INFO:tasks.workunit.client.0.smithi090.stdout:5/563: write d8/da/d3a/d2d/d37/d93/d7a/d50/f5e [5202152,14077] 0 2024-04-24T08:25:01.633 INFO:tasks.workunit.client.1.smithi149.stdout:8/717: unlink d1/de/d18/d32/l3f 0 2024-04-24T08:25:01.634 INFO:tasks.workunit.client.0.smithi090.stdout:8/432: fsync d2/da/d13/d64/d50/f57 0 2024-04-24T08:25:01.635 INFO:tasks.workunit.client.0.smithi090.stdout:5/564: chown d8/l32 1 1 2024-04-24T08:25:01.636 INFO:tasks.workunit.client.0.smithi090.stdout:6/524: symlink d8/l9d 0 2024-04-24T08:25:01.637 INFO:tasks.workunit.client.0.smithi090.stdout:5/565: chown d8/d25/f72 210635 1 2024-04-24T08:25:01.638 INFO:tasks.workunit.client.1.smithi149.stdout:7/815: mkdir d9/dc/d11/d83/da6/d67/d69/d10a 0 2024-04-24T08:25:01.639 INFO:tasks.workunit.client.0.smithi090.stdout:5/566: read d8/d56/f5f [161018,23416] 0 2024-04-24T08:25:01.642 INFO:tasks.workunit.client.1.smithi149.stdout:8/718: creat d1/de/d18/d32/d65/d8b/d7d/ddc/ff0 x:0 0 0 2024-04-24T08:25:01.644 INFO:tasks.workunit.client.1.smithi149.stdout:8/719: fsync d1/d1f/d31/f84 0 2024-04-24T08:25:01.644 INFO:tasks.workunit.client.1.smithi149.stdout:5/925: dwrite db/d12/d2f/d74/dc6/d113/f10d [0,4194304] 0 2024-04-24T08:25:01.647 INFO:tasks.workunit.client.0.smithi090.stdout:0/522: dwrite d1a/d4b/d53/d79/d35/d68/f76 [0,4194304] 0 2024-04-24T08:25:01.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/771: getdents d2/d24/d2e 0 2024-04-24T08:25:01.651 INFO:tasks.workunit.client.0.smithi090.stdout:0/523: write d1a/d4b/d53/d57/fa4 [419404,97610] 0 2024-04-24T08:25:01.653 INFO:tasks.workunit.client.1.smithi149.stdout:7/816: symlink d9/dc/d2e/dc9/l10b 0 2024-04-24T08:25:01.656 INFO:tasks.workunit.client.1.smithi149.stdout:5/926: mknod db/da4/d120/c13b 0 2024-04-24T08:25:01.657 INFO:tasks.workunit.client.0.smithi090.stdout:2/557: dwrite d0/d14/d17/f18 [4194304,4194304] 0 2024-04-24T08:25:01.658 INFO:tasks.workunit.client.1.smithi149.stdout:4/772: mkdir d2/d10/d91/df7 0 2024-04-24T08:25:01.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/567: symlink d8/d99/lbb 0 2024-04-24T08:25:01.665 INFO:tasks.workunit.client.0.smithi090.stdout:0/524: dread d1a/d4b/d53/d79/d3c/f54 [0,4194304] 0 2024-04-24T08:25:01.666 INFO:tasks.workunit.client.1.smithi149.stdout:4/773: creat d2/d24/d2e/d3f/ff8 x:0 0 0 2024-04-24T08:25:01.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/448: link d2/d25/d2c/l3b d2/d25/d2c/l99 0 2024-04-24T08:25:01.672 INFO:tasks.workunit.client.1.smithi149.stdout:6/689: sync 2024-04-24T08:25:01.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/827: sync 2024-04-24T08:25:01.672 INFO:tasks.workunit.client.1.smithi149.stdout:2/698: sync 2024-04-24T08:25:01.672 INFO:tasks.workunit.client.1.smithi149.stdout:1/887: sync 2024-04-24T08:25:01.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/525: dread f18 [0,4194304] 0 2024-04-24T08:25:01.681 INFO:tasks.workunit.client.0.smithi090.stdout:2/558: symlink d0/d14/d17/lc3 0 2024-04-24T08:25:01.682 INFO:tasks.workunit.client.1.smithi149.stdout:4/774: stat d2/d18/d1e/d3c/d48/d69/cde 0 2024-04-24T08:25:01.687 INFO:tasks.workunit.client.1.smithi149.stdout:2/699: stat d0/d6/db/d39/d3d/d58/cce 0 2024-04-24T08:25:01.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/472: link d0/dc/l48 d0/l8b 0 2024-04-24T08:25:01.688 INFO:tasks.workunit.client.0.smithi090.stdout:4/413: dread d3/d38/d49/f4c [0,4194304] 0 2024-04-24T08:25:01.692 INFO:tasks.workunit.client.1.smithi149.stdout:9/828: fsync d1/da/dd/f24 0 2024-04-24T08:25:01.695 INFO:tasks.workunit.client.0.smithi090.stdout:7/554: getdents db/d54 0 2024-04-24T08:25:01.695 INFO:tasks.workunit.client.1.smithi149.stdout:5/927: link db/d12/d22/l4e db/d12/d60/db2/d83/dfb/l13c 0 2024-04-24T08:25:01.695 INFO:tasks.workunit.client.1.smithi149.stdout:6/690: mknod d0/d1a/d28/d31/d4c/ceb 0 2024-04-24T08:25:01.697 INFO:tasks.workunit.client.1.smithi149.stdout:8/720: dread d1/d1f/d25/f8f [0,4194304] 0 2024-04-24T08:25:01.698 INFO:tasks.workunit.client.0.smithi090.stdout:3/449: dread d2/d15/f1d [0,4194304] 0 2024-04-24T08:25:01.699 INFO:tasks.workunit.client.1.smithi149.stdout:6/691: dread - d0/d1a/d28/d31/d4c/d8a/fb5 zero size 2024-04-24T08:25:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:9/829: dread d1/d2/d33/f5f [0,4194304] 0 2024-04-24T08:25:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:6/692: fdatasync d0/db7/fda 0 2024-04-24T08:25:01.703 INFO:tasks.workunit.client.0.smithi090.stdout:1/534: truncate d3/d5/d3e/f18 5132006 0 2024-04-24T08:25:01.704 INFO:tasks.workunit.client.0.smithi090.stdout:9/473: dwrite d0/dc/d24/d7c/d4d/f84 [0,4194304] 0 2024-04-24T08:25:01.706 INFO:tasks.workunit.client.1.smithi149.stdout:2/700: write d0/d6/db/d27/f31 [2407167,29607] 0 2024-04-24T08:25:01.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/555: mknod db/d17/d29/d32/d5a/cbf 0 2024-04-24T08:25:01.709 INFO:tasks.workunit.client.1.smithi149.stdout:5/928: creat db/d12/d1e/d85/f13d x:0 0 0 2024-04-24T08:25:01.710 INFO:tasks.workunit.client.0.smithi090.stdout:8/433: dwrite d2/da/f35 [0,4194304] 0 2024-04-24T08:25:01.710 INFO:tasks.workunit.client.0.smithi090.stdout:0/526: getdents d1a/d4b/d53/d79/d82/d9d 0 2024-04-24T08:25:01.712 INFO:tasks.workunit.client.1.smithi149.stdout:7/817: dwrite d9/dc/d7b/fa2 [0,4194304] 0 2024-04-24T08:25:01.713 INFO:tasks.workunit.client.0.smithi090.stdout:3/450: rmdir d2/df/d1a/d37 39 2024-04-24T08:25:01.718 INFO:tasks.workunit.client.0.smithi090.stdout:6/525: dwrite d8/f15 [0,4194304] 0 2024-04-24T08:25:01.718 INFO:tasks.workunit.client.0.smithi090.stdout:0/527: chown d1a/d4b/d53/d79/d58/d96 3455 1 2024-04-24T08:25:01.719 INFO:tasks.workunit.client.1.smithi149.stdout:6/693: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec 0 2024-04-24T08:25:01.720 INFO:tasks.workunit.client.1.smithi149.stdout:8/721: write d1/de/d18/d32/d65/d8b/d7d/dd1/fe4 [623240,30154] 0 2024-04-24T08:25:01.721 INFO:tasks.workunit.client.1.smithi149.stdout:5/929: symlink db/d12/d1e/d65/de8/l13e 0 2024-04-24T08:25:01.722 INFO:tasks.workunit.client.1.smithi149.stdout:1/888: getdents d0/d1b/d4b/da1 0 2024-04-24T08:25:01.723 INFO:tasks.workunit.client.0.smithi090.stdout:6/526: stat d8/dc/d19/d8e/f52 0 2024-04-24T08:25:01.724 INFO:tasks.workunit.client.0.smithi090.stdout:5/568: dread d8/da/fc [0,4194304] 0 2024-04-24T08:25:01.727 INFO:tasks.workunit.client.0.smithi090.stdout:5/569: chown d8/d25 13099168 1 2024-04-24T08:25:01.730 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:01 smithi090 ceph-mon[25016]: pgmap v23: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 193 MiB/s wr, 356 op/s 2024-04-24T08:25:01.732 INFO:tasks.workunit.client.1.smithi149.stdout:7/818: rename d9/dc/d7b/dc2/lf5 to d9/dc/d2e/dc9/l10c 0 2024-04-24T08:25:01.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/451: rmdir d2/d15/d36/d6f/d92 39 2024-04-24T08:25:01.733 INFO:tasks.workunit.client.0.smithi090.stdout:7/556: unlink f9 0 2024-04-24T08:25:01.733 INFO:tasks.workunit.client.0.smithi090.stdout:9/474: dwrite d0/dc/f2c [0,4194304] 0 2024-04-24T08:25:01.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/434: dread d2/da/d13/d64/d24/f5b [0,4194304] 0 2024-04-24T08:25:01.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/414: mkdir d3/df/d77 0 2024-04-24T08:25:01.743 INFO:tasks.workunit.client.1.smithi149.stdout:7/819: unlink c7 0 2024-04-24T08:25:01.746 INFO:tasks.workunit.client.1.smithi149.stdout:5/930: rename db/d12/d22/d45/f42 to db/d12/d60/db2/d83/dc5/f13f 0 2024-04-24T08:25:01.747 INFO:tasks.workunit.client.0.smithi090.stdout:4/415: dread d3/d38/d49/f53 [0,4194304] 0 2024-04-24T08:25:01.749 INFO:tasks.workunit.client.0.smithi090.stdout:5/570: mkdir d8/da/d3a/d2d/d37/d93/d7a/dbc 0 2024-04-24T08:25:01.749 INFO:tasks.workunit.client.0.smithi090.stdout:0/528: dread d1a/f56 [4194304,4194304] 0 2024-04-24T08:25:01.749 INFO:tasks.workunit.client.1.smithi149.stdout:5/931: chown db/d12/d2f/d95/f9e 1873008 1 2024-04-24T08:25:01.751 INFO:tasks.workunit.client.0.smithi090.stdout:5/571: read - d8/d76/fb2 zero size 2024-04-24T08:25:01.753 INFO:tasks.workunit.client.1.smithi149.stdout:9/830: write d1/da/dd/f69 [4560626,109931] 0 2024-04-24T08:25:01.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/475: mkdir d0/dc/d24/d7c/d4d/d8c 0 2024-04-24T08:25:01.756 INFO:tasks.workunit.client.0.smithi090.stdout:8/435: mkdir d2/da/d13/d60/d91 0 2024-04-24T08:25:01.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/535: creat d3/d5/da/fc1 x:0 0 0 2024-04-24T08:25:01.762 INFO:tasks.workunit.client.0.smithi090.stdout:1/536: readlink d3/d5/da/dc/l4b 0 2024-04-24T08:25:01.765 INFO:tasks.workunit.client.0.smithi090.stdout:5/572: dwrite d8/d99/f9d [0,4194304] 0 2024-04-24T08:25:01.768 INFO:tasks.workunit.client.0.smithi090.stdout:4/416: mknod d3/d25/c78 0 2024-04-24T08:25:01.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/701: dwrite d0/d6/db/f1f [0,4194304] 0 2024-04-24T08:25:01.773 INFO:tasks.workunit.client.1.smithi149.stdout:9/831: mkdir d1/da/d4a/dad/db7/d12d 0 2024-04-24T08:25:01.779 INFO:tasks.workunit.client.0.smithi090.stdout:0/529: chown d1a/d4b/d53/d79/d69/l89 30068 1 2024-04-24T08:25:01.780 INFO:tasks.workunit.client.1.smithi149.stdout:1/889: write d0/d5/d4f/d60/fc8 [3837894,41860] 0 2024-04-24T08:25:01.782 INFO:tasks.workunit.client.1.smithi149.stdout:9/832: chown d1/d2/d33/d5d/d83/dbf/fc1 26564 1 2024-04-24T08:25:01.783 INFO:tasks.workunit.client.1.smithi149.stdout:6/694: rename d0/d1a/d28/d31/d4c/d4e/f64 to d0/d1a/d28/d31/d4c/d80/d6f/dd7/fed 0 2024-04-24T08:25:01.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/530: dread d1a/d4b/d53/d79/d35/d62/f7a [0,4194304] 0 2024-04-24T08:25:01.787 INFO:tasks.workunit.client.1.smithi149.stdout:7/820: link d9/dc/d11/d83/da6/d67/cc7 d9/dc/d2e/d3c/c10d 0 2024-04-24T08:25:01.789 INFO:tasks.workunit.client.1.smithi149.stdout:8/722: dwrite d1/d1f/d25/f34 [0,4194304] 0 2024-04-24T08:25:01.791 INFO:tasks.workunit.client.0.smithi090.stdout:9/476: unlink d0/c60 0 2024-04-24T08:25:01.792 INFO:tasks.workunit.client.1.smithi149.stdout:8/723: dread - d1/d1f/d5e/d92/fee zero size 2024-04-24T08:25:01.792 INFO:tasks.workunit.client.0.smithi090.stdout:8/436: fsync d2/da/d13/d64/d50/f5d 0 2024-04-24T08:25:01.794 INFO:tasks.workunit.client.0.smithi090.stdout:8/437: chown d2/l11 0 1 2024-04-24T08:25:01.797 INFO:tasks.workunit.client.0.smithi090.stdout:1/537: read - d3/d25/d30/f7e zero size 2024-04-24T08:25:01.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/527: creat d8/d1d/d1f/d3e/d8f/f9e x:0 0 0 2024-04-24T08:25:01.800 INFO:tasks.workunit.client.0.smithi090.stdout:5/573: creat d8/da/d3a/fbd x:0 0 0 2024-04-24T08:25:01.800 INFO:tasks.workunit.client.1.smithi149.stdout:8/724: fdatasync d1/fbf 0 2024-04-24T08:25:01.801 INFO:tasks.workunit.client.1.smithi149.stdout:2/702: symlink d0/d6/db/d39/d3d/d44/d57/ld2 0 2024-04-24T08:25:01.806 INFO:tasks.workunit.client.0.smithi090.stdout:7/557: mkdir db/d17/d29/dc0 0 2024-04-24T08:25:01.818 INFO:tasks.workunit.client.0.smithi090.stdout:8/438: creat d2/da/d13/d64/d24/f92 x:0 0 0 2024-04-24T08:25:01.819 INFO:tasks.workunit.client.1.smithi149.stdout:5/932: truncate db/d12/d1e/d85/d92/d107/d7f/fd0 189892 0 2024-04-24T08:25:01.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/574: dread d8/d99/f9d [0,4194304] 0 2024-04-24T08:25:01.821 INFO:tasks.workunit.client.0.smithi090.stdout:8/439: write d2/da/f38 [1377236,47205] 0 2024-04-24T08:25:01.823 INFO:tasks.workunit.client.1.smithi149.stdout:7/821: dread d9/dc/f17 [4194304,4194304] 0 2024-04-24T08:25:01.825 INFO:tasks.workunit.client.0.smithi090.stdout:1/538: creat d3/d1d/d54/fc2 x:0 0 0 2024-04-24T08:25:01.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/539: read - d3/d1d/d54/fc2 zero size 2024-04-24T08:25:01.828 INFO:tasks.workunit.client.1.smithi149.stdout:2/703: rename d0/d6/db/d39/d3d/lc8 to d0/d6/db/d39/d78/ld3 0 2024-04-24T08:25:01.828 INFO:tasks.workunit.client.0.smithi090.stdout:3/452: dwrite d2/f24 [0,4194304] 0 2024-04-24T08:25:01.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/540: chown d3/d1d/d34/lbd 107 1 2024-04-24T08:25:01.831 INFO:tasks.workunit.client.1.smithi149.stdout:7/822: creat d9/d43/d4f/da5/f10e x:0 0 0 2024-04-24T08:25:01.833 INFO:tasks.workunit.client.1.smithi149.stdout:1/890: write d0/d5/f41 [1832660,13393] 0 2024-04-24T08:25:01.834 INFO:tasks.workunit.client.1.smithi149.stdout:9/833: getdents d1/d2/d33/d5d/d83 0 2024-04-24T08:25:01.836 INFO:tasks.workunit.client.1.smithi149.stdout:9/834: read - d1/d2/d33/f120 zero size 2024-04-24T08:25:01.836 INFO:tasks.workunit.client.0.smithi090.stdout:4/417: dread d3/d18/d1d/f46 [0,4194304] 0 2024-04-24T08:25:01.837 INFO:tasks.workunit.client.0.smithi090.stdout:4/418: stat d3/d18 0 2024-04-24T08:25:01.841 INFO:tasks.workunit.client.1.smithi149.stdout:5/933: rename db/d12/d2f/d74/df6/fda to db/d12/d2f/f140 0 2024-04-24T08:25:01.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/477: dread - d0/d2a/f38 zero size 2024-04-24T08:25:01.845 INFO:tasks.workunit.client.1.smithi149.stdout:2/704: rmdir d0/d6/db/d27/d81/d9f/dba 39 2024-04-24T08:25:01.846 INFO:tasks.workunit.client.1.smithi149.stdout:8/725: dwrite d1/d1f/d25/f80 [0,4194304] 0 2024-04-24T08:25:01.847 INFO:tasks.workunit.client.1.smithi149.stdout:7/823: mkdir d9/dc/d11/d83/da6/d67/d69/d10f 0 2024-04-24T08:25:01.850 INFO:tasks.workunit.client.1.smithi149.stdout:3/867: sync 2024-04-24T08:25:01.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/478: truncate d0/dc/d24/d2d/d49/d81/d87/f8a 133351 0 2024-04-24T08:25:01.852 INFO:tasks.workunit.client.1.smithi149.stdout:7/824: read d9/dc/d72/f76 [3597318,42140] 0 2024-04-24T08:25:01.853 INFO:tasks.workunit.client.0.smithi090.stdout:8/440: creat d2/d62/f93 x:0 0 0 2024-04-24T08:25:01.854 INFO:tasks.workunit.client.1.smithi149.stdout:2/705: read d0/d6/db/d39/d3d/d44/f53 [162420,47470] 0 2024-04-24T08:25:01.855 INFO:tasks.workunit.client.1.smithi149.stdout:1/891: mkdir d0/d1b/d10d/d121 0 2024-04-24T08:25:01.855 INFO:tasks.workunit.client.0.smithi090.stdout:6/528: mkdir d8/d1d/d2f/d85/d9f 0 2024-04-24T08:25:01.856 INFO:tasks.workunit.client.0.smithi090.stdout:8/441: fdatasync d2/da/d13/d34/f6f 0 2024-04-24T08:25:01.857 INFO:tasks.workunit.client.0.smithi090.stdout:6/529: chown d8/d1d/d1f/d3e/d44 231 1 2024-04-24T08:25:01.859 INFO:tasks.workunit.client.1.smithi149.stdout:3/868: creat d0/d5/da/d19/d80/f126 x:0 0 0 2024-04-24T08:25:01.859 INFO:tasks.workunit.client.1.smithi149.stdout:9/835: mknod d1/de/c12e 0 2024-04-24T08:25:01.862 INFO:tasks.workunit.client.0.smithi090.stdout:9/479: mknod d0/dc/d24/d2d/d49/d62/c8d 0 2024-04-24T08:25:01.862 INFO:tasks.workunit.client.1.smithi149.stdout:3/869: write d0/d5/da/d5a/d63/d6a/f7c [1756145,20789] 0 2024-04-24T08:25:01.867 INFO:tasks.workunit.client.0.smithi090.stdout:5/575: write d8/da/d18/f1f [1175920,86525] 0 2024-04-24T08:25:01.868 INFO:tasks.workunit.client.0.smithi090.stdout:7/558: dread - db/d17/d29/d32/d5a/f8d zero size 2024-04-24T08:25:01.869 INFO:tasks.workunit.client.1.smithi149.stdout:7/825: creat d9/dc/d11/d83/da6/d67/d69/dad/f110 x:0 0 0 2024-04-24T08:25:01.870 INFO:tasks.workunit.client.0.smithi090.stdout:4/419: symlink d3/df/d77/l79 0 2024-04-24T08:25:01.871 INFO:tasks.workunit.client.0.smithi090.stdout:6/530: mknod d8/d1d/d1f/ca0 0 2024-04-24T08:25:01.872 INFO:tasks.workunit.client.1.smithi149.stdout:7/826: stat d9/dc/d11/d26/f28 0 2024-04-24T08:25:01.874 INFO:tasks.workunit.client.0.smithi090.stdout:4/420: write d3/d38/d49/d6f/f51 [3136441,52882] 0 2024-04-24T08:25:01.876 INFO:tasks.workunit.client.1.smithi149.stdout:2/706: dread d0/f55 [0,4194304] 0 2024-04-24T08:25:01.877 INFO:tasks.workunit.client.0.smithi090.stdout:3/453: creat d2/d15/d36/d6f/f9a x:0 0 0 2024-04-24T08:25:01.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/531: write d8/d1d/d2f/f3c [73770,9094] 0 2024-04-24T08:25:01.879 INFO:tasks.workunit.client.1.smithi149.stdout:8/726: getdents d1/de/d18/d32/d65 0 2024-04-24T08:25:01.880 INFO:tasks.workunit.client.1.smithi149.stdout:9/836: symlink d1/d2/d15/d1c/dca/d128/l12f 0 2024-04-24T08:25:01.881 INFO:tasks.workunit.client.0.smithi090.stdout:8/442: creat d2/da/d13/f94 x:0 0 0 2024-04-24T08:25:01.884 INFO:tasks.workunit.client.1.smithi149.stdout:9/837: readlink d1/d2/d33/d5d/l9f 0 2024-04-24T08:25:01.884 INFO:tasks.workunit.client.1.smithi149.stdout:5/934: dwrite db/d12/d2f/d74/f97 [0,4194304] 0 2024-04-24T08:25:01.884 INFO:tasks.workunit.client.1.smithi149.stdout:2/707: write d0/d6/db/d39/d78/d7c/f82 [405768,123241] 0 2024-04-24T08:25:01.889 INFO:tasks.workunit.client.1.smithi149.stdout:5/935: read db/d12/f69 [279325,97440] 0 2024-04-24T08:25:01.889 INFO:tasks.workunit.client.1.smithi149.stdout:2/708: write d0/d6/db/f10 [7356721,130189] 0 2024-04-24T08:25:01.892 INFO:tasks.workunit.client.1.smithi149.stdout:7/827: symlink d9/dc/d11/d83/da6/d67/d69/d10f/l111 0 2024-04-24T08:25:01.893 INFO:tasks.workunit.client.0.smithi090.stdout:1/541: dread d3/d5/da/dc/f3a [0,4194304] 0 2024-04-24T08:25:01.895 INFO:tasks.workunit.client.0.smithi090.stdout:1/542: write d3/d68/d7f/f9e [1571557,90142] 0 2024-04-24T08:25:01.899 INFO:tasks.workunit.client.0.smithi090.stdout:8/443: symlink d2/d62/d71/l95 0 2024-04-24T08:25:01.903 INFO:tasks.workunit.client.1.smithi149.stdout:9/838: creat d1/d2/d33/d60/d10b/f130 x:0 0 0 2024-04-24T08:25:01.903 INFO:tasks.workunit.client.1.smithi149.stdout:3/870: write d0/d5/d53/d34/d3a/d93/ff1 [461818,77030] 0 2024-04-24T08:25:01.905 INFO:tasks.workunit.client.0.smithi090.stdout:7/559: write db/d17/d29/d32/d5a/d75/f8a [590154,13695] 0 2024-04-24T08:25:01.906 INFO:tasks.workunit.client.0.smithi090.stdout:3/454: chown d2/df/d1a/d37/d59/d87/l91 279602 1 2024-04-24T08:25:01.906 INFO:tasks.workunit.client.1.smithi149.stdout:2/709: chown d0/d6/l88 66842721 1 2024-04-24T08:25:01.907 INFO:tasks.workunit.client.1.smithi149.stdout:7/828: dread d9/dc/de/f33 [4194304,4194304] 0 2024-04-24T08:25:01.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/480: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:25:01.910 INFO:tasks.workunit.client.1.smithi149.stdout:7/829: chown d9/d43/d4f/d66/c75 2470 1 2024-04-24T08:25:01.912 INFO:tasks.workunit.client.0.smithi090.stdout:5/576: truncate d8/da/d3a/d2d/f82 857523 0 2024-04-24T08:25:01.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/455: dread d2/df/d21/f30 [0,4194304] 0 2024-04-24T08:25:01.916 INFO:tasks.workunit.client.0.smithi090.stdout:1/543: dwrite d3/d1d/d34/d73/f99 [0,4194304] 0 2024-04-24T08:25:01.917 INFO:tasks.workunit.client.0.smithi090.stdout:3/456: chown d2/f24 610606740 1 2024-04-24T08:25:01.919 INFO:tasks.workunit.client.0.smithi090.stdout:6/532: rename d8/dc/f3a to d8/dc/fa1 0 2024-04-24T08:25:01.920 INFO:tasks.workunit.client.1.smithi149.stdout:3/871: mknod d0/d5/da/d5a/d63/d6a/dd9/c127 0 2024-04-24T08:25:01.921 INFO:tasks.workunit.client.0.smithi090.stdout:7/560: dread db/d17/f3a [0,4194304] 0 2024-04-24T08:25:01.927 INFO:tasks.workunit.client.0.smithi090.stdout:9/481: dwrite d0/f2 [4194304,4194304] 0 2024-04-24T08:25:01.930 INFO:tasks.workunit.client.0.smithi090.stdout:4/421: link d3/df/d77/l79 d3/d38/d6b/l7a 0 2024-04-24T08:25:01.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/577: write d8/da/d14/d2f/d81/fb8 [841350,88875] 0 2024-04-24T08:25:01.941 INFO:tasks.workunit.client.1.smithi149.stdout:5/936: dwrite db/d12/d1e/fc1 [0,4194304] 0 2024-04-24T08:25:01.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/444: symlink d2/da/d13/d64/l96 0 2024-04-24T08:25:01.949 INFO:tasks.workunit.client.1.smithi149.stdout:2/710: mkdir d0/d6/db/d27/d64/d66/db1/dcf/dd4 0 2024-04-24T08:25:01.950 INFO:tasks.workunit.client.0.smithi090.stdout:1/544: symlink d3/d25/d30/lc3 0 2024-04-24T08:25:01.953 INFO:tasks.workunit.client.1.smithi149.stdout:8/727: getdents d1/de/d18/d32/d65/d8b/d7d/ddc 0 2024-04-24T08:25:01.954 INFO:tasks.workunit.client.1.smithi149.stdout:3/872: getdents d0/d88/df2 0 2024-04-24T08:25:01.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/830: link d9/d43/cda d9/dc/d11/d38/dac/c112 0 2024-04-24T08:25:01.958 INFO:tasks.workunit.client.0.smithi090.stdout:5/578: dread d8/f31 [0,4194304] 0 2024-04-24T08:25:01.958 INFO:tasks.workunit.client.1.smithi149.stdout:2/711: mkdir d0/d6/db/d28/dd5 0 2024-04-24T08:25:01.960 INFO:tasks.workunit.client.1.smithi149.stdout:8/728: rmdir d1/d1f/d25 39 2024-04-24T08:25:01.962 INFO:tasks.workunit.client.1.smithi149.stdout:8/729: fsync d1/d1f/f8d 0 2024-04-24T08:25:01.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/457: dread d2/df/d1a/d37/f47 [0,4194304] 0 2024-04-24T08:25:01.964 INFO:tasks.workunit.client.0.smithi090.stdout:9/482: creat d0/d2a/f8e x:0 0 0 2024-04-24T08:25:01.965 INFO:tasks.workunit.client.1.smithi149.stdout:5/937: symlink db/d12/d1e/d65/l141 0 2024-04-24T08:25:01.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/545: dread d3/d5/da/f82 [0,4194304] 0 2024-04-24T08:25:01.971 INFO:tasks.workunit.client.1.smithi149.stdout:8/730: fdatasync d1/d1f/d5e/f8c 0 2024-04-24T08:25:01.971 INFO:tasks.workunit.client.0.smithi090.stdout:6/533: dwrite d8/fa [4194304,4194304] 0 2024-04-24T08:25:01.973 INFO:tasks.workunit.client.0.smithi090.stdout:7/561: mknod db/d17/d29/d38/d99/cc1 0 2024-04-24T08:25:01.975 INFO:tasks.workunit.client.1.smithi149.stdout:5/938: fdatasync db/d12/d2f/d74/dc6/fe9 0 2024-04-24T08:25:01.978 INFO:tasks.workunit.client.0.smithi090.stdout:9/483: symlink d0/d5e/l8f 0 2024-04-24T08:25:01.978 INFO:tasks.workunit.client.1.smithi149.stdout:3/873: write d0/f11 [4380131,96247] 0 2024-04-24T08:25:01.980 INFO:tasks.workunit.client.1.smithi149.stdout:8/731: readlink d1/l6 0 2024-04-24T08:25:01.981 INFO:tasks.workunit.client.0.smithi090.stdout:3/458: mkdir d2/df/d1a/d37/d53/d60/d9b 0 2024-04-24T08:25:01.981 INFO:tasks.workunit.client.0.smithi090.stdout:9/484: chown d0/dc/c11 99810778 1 2024-04-24T08:25:01.982 INFO:tasks.workunit.client.1.smithi149.stdout:7/831: creat d9/f113 x:0 0 0 2024-04-24T08:25:01.983 INFO:tasks.workunit.client.1.smithi149.stdout:3/874: readlink d0/d5/da/d19/d1b/d72/ldd 0 2024-04-24T08:25:01.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/832: stat d9/d43/d64 0 2024-04-24T08:25:01.984 INFO:tasks.workunit.client.0.smithi090.stdout:1/546: mkdir d3/d5/da/d97/d9c/dc4 0 2024-04-24T08:25:01.986 INFO:tasks.workunit.client.1.smithi149.stdout:2/712: creat d0/d6/db/d39/d3d/fd6 x:0 0 0 2024-04-24T08:25:01.987 INFO:tasks.workunit.client.0.smithi090.stdout:4/422: dwrite d3/d38/d49/d6f/f72 [0,4194304] 0 2024-04-24T08:25:01.988 INFO:tasks.workunit.client.0.smithi090.stdout:6/534: fdatasync d8/d1d/d1f/d3e/f5c 0 2024-04-24T08:25:01.989 INFO:tasks.workunit.client.1.smithi149.stdout:5/939: mknod db/d12/d1e/d85/d92/da6/c142 0 2024-04-24T08:25:01.992 INFO:tasks.workunit.client.0.smithi090.stdout:8/445: mkdir d2/d97 0 2024-04-24T08:25:01.995 INFO:tasks.workunit.client.1.smithi149.stdout:3/875: rename d0/d5/da/d5a/d63/d6a/dd9 to d0/d5/d1a/d128 0 2024-04-24T08:25:01.996 INFO:tasks.workunit.client.0.smithi090.stdout:7/562: creat db/d17/d39/d59/fc2 x:0 0 0 2024-04-24T08:25:02.000 INFO:tasks.workunit.client.1.smithi149.stdout:5/940: rmdir db/d35/d123 39 2024-04-24T08:25:02.002 INFO:tasks.workunit.client.0.smithi090.stdout:9/485: symlink d0/dc/d24/d7c/d4d/l90 0 2024-04-24T08:25:02.003 INFO:tasks.workunit.client.1.smithi149.stdout:7/833: symlink d9/dc/d11/d38/dac/d103/l114 0 2024-04-24T08:25:02.005 INFO:tasks.workunit.client.1.smithi149.stdout:7/834: chown d9/d43/d4f/da5 195477 1 2024-04-24T08:25:02.005 INFO:tasks.workunit.client.0.smithi090.stdout:1/547: mkdir d3/d25/d30/dc5 0 2024-04-24T08:25:02.008 INFO:tasks.workunit.client.0.smithi090.stdout:6/535: read d8/d1d/d1f/d3e/f6a [356381,8592] 0 2024-04-24T08:25:02.010 INFO:tasks.workunit.client.1.smithi149.stdout:2/713: link d0/d6/db/d39/d3d/fb8 d0/d6/db/d27/d64/d66/db1/dc9/fd7 0 2024-04-24T08:25:02.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/446: rename d2/da/d13/l25 to d2/da/d13/d64/d24/l98 0 2024-04-24T08:25:02.015 INFO:tasks.workunit.client.1.smithi149.stdout:5/941: mknod db/da4/c143 0 2024-04-24T08:25:02.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/876: write d0/d5/da/d5a/f103 [861623,77334] 0 2024-04-24T08:25:02.022 INFO:tasks.workunit.client.1.smithi149.stdout:8/732: dwrite d1/de/d18/d32/d65/d8b/d7d/dd1/fe2 [0,4194304] 0 2024-04-24T08:25:02.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:01 smithi149 ceph-mon[31282]: pgmap v23: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 193 MiB/s wr, 356 op/s 2024-04-24T08:25:02.028 INFO:tasks.workunit.client.1.smithi149.stdout:7/835: dwrite d9/dc/de/f33 [8388608,4194304] 0 2024-04-24T08:25:02.038 INFO:tasks.workunit.client.1.smithi149.stdout:2/714: fdatasync d0/d6/db/d39/d3d/d58/d98/faf 0 2024-04-24T08:25:02.040 INFO:tasks.workunit.client.1.smithi149.stdout:5/942: dwrite db/d12/d1e/f3c [0,4194304] 0 2024-04-24T08:25:02.042 INFO:tasks.workunit.client.1.smithi149.stdout:3/877: mkdir d0/d5/da/d5a/d63/d6a/d129 0 2024-04-24T08:25:02.048 INFO:tasks.workunit.client.1.smithi149.stdout:5/943: mknod db/d12/d60/c144 0 2024-04-24T08:25:02.051 INFO:tasks.workunit.client.1.smithi149.stdout:3/878: unlink d0/d5/da/d5a/f103 0 2024-04-24T08:25:02.052 INFO:tasks.workunit.client.1.smithi149.stdout:2/715: symlink d0/d6/db/d27/d81/db4/ld8 0 2024-04-24T08:25:02.055 INFO:tasks.workunit.client.1.smithi149.stdout:7/836: rename d9/dc/d2e/cbb to d9/d43/d95/c115 0 2024-04-24T08:25:02.057 INFO:tasks.workunit.client.1.smithi149.stdout:8/733: write d1/de/d18/fae [1053631,24662] 0 2024-04-24T08:25:02.058 INFO:tasks.workunit.client.1.smithi149.stdout:5/944: truncate db/d12/d60/fd7 208898 0 2024-04-24T08:25:02.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/879: truncate d0/d26/d3c/db1/fb6 702309 0 2024-04-24T08:25:02.064 INFO:tasks.workunit.client.1.smithi149.stdout:7/837: readlink d9/dc/d11/d83/da6/d67/lee 0 2024-04-24T08:25:02.064 INFO:tasks.workunit.client.1.smithi149.stdout:5/945: symlink db/d12/d2f/d95/l145 0 2024-04-24T08:25:02.068 INFO:tasks.workunit.client.1.smithi149.stdout:5/946: chown db/d12/d2f/d74/c88 28640156 1 2024-04-24T08:25:02.068 INFO:tasks.workunit.client.1.smithi149.stdout:7/838: write d9/dc/d11/d38/fe8 [3425285,61825] 0 2024-04-24T08:25:02.068 INFO:tasks.workunit.client.1.smithi149.stdout:8/734: rename d1/de/d18/d32/d63/la9 to d1/d1f/d25/lf1 0 2024-04-24T08:25:02.071 INFO:tasks.workunit.client.1.smithi149.stdout:8/735: chown d1/de/ca4 112907224 1 2024-04-24T08:25:02.073 INFO:tasks.workunit.client.1.smithi149.stdout:8/736: chown d1/de/d18/l60 150309675 1 2024-04-24T08:25:02.073 INFO:tasks.workunit.client.1.smithi149.stdout:5/947: creat db/d23/f146 x:0 0 0 2024-04-24T08:25:02.074 INFO:tasks.workunit.client.1.smithi149.stdout:7/839: readlink d9/dc/d11/d83/da6/d67/lb1 0 2024-04-24T08:25:02.077 INFO:tasks.workunit.client.1.smithi149.stdout:8/737: unlink d1/d1f/d22/cc2 0 2024-04-24T08:25:02.079 INFO:tasks.workunit.client.1.smithi149.stdout:8/738: fsync d1/de/d18/d32/d63/d76/fe1 0 2024-04-24T08:25:02.080 INFO:tasks.workunit.client.1.smithi149.stdout:7/840: rmdir d9/dc/d11/d83/da6/d67/d69 39 2024-04-24T08:25:02.085 INFO:tasks.workunit.client.1.smithi149.stdout:7/841: getdents d9/dc/d11/d84 0 2024-04-24T08:25:02.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/729: sync 2024-04-24T08:25:02.089 INFO:tasks.workunit.client.1.smithi149.stdout:5/948: write db/d12/d1e/d85/d92/d107/d7f/fd0 [453241,55123] 0 2024-04-24T08:25:02.089 INFO:tasks.workunit.client.1.smithi149.stdout:0/730: stat d2/d9/df/d40/fa2 0 2024-04-24T08:25:02.092 INFO:tasks.workunit.client.1.smithi149.stdout:3/880: dwrite d0/d5/da/d5e/fe8 [0,4194304] 0 2024-04-24T08:25:02.096 INFO:tasks.workunit.client.1.smithi149.stdout:2/716: dread d0/d6/db/d28/d7f/d8f/fcc [0,4194304] 0 2024-04-24T08:25:02.096 INFO:tasks.workunit.client.1.smithi149.stdout:7/842: creat d9/dc/d7b/dfc/f116 x:0 0 0 2024-04-24T08:25:02.099 INFO:tasks.workunit.client.1.smithi149.stdout:5/949: symlink db/d12/d1e/l147 0 2024-04-24T08:25:02.102 INFO:tasks.workunit.client.1.smithi149.stdout:3/881: mkdir d0/d88/df2/d12a 0 2024-04-24T08:25:02.102 INFO:tasks.workunit.client.1.smithi149.stdout:8/739: dwrite d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:25:02.108 INFO:tasks.workunit.client.1.smithi149.stdout:2/717: truncate d0/d6/db/fbd 96433 0 2024-04-24T08:25:02.114 INFO:tasks.workunit.client.1.smithi149.stdout:2/718: readlink d0/d6/db/l32 0 2024-04-24T08:25:02.117 INFO:tasks.workunit.client.0.smithi090.stdout:2/559: sync 2024-04-24T08:25:02.117 INFO:tasks.workunit.client.0.smithi090.stdout:0/531: sync 2024-04-24T08:25:02.119 INFO:tasks.workunit.client.0.smithi090.stdout:7/563: symlink db/d17/d29/d32/d5a/d75/lc3 0 2024-04-24T08:25:02.120 INFO:tasks.workunit.client.0.smithi090.stdout:3/459: fdatasync d2/df/d5b/f8e 0 2024-04-24T08:25:02.121 INFO:tasks.workunit.client.0.smithi090.stdout:1/548: truncate d3/d5/da/dc/f3a 3621632 0 2024-04-24T08:25:02.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/536: rmdir d8 39 2024-04-24T08:25:02.126 INFO:tasks.workunit.client.0.smithi090.stdout:5/579: link d8/d99/lbb d8/d59/lbe 0 2024-04-24T08:25:02.126 INFO:tasks.workunit.client.1.smithi149.stdout:0/731: dwrite d2/d9/df/d2f/d6c/dc1/d4e/fa0 [0,4194304] 0 2024-04-24T08:25:02.127 INFO:tasks.workunit.client.0.smithi090.stdout:2/560: dread d0/d2c/f43 [0,4194304] 0 2024-04-24T08:25:02.129 INFO:tasks.workunit.client.1.smithi149.stdout:8/740: rename d1/d1f/d25/c4e to d1/d1f/d5e/cf2 0 2024-04-24T08:25:02.130 INFO:tasks.workunit.client.1.smithi149.stdout:7/843: link d9/dc/d2e/dc9/l10c d9/dc/d2e/dc9/l117 0 2024-04-24T08:25:02.143 INFO:tasks.workunit.client.0.smithi090.stdout:7/564: creat db/d17/d29/d32/d5a/d75/fc4 x:0 0 0 2024-04-24T08:25:02.144 INFO:tasks.workunit.client.1.smithi149.stdout:5/950: write db/d12/d1e/d65/fb4 [1857523,122549] 0 2024-04-24T08:25:02.144 INFO:tasks.workunit.client.0.smithi090.stdout:9/486: truncate d0/d19/f46 2300247 0 2024-04-24T08:25:02.145 INFO:tasks.workunit.client.1.smithi149.stdout:3/882: write d0/d26/fee [4347629,102167] 0 2024-04-24T08:25:02.145 INFO:tasks.workunit.client.1.smithi149.stdout:0/732: mknod d2/d9/df/d2f/ceb 0 2024-04-24T08:25:02.148 INFO:tasks.workunit.client.0.smithi090.stdout:8/447: mknod d2/d97/c99 0 2024-04-24T08:25:02.149 INFO:tasks.workunit.client.1.smithi149.stdout:3/883: write d0/d5/d53/d34/f35 [4210729,33584] 0 2024-04-24T08:25:02.151 INFO:tasks.workunit.client.0.smithi090.stdout:3/460: fsync d2/f17 0 2024-04-24T08:25:02.153 INFO:tasks.workunit.client.0.smithi090.stdout:6/537: truncate d8/d1d/d1f/d3e/f5c 8506792 0 2024-04-24T08:25:02.167 INFO:tasks.workunit.client.0.smithi090.stdout:2/561: dwrite d0/f9e [0,4194304] 0 2024-04-24T08:25:02.169 INFO:tasks.workunit.client.1.smithi149.stdout:8/741: dread d1/de/d18/d32/d63/d76/f9a [0,4194304] 0 2024-04-24T08:25:02.171 INFO:tasks.workunit.client.0.smithi090.stdout:8/448: dwrite d2/d62/f93 [0,4194304] 0 2024-04-24T08:25:02.174 INFO:tasks.workunit.client.0.smithi090.stdout:8/449: dread - d2/d62/d6b/f90 zero size 2024-04-24T08:25:02.189 INFO:tasks.workunit.client.1.smithi149.stdout:8/742: dwrite d1/de/d18/d32/d63/d76/fe1 [0,4194304] 0 2024-04-24T08:25:02.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/423: sync 2024-04-24T08:25:02.297 INFO:tasks.workunit.client.0.smithi090.stdout:7/565: symlink db/d54/d83/lc5 0 2024-04-24T08:25:02.300 INFO:tasks.workunit.client.1.smithi149.stdout:3/884: symlink d0/d5/da/d19/d1b/d22/d38/l12b 0 2024-04-24T08:25:02.311 INFO:tasks.workunit.client.0.smithi090.stdout:3/461: mknod d2/df/d1a/d37/d53/c9c 0 2024-04-24T08:25:02.320 INFO:tasks.workunit.client.1.smithi149.stdout:7/844: write d9/dc/d11/d83/da6/d67/d69/fe9 [1584879,18744] 0 2024-04-24T08:25:02.320 INFO:tasks.workunit.client.0.smithi090.stdout:2/562: mknod d0/d14/cc4 0 2024-04-24T08:25:02.321 INFO:tasks.workunit.client.1.smithi149.stdout:8/743: symlink d1/lf3 0 2024-04-24T08:25:02.326 INFO:tasks.workunit.client.1.smithi149.stdout:5/951: dwrite db/d12/d2f/f57 [0,4194304] 0 2024-04-24T08:25:02.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/424: fsync d3/d24/f44 0 2024-04-24T08:25:02.332 INFO:tasks.workunit.client.0.smithi090.stdout:2/563: chown d0/d7/d67/d7f/d31/d46/d60/fa4 1 1 2024-04-24T08:25:02.337 INFO:tasks.workunit.client.0.smithi090.stdout:0/532: link d1a/d4b/d53/f8b d1a/d4b/d53/d79/fa9 0 2024-04-24T08:25:02.342 INFO:tasks.workunit.client.0.smithi090.stdout:4/425: chown d3/d18/d5c/f3d 273 1 2024-04-24T08:25:02.343 INFO:tasks.workunit.client.1.smithi149.stdout:7/845: mknod d9/dc/d11/c118 0 2024-04-24T08:25:02.345 INFO:tasks.workunit.client.0.smithi090.stdout:5/580: dwrite d8/f1c [0,4194304] 0 2024-04-24T08:25:02.346 INFO:tasks.workunit.client.1.smithi149.stdout:3/885: getdents d0/d5/da 0 2024-04-24T08:25:02.349 INFO:tasks.workunit.client.1.smithi149.stdout:5/952: truncate db/d12/d2f/d74/dc6/fcd 112941 0 2024-04-24T08:25:02.349 INFO:tasks.workunit.client.0.smithi090.stdout:2/564: fdatasync d0/d7/d67/d7f/d31/f56 0 2024-04-24T08:25:02.351 INFO:tasks.workunit.client.1.smithi149.stdout:7/846: mknod d9/dc/d2e/d51/c119 0 2024-04-24T08:25:02.353 INFO:tasks.workunit.client.1.smithi149.stdout:3/886: fdatasync d0/d5/d1a/d70/fba 0 2024-04-24T08:25:02.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/487: dwrite d0/dc/d24/d2d/f47 [4194304,4194304] 0 2024-04-24T08:25:02.354 INFO:tasks.workunit.client.1.smithi149.stdout:3/887: chown d0/d88 1970540 1 2024-04-24T08:25:02.356 INFO:tasks.workunit.client.1.smithi149.stdout:3/888: readlink d0/d5/d53/dd6/le4 0 2024-04-24T08:25:02.358 INFO:tasks.workunit.client.1.smithi149.stdout:5/953: dread - db/da4/f128 zero size 2024-04-24T08:25:02.360 INFO:tasks.workunit.client.0.smithi090.stdout:5/581: write d8/da/d3a/d2d/d37/d93/d7a/d50/f68 [1020163,116202] 0 2024-04-24T08:25:02.362 INFO:tasks.workunit.client.0.smithi090.stdout:3/462: dwrite d2/d15/d36/d6f/d7a/f8f [0,4194304] 0 2024-04-24T08:25:02.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/775: sync 2024-04-24T08:25:02.373 INFO:tasks.workunit.client.1.smithi149.stdout:8/744: dwrite d1/d1f/d22/f71 [0,4194304] 0 2024-04-24T08:25:02.379 INFO:tasks.workunit.client.1.smithi149.stdout:4/776: write d2/d18/d1e/d3c/d48/f87 [3348091,66175] 0 2024-04-24T08:25:02.381 INFO:tasks.workunit.client.0.smithi090.stdout:3/463: dread d2/d15/d36/d6f/d7a/f8f [0,4194304] 0 2024-04-24T08:25:02.390 INFO:tasks.workunit.client.1.smithi149.stdout:8/745: symlink d1/d1f/d5e/d92/lf4 0 2024-04-24T08:25:02.392 INFO:tasks.workunit.client.0.smithi090.stdout:9/488: dread d0/dc/f1f [0,4194304] 0 2024-04-24T08:25:02.395 INFO:tasks.workunit.client.1.smithi149.stdout:8/746: write d1/de/d18/d43/fb6 [8716049,96795] 0 2024-04-24T08:25:02.395 INFO:tasks.workunit.client.0.smithi090.stdout:9/489: write d0/dc/d24/d7c/d4d/d32/f3c [335347,110962] 0 2024-04-24T08:25:02.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/549: link d3/d5/c15 d3/d1d/d34/d73/cc6 0 2024-04-24T08:25:02.398 INFO:tasks.workunit.client.0.smithi090.stdout:1/550: write d3/d1d/f55 [2380268,2577] 0 2024-04-24T08:25:02.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/538: mkdir d8/d1d/d1f/d3e/d44/d90/da2 0 2024-04-24T08:25:02.403 INFO:tasks.workunit.client.1.smithi149.stdout:7/847: dwrite d9/dc/f6b [0,4194304] 0 2024-04-24T08:25:02.404 INFO:tasks.workunit.client.0.smithi090.stdout:6/539: fdatasync d8/d1d/d2f/f88 0 2024-04-24T08:25:02.410 INFO:tasks.workunit.client.0.smithi090.stdout:0/533: rmdir d1a 39 2024-04-24T08:25:02.412 INFO:tasks.workunit.client.1.smithi149.stdout:3/889: dwrite d0/d5/d1a/f1e [0,4194304] 0 2024-04-24T08:25:02.423 INFO:tasks.workunit.client.0.smithi090.stdout:5/582: rename d8/lb to d8/da/d18/d21/lbf 0 2024-04-24T08:25:02.426 INFO:tasks.workunit.client.0.smithi090.stdout:5/583: stat d8/da/d3a/d2d/d37/d93 0 2024-04-24T08:25:02.426 INFO:tasks.workunit.client.1.smithi149.stdout:5/954: write db/d12/d2f/d74/dc6/fcd [722528,47575] 0 2024-04-24T08:25:02.427 INFO:tasks.workunit.client.0.smithi090.stdout:7/566: dread db/d17/d29/d32/f61 [0,4194304] 0 2024-04-24T08:25:02.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/567: write db/f12 [745431,62162] 0 2024-04-24T08:25:02.431 INFO:tasks.workunit.client.0.smithi090.stdout:9/490: creat d0/d5e/d78/f91 x:0 0 0 2024-04-24T08:25:02.434 INFO:tasks.workunit.client.0.smithi090.stdout:6/540: creat d8/d1d/fa3 x:0 0 0 2024-04-24T08:25:02.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/426: unlink d3/d18/d1d/l3b 0 2024-04-24T08:25:02.455 INFO:tasks.workunit.client.1.smithi149.stdout:4/777: rename d2/d18/d1e/c96 to d2/d24/d2e/d3f/d94/cf9 0 2024-04-24T08:25:02.456 INFO:tasks.workunit.client.1.smithi149.stdout:5/955: creat db/d12/d1e/d12d/f148 x:0 0 0 2024-04-24T08:25:02.458 INFO:tasks.workunit.client.0.smithi090.stdout:3/464: mkdir d2/d15/d36/d6f/d9d 0 2024-04-24T08:25:02.458 INFO:tasks.workunit.client.1.smithi149.stdout:3/890: symlink d0/d5/da/d19/d1b/l12c 0 2024-04-24T08:25:02.460 INFO:tasks.workunit.client.0.smithi090.stdout:8/450: getdents d2/d97 0 2024-04-24T08:25:02.460 INFO:tasks.workunit.client.0.smithi090.stdout:9/491: chown d0/l8b 9 1 2024-04-24T08:25:02.465 INFO:tasks.workunit.client.0.smithi090.stdout:8/451: symlink d2/d62/l9a 0 2024-04-24T08:25:02.466 INFO:tasks.workunit.client.1.smithi149.stdout:3/891: rmdir d0/d26/d3c/db1 39 2024-04-24T08:25:02.468 INFO:tasks.workunit.client.1.smithi149.stdout:5/956: mknod db/d35/c149 0 2024-04-24T08:25:02.469 INFO:tasks.workunit.client.0.smithi090.stdout:3/465: symlink d2/df/d1a/d37/d4b/l9e 0 2024-04-24T08:25:02.471 INFO:tasks.workunit.client.1.smithi149.stdout:3/892: mkdir d0/d5/da/d59/d12d 0 2024-04-24T08:25:02.473 INFO:tasks.workunit.client.1.smithi149.stdout:3/893: fdatasync d0/d5/d53/d34/d3a/d93/ff1 0 2024-04-24T08:25:02.473 INFO:tasks.workunit.client.1.smithi149.stdout:5/957: dread db/d12/d60/db2/d83/dc8/ffe [0,4194304] 0 2024-04-24T08:25:02.478 INFO:tasks.workunit.client.1.smithi149.stdout:5/958: mkdir db/d12/d60/db2/d83/dc5/d14a 0 2024-04-24T08:25:02.478 INFO:tasks.workunit.client.0.smithi090.stdout:7/568: rename db/d54/c6e to db/d17/d29/cc6 0 2024-04-24T08:25:02.479 INFO:tasks.workunit.client.0.smithi090.stdout:3/466: stat d2/d25/d2c/l41 0 2024-04-24T08:25:02.479 INFO:tasks.workunit.client.0.smithi090.stdout:6/541: creat d8/d1d/d1f/d3e/d44/d90/da2/fa4 x:0 0 0 2024-04-24T08:25:02.480 INFO:tasks.workunit.client.1.smithi149.stdout:3/894: truncate d0/d5/da/d19/d80/dac/fc5 129886 0 2024-04-24T08:25:02.480 INFO:tasks.workunit.client.1.smithi149.stdout:5/959: chown db/d12/d22/d45/f116 472239174 1 2024-04-24T08:25:02.481 INFO:tasks.workunit.client.0.smithi090.stdout:7/569: stat db/d17/l93 0 2024-04-24T08:25:02.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/492: mkdir d0/d69/d7d/d5b/d92 0 2024-04-24T08:25:02.482 INFO:tasks.workunit.client.0.smithi090.stdout:2/565: dread d0/d2e/f49 [0,4194304] 0 2024-04-24T08:25:02.482 INFO:tasks.workunit.client.0.smithi090.stdout:8/452: mknod d2/c9b 0 2024-04-24T08:25:02.486 INFO:tasks.workunit.client.0.smithi090.stdout:7/570: rmdir db/d90 39 2024-04-24T08:25:02.487 INFO:tasks.workunit.client.1.smithi149.stdout:5/960: mknod db/d12/d1e/d11d/c14b 0 2024-04-24T08:25:02.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/427: dwrite d3/f32 [0,4194304] 0 2024-04-24T08:25:02.491 INFO:tasks.workunit.client.0.smithi090.stdout:3/467: symlink d2/d15/d36/d6f/d9d/l9f 0 2024-04-24T08:25:02.491 INFO:tasks.workunit.client.0.smithi090.stdout:4/428: dread - d3/d18/d5c/f74 zero size 2024-04-24T08:25:02.493 INFO:tasks.workunit.client.0.smithi090.stdout:2/566: rename d0/d14/f58 to d0/d7/d28/daa/db1/fc5 0 2024-04-24T08:25:02.493 INFO:tasks.workunit.client.1.smithi149.stdout:5/961: truncate db/d12/d2f/f36 1940236 0 2024-04-24T08:25:02.495 INFO:tasks.workunit.client.0.smithi090.stdout:9/493: dread d0/dc/d24/d2d/d49/d62/f51 [0,4194304] 0 2024-04-24T08:25:02.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/468: read d2/df/d1a/d37/d53/f74 [1127136,1562] 0 2024-04-24T08:25:02.501 INFO:tasks.workunit.client.0.smithi090.stdout:2/567: creat d0/d7/d6f/fc6 x:0 0 0 2024-04-24T08:25:02.502 INFO:tasks.workunit.client.1.smithi149.stdout:5/962: fdatasync db/d12/d1e/d85/d92/d107/feb 0 2024-04-24T08:25:02.507 INFO:tasks.workunit.client.0.smithi090.stdout:9/494: creat d0/dc/d24/d2d/f93 x:0 0 0 2024-04-24T08:25:02.511 INFO:tasks.workunit.client.0.smithi090.stdout:6/542: dwrite d8/d1d/d29/f40 [0,4194304] 0 2024-04-24T08:25:02.512 INFO:tasks.workunit.client.1.smithi149.stdout:3/895: dread d0/d5/f16 [0,4194304] 0 2024-04-24T08:25:02.513 INFO:tasks.workunit.client.0.smithi090.stdout:7/571: getdents db/d17/d29/d32/d5a/d75 0 2024-04-24T08:25:02.517 INFO:tasks.workunit.client.0.smithi090.stdout:8/453: dwrite d2/da/d13/f5f [0,4194304] 0 2024-04-24T08:25:02.523 INFO:tasks.workunit.client.0.smithi090.stdout:1/551: dwrite d3/f41 [0,4194304] 0 2024-04-24T08:25:02.524 INFO:tasks.workunit.client.0.smithi090.stdout:7/572: read db/d17/d29/d38/d3e/f58 [3620265,130782] 0 2024-04-24T08:25:02.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/963: getdents db/d12/d2f/d78 0 2024-04-24T08:25:02.527 INFO:tasks.workunit.client.0.smithi090.stdout:9/495: dread - d0/dc/d24/d7c/f40 zero size 2024-04-24T08:25:02.529 INFO:tasks.workunit.client.1.smithi149.stdout:3/896: creat d0/d26/d3c/db1/f12e x:0 0 0 2024-04-24T08:25:02.531 INFO:tasks.workunit.client.0.smithi090.stdout:4/429: dwrite d3/d38/d49/f65 [0,4194304] 0 2024-04-24T08:25:02.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/573: write db/f21 [1773303,127301] 0 2024-04-24T08:25:02.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/695: sync 2024-04-24T08:25:02.533 INFO:tasks.workunit.client.0.smithi090.stdout:6/543: write d8/dc/d19/d8e/d38/f49 [13514567,72742] 0 2024-04-24T08:25:02.534 INFO:tasks.workunit.client.0.smithi090.stdout:8/454: mkdir d2/d62/d6b/d9c 0 2024-04-24T08:25:02.537 INFO:tasks.workunit.client.1.smithi149.stdout:5/964: creat db/d12/d1e/d65/de8/dfa/f14c x:0 0 0 2024-04-24T08:25:02.540 INFO:tasks.workunit.client.0.smithi090.stdout:7/574: write db/f21 [1009753,101313] 0 2024-04-24T08:25:02.541 INFO:tasks.workunit.client.1.smithi149.stdout:5/965: fdatasync db/d12/d60/db2/d83/dc5/f11a 0 2024-04-24T08:25:02.543 INFO:tasks.workunit.client.1.smithi149.stdout:5/966: stat db/d12/f72 0 2024-04-24T08:25:02.543 INFO:tasks.workunit.client.0.smithi090.stdout:2/568: dwrite d0/d7/d67/d7f/d31/d46/d60/f74 [4194304,4194304] 0 2024-04-24T08:25:02.544 INFO:tasks.workunit.client.0.smithi090.stdout:7/575: chown db/d17/d29/d38/d3e/la1 1337 1 2024-04-24T08:25:02.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/576: chown db/d17/d39/c5d 465279 1 2024-04-24T08:25:02.552 INFO:tasks.workunit.client.1.smithi149.stdout:6/696: mknod d0/db7/cee 0 2024-04-24T08:25:02.552 INFO:tasks.workunit.client.0.smithi090.stdout:9/496: stat d0/dc/d24/d7c/l75 0 2024-04-24T08:25:02.556 INFO:tasks.workunit.client.1.smithi149.stdout:6/697: readlink d0/d1a/d28/d31/d38/l47 0 2024-04-24T08:25:02.558 INFO:tasks.workunit.client.1.smithi149.stdout:6/698: write d0/f3f [1496920,56313] 0 2024-04-24T08:25:02.561 INFO:tasks.workunit.client.1.smithi149.stdout:6/699: write d0/d1a/d28/d31/d4c/d80/f74 [4963144,16036] 0 2024-04-24T08:25:02.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/569: unlink d0/d7/d67/d7f/d31/d46/l54 0 2024-04-24T08:25:02.567 INFO:tasks.workunit.client.1.smithi149.stdout:2/719: sync 2024-04-24T08:25:02.568 INFO:tasks.workunit.client.1.smithi149.stdout:7/848: sync 2024-04-24T08:25:02.568 INFO:tasks.workunit.client.1.smithi149.stdout:9/839: sync 2024-04-24T08:25:02.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/892: sync 2024-04-24T08:25:02.568 INFO:tasks.workunit.client.1.smithi149.stdout:0/733: sync 2024-04-24T08:25:02.569 INFO:tasks.workunit.client.0.smithi090.stdout:4/430: dwrite d3/d18/d1d/f40 [0,4194304] 0 2024-04-24T08:25:02.569 INFO:tasks.workunit.client.0.smithi090.stdout:7/577: rmdir db/d17/d29/d38/d3e 39 2024-04-24T08:25:02.570 INFO:tasks.workunit.client.1.smithi149.stdout:4/778: write d2/f8a [227092,8738] 0 2024-04-24T08:25:02.572 INFO:tasks.workunit.client.1.smithi149.stdout:9/840: dread - d1/d2/d33/f120 zero size 2024-04-24T08:25:02.573 INFO:tasks.workunit.client.0.smithi090.stdout:9/497: creat d0/dc/d24/d7c/f94 x:0 0 0 2024-04-24T08:25:02.574 INFO:tasks.workunit.client.1.smithi149.stdout:8/747: dwrite d1/d1f/d5e/d9c/fb5 [0,4194304] 0 2024-04-24T08:25:02.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/498: fdatasync d0/dc/d24/d7c/f82 0 2024-04-24T08:25:02.581 INFO:tasks.workunit.client.0.smithi090.stdout:2/570: dwrite d0/d2c/fa1 [0,4194304] 0 2024-04-24T08:25:02.583 INFO:tasks.workunit.client.1.smithi149.stdout:4/779: chown d2/d18/da4/ce2 10893989 1 2024-04-24T08:25:02.583 INFO:tasks.workunit.client.0.smithi090.stdout:2/571: readlink d0/d7/d67/d7f/d31/d46/l4f 0 2024-04-24T08:25:02.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/578: creat db/d17/d39/d86/dae/fc7 x:0 0 0 2024-04-24T08:25:02.588 INFO:tasks.workunit.client.1.smithi149.stdout:7/849: symlink d9/dc/d7b/dfc/l11a 0 2024-04-24T08:25:02.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/455: link d2/da/d13/d34/f6f d2/f9d 0 2024-04-24T08:25:02.591 INFO:tasks.workunit.client.0.smithi090.stdout:8/456: write d2/da/f82 [2252643,75907] 0 2024-04-24T08:25:02.595 INFO:tasks.workunit.client.1.smithi149.stdout:6/700: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/def 0 2024-04-24T08:25:02.600 INFO:tasks.workunit.client.1.smithi149.stdout:2/720: unlink d0/d6/db/d27/f3b 0 2024-04-24T08:25:02.601 INFO:tasks.workunit.client.0.smithi090.stdout:9/499: fdatasync d0/dc/d24/d7c/f40 0 2024-04-24T08:25:02.604 INFO:tasks.workunit.client.1.smithi149.stdout:9/841: symlink d1/de/d9d/dfd/l131 0 2024-04-24T08:25:02.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/734: dread d2/d9/df/d2f/f8f [0,4194304] 0 2024-04-24T08:25:02.607 INFO:tasks.workunit.client.1.smithi149.stdout:9/842: fsync d1/d2/d34/f40 0 2024-04-24T08:25:02.607 INFO:tasks.workunit.client.0.smithi090.stdout:4/431: mknod d3/df/c7b 0 2024-04-24T08:25:02.617 INFO:tasks.workunit.client.1.smithi149.stdout:4/780: creat d2/d18/db9/ffa x:0 0 0 2024-04-24T08:25:02.617 INFO:tasks.workunit.client.1.smithi149.stdout:8/748: dread d1/de/d18/f6a [0,4194304] 0 2024-04-24T08:25:02.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/457: symlink d2/d62/d71/l9e 0 2024-04-24T08:25:02.619 INFO:tasks.workunit.client.0.smithi090.stdout:3/469: write d2/df/d1a/f6d [129640,84188] 0 2024-04-24T08:25:02.619 INFO:tasks.workunit.client.1.smithi149.stdout:7/850: rmdir d9/dc/de/d44 39 2024-04-24T08:25:02.620 INFO:tasks.workunit.client.0.smithi090.stdout:6/544: dread d8/dc/f11 [0,4194304] 0 2024-04-24T08:25:02.622 INFO:tasks.workunit.client.0.smithi090.stdout:9/500: unlink d0/dc/d24/d7c/f40 0 2024-04-24T08:25:02.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/851: chown d9/dc/d11/d38/dac/d103 151 1 2024-04-24T08:25:02.623 INFO:tasks.workunit.client.0.smithi090.stdout:9/501: fdatasync d0/f59 0 2024-04-24T08:25:02.624 INFO:tasks.workunit.client.1.smithi149.stdout:6/701: fsync d0/d1a/f29 0 2024-04-24T08:25:02.626 INFO:tasks.workunit.client.0.smithi090.stdout:7/579: creat db/d17/d29/d38/d3e/fc8 x:0 0 0 2024-04-24T08:25:02.627 INFO:tasks.workunit.client.1.smithi149.stdout:6/702: read d0/d1a/f32 [77166,116663] 0 2024-04-24T08:25:02.628 INFO:tasks.workunit.client.0.smithi090.stdout:7/580: dread - db/d17/d29/d32/d5a/d75/fbe zero size 2024-04-24T08:25:02.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/581: chown db/dab 20827 1 2024-04-24T08:25:02.631 INFO:tasks.workunit.client.0.smithi090.stdout:8/458: creat d2/da/d13/d64/d50/f9f x:0 0 0 2024-04-24T08:25:02.633 INFO:tasks.workunit.client.1.smithi149.stdout:9/843: mknod d1/d2/d33/d5d/c132 0 2024-04-24T08:25:02.635 INFO:tasks.workunit.client.1.smithi149.stdout:5/967: write db/d12/d2f/d74/f106 [682161,128867] 0 2024-04-24T08:25:02.637 INFO:tasks.workunit.client.1.smithi149.stdout:3/897: dwrite d0/d5/d53/d34/d3a/d93/fb5 [0,4194304] 0 2024-04-24T08:25:02.641 INFO:tasks.workunit.client.1.smithi149.stdout:8/749: creat d1/d1f/d5e/d92/ff5 x:0 0 0 2024-04-24T08:25:02.643 INFO:tasks.workunit.client.1.smithi149.stdout:5/968: read db/d12/d60/db2/d83/ff4 [388339,103551] 0 2024-04-24T08:25:02.644 INFO:tasks.workunit.client.1.smithi149.stdout:7/852: readlink d9/dc/d2e/l3d 0 2024-04-24T08:25:02.645 INFO:tasks.workunit.client.1.smithi149.stdout:7/853: fdatasync d9/dc/de/f33 0 2024-04-24T08:25:02.653 INFO:tasks.workunit.client.1.smithi149.stdout:9/844: truncate d1/d99/fdf 419369 0 2024-04-24T08:25:02.657 INFO:tasks.workunit.client.1.smithi149.stdout:1/893: dwrite d0/d1b/d2a/ff9 [0,4194304] 0 2024-04-24T08:25:02.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/750: unlink d1/d1f/d5e/d92/lf4 0 2024-04-24T08:25:02.662 INFO:tasks.workunit.client.1.smithi149.stdout:3/898: read - d0/d5/d53/dd6/f100 zero size 2024-04-24T08:25:02.667 INFO:tasks.workunit.client.1.smithi149.stdout:7/854: creat d9/dc/d11/d38/dac/d103/f11b x:0 0 0 2024-04-24T08:25:02.668 INFO:tasks.workunit.client.1.smithi149.stdout:8/751: write d1/de/d18/d32/d63/d9e/fd6 [844267,47136] 0 2024-04-24T08:25:02.671 INFO:tasks.workunit.client.1.smithi149.stdout:7/855: stat d9/d43/d64/f73 0 2024-04-24T08:25:02.677 INFO:tasks.workunit.client.1.smithi149.stdout:9/845: rename d1/da/d4a to d1/d2/d33/dea/d133 0 2024-04-24T08:25:02.681 INFO:tasks.workunit.client.1.smithi149.stdout:0/735: dwrite d2/d9/d1a/d36/f74 [4194304,4194304] 0 2024-04-24T08:25:02.682 INFO:tasks.workunit.client.0.smithi090.stdout:9/502: creat d0/dc/d24/d7c/d4d/d32/f95 x:0 0 0 2024-04-24T08:25:02.683 INFO:tasks.workunit.client.1.smithi149.stdout:0/736: stat d2/d9/df/d48/dba 0 2024-04-24T08:25:02.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/969: mknod db/d12/d22/c14d 0 2024-04-24T08:25:02.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/582: chown db/d17/d29/f42 222979947 1 2024-04-24T08:25:02.685 INFO:tasks.workunit.client.0.smithi090.stdout:9/503: chown d0/f5 1 1 2024-04-24T08:25:02.687 INFO:tasks.workunit.client.0.smithi090.stdout:6/545: dwrite d8/dc/d19/f3d [0,4194304] 0 2024-04-24T08:25:02.689 INFO:tasks.workunit.client.0.smithi090.stdout:8/459: rmdir d2/da/d13/d64 39 2024-04-24T08:25:02.695 INFO:tasks.workunit.client.0.smithi090.stdout:2/572: dwrite d0/d14/f15 [0,4194304] 0 2024-04-24T08:25:02.698 INFO:tasks.workunit.client.0.smithi090.stdout:4/432: getdents d3 0 2024-04-24T08:25:02.698 INFO:tasks.workunit.client.0.smithi090.stdout:2/573: chown d0/d7/d6f 2530 1 2024-04-24T08:25:02.702 INFO:tasks.workunit.client.1.smithi149.stdout:0/737: dread d2/d9/df/d56/f80 [0,4194304] 0 2024-04-24T08:25:02.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/470: dwrite d2/df/d5b/f63 [4194304,4194304] 0 2024-04-24T08:25:02.706 INFO:tasks.workunit.client.1.smithi149.stdout:0/738: readlink d2/d3c/l6e 0 2024-04-24T08:25:02.709 INFO:tasks.workunit.client.1.smithi149.stdout:1/894: dwrite d0/d5/f9 [0,4194304] 0 2024-04-24T08:25:02.710 INFO:tasks.workunit.client.1.smithi149.stdout:5/970: dwrite db/d12/d22/d45/d5d/f127 [0,4194304] 0 2024-04-24T08:25:02.714 INFO:tasks.workunit.client.1.smithi149.stdout:6/703: rmdir d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/def 0 2024-04-24T08:25:02.716 INFO:tasks.workunit.client.1.smithi149.stdout:4/781: link d2/d18/d1e/d3c/d48/d69/f77 d2/d24/d7f/ffb 0 2024-04-24T08:25:02.716 INFO:tasks.workunit.client.1.smithi149.stdout:8/752: mknod d1/de/d18/d32/d65/d8b/cf6 0 2024-04-24T08:25:02.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/846: creat d1/da/d13/f134 x:0 0 0 2024-04-24T08:25:02.722 INFO:tasks.workunit.client.1.smithi149.stdout:8/753: dread - d1/de/d18/d32/d65/d8b/d7d/ddc/fe0 zero size 2024-04-24T08:25:02.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/583: symlink db/d17/d29/lc9 0 2024-04-24T08:25:02.723 INFO:tasks.workunit.client.0.smithi090.stdout:8/460: chown d2/da/d13/d64/f6d 79218 1 2024-04-24T08:25:02.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/899: creat d0/d26/d3c/db1/deb/f12f x:0 0 0 2024-04-24T08:25:02.724 INFO:tasks.workunit.client.1.smithi149.stdout:2/721: link d0/d6/db/d39/d3d/d44/l5d d0/d6/db/d39/d3d/d44/ld9 0 2024-04-24T08:25:02.728 INFO:tasks.workunit.client.0.smithi090.stdout:4/433: mkdir d3/df/d7c 0 2024-04-24T08:25:02.731 INFO:tasks.workunit.client.1.smithi149.stdout:5/971: symlink db/d12/d1e/d12d/l14e 0 2024-04-24T08:25:02.732 INFO:tasks.workunit.client.0.smithi090.stdout:7/584: creat db/d17/d29/d32/fca x:0 0 0 2024-04-24T08:25:02.737 INFO:tasks.workunit.client.1.smithi149.stdout:5/972: write db/d12/d1e/d65/de8/ff5 [446500,96745] 0 2024-04-24T08:25:02.737 INFO:tasks.workunit.client.0.smithi090.stdout:2/574: link d0/d2e/f6a d0/d7/d67/fc7 0 2024-04-24T08:25:02.738 INFO:tasks.workunit.client.0.smithi090.stdout:3/471: link d2/d15/l1c d2/df/d57/d4f/la0 0 2024-04-24T08:25:02.743 INFO:tasks.workunit.client.0.smithi090.stdout:4/434: creat d3/d38/d49/d6f/f7d x:0 0 0 2024-04-24T08:25:02.743 INFO:tasks.workunit.client.0.smithi090.stdout:6/546: getdents d8/dc/d19/d8e/d81 0 2024-04-24T08:25:02.744 INFO:tasks.workunit.client.0.smithi090.stdout:7/585: rmdir db/d17 39 2024-04-24T08:25:02.747 INFO:tasks.workunit.client.0.smithi090.stdout:3/472: rmdir d2/d15 39 2024-04-24T08:25:02.755 INFO:tasks.workunit.client.0.smithi090.stdout:4/435: creat d3/d38/f7e x:0 0 0 2024-04-24T08:25:02.756 INFO:tasks.workunit.client.1.smithi149.stdout:6/704: rename d0/d1a/d28/d31/d4c/d80/d6f/d97/cc0 to d0/d1a/d28/d31/d4c/d80/d6f/dc2/cf0 0 2024-04-24T08:25:02.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/973: dread db/d12/d1e/fc1 [0,4194304] 0 2024-04-24T08:25:02.758 INFO:tasks.workunit.client.1.smithi149.stdout:1/895: write d0/d5/db/fcd [1001201,60457] 0 2024-04-24T08:25:02.759 INFO:tasks.workunit.client.1.smithi149.stdout:5/974: stat db/da4/ce3 0 2024-04-24T08:25:02.760 INFO:tasks.workunit.client.0.smithi090.stdout:2/575: creat d0/fc8 x:0 0 0 2024-04-24T08:25:02.762 INFO:tasks.workunit.client.1.smithi149.stdout:4/782: write d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf [842784,27408] 0 2024-04-24T08:25:02.762 INFO:tasks.workunit.client.1.smithi149.stdout:5/975: fdatasync db/d12/d1e/d65/fb4 0 2024-04-24T08:25:02.762 INFO:tasks.workunit.client.0.smithi090.stdout:6/547: dwrite d8/d1d/d1f/d3e/d44/f91 [0,4194304] 0 2024-04-24T08:25:02.763 INFO:tasks.workunit.client.1.smithi149.stdout:0/739: creat d2/d9/df/fec x:0 0 0 2024-04-24T08:25:02.764 INFO:tasks.workunit.client.1.smithi149.stdout:9/847: write d1/d2/d33/dea/fef [1151700,50213] 0 2024-04-24T08:25:02.765 INFO:tasks.workunit.client.0.smithi090.stdout:7/586: creat db/d17/d29/d38/d3e/fcb x:0 0 0 2024-04-24T08:25:02.765 INFO:tasks.workunit.client.1.smithi149.stdout:2/722: creat d0/d6/db/d28/dd5/fda x:0 0 0 2024-04-24T08:25:02.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/436: creat d3/d18/d5c/f7f x:0 0 0 2024-04-24T08:25:02.766 INFO:tasks.workunit.client.0.smithi090.stdout:8/461: dread d2/da/d13/d34/f6f [0,4194304] 0 2024-04-24T08:25:02.767 INFO:tasks.workunit.client.1.smithi149.stdout:5/976: chown db/d12/d2f/d95/fdc 276900566 1 2024-04-24T08:25:02.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/504: dread d0/dc/d24/d7c/d4d/f52 [0,4194304] 0 2024-04-24T08:25:02.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/856: getdents d9 0 2024-04-24T08:25:02.771 INFO:tasks.workunit.client.0.smithi090.stdout:2/576: chown d0/d7/d6f/c9b 539918449 1 2024-04-24T08:25:02.772 INFO:tasks.workunit.client.1.smithi149.stdout:3/900: write d0/d5/d53/d34/d3a/d93/fed [682274,106409] 0 2024-04-24T08:25:02.773 INFO:tasks.workunit.client.0.smithi090.stdout:2/577: chown d0/d7/d67/d7f/f78 933 1 2024-04-24T08:25:02.775 INFO:tasks.workunit.client.1.smithi149.stdout:1/896: fdatasync d0/ddf/da0/f9e 0 2024-04-24T08:25:02.776 INFO:tasks.workunit.client.1.smithi149.stdout:7/857: fdatasync d9/dc/d11/d38/fe8 0 2024-04-24T08:25:02.778 INFO:tasks.workunit.client.1.smithi149.stdout:4/783: readlink d2/l8 0 2024-04-24T08:25:02.781 INFO:tasks.workunit.client.1.smithi149.stdout:9/848: mkdir d1/de/d9d/dfd/d135 0 2024-04-24T08:25:02.787 INFO:tasks.workunit.client.1.smithi149.stdout:6/705: symlink d0/d1a/d28/d31/d4c/d80/d6f/dd7/lf1 0 2024-04-24T08:25:02.788 INFO:tasks.workunit.client.0.smithi090.stdout:9/505: truncate d0/dc/d24/d2d/d49/d62/f51 1597374 0 2024-04-24T08:25:02.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/506: chown d0/dc/d24/d2d/f47 22235635 1 2024-04-24T08:25:02.791 INFO:tasks.workunit.client.0.smithi090.stdout:0/534: sync 2024-04-24T08:25:02.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/552: sync 2024-04-24T08:25:02.791 INFO:tasks.workunit.client.0.smithi090.stdout:5/584: sync 2024-04-24T08:25:02.792 INFO:tasks.workunit.client.1.smithi149.stdout:8/754: dread d1/d1f/d22/f5f [0,4194304] 0 2024-04-24T08:25:02.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/507: write d0/dc/f2c [430138,53171] 0 2024-04-24T08:25:02.793 INFO:tasks.workunit.client.0.smithi090.stdout:8/462: fdatasync d2/da/d13/f4b 0 2024-04-24T08:25:02.796 INFO:tasks.workunit.client.1.smithi149.stdout:8/755: stat d1/de/d18/d32/d63/c72 0 2024-04-24T08:25:02.796 INFO:tasks.workunit.client.0.smithi090.stdout:1/553: write d3/d5/da/dc/f87 [1643917,105002] 0 2024-04-24T08:25:02.796 INFO:tasks.workunit.client.0.smithi090.stdout:4/437: rename d3/d18/d1d/l54 to d3/d38/d49/l80 0 2024-04-24T08:25:02.798 INFO:tasks.workunit.client.0.smithi090.stdout:3/473: getdents d2/df/d1a/d37/d4b 0 2024-04-24T08:25:02.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/901: unlink d0/d5/da/d5e/f30 0 2024-04-24T08:25:02.801 INFO:tasks.workunit.client.0.smithi090.stdout:3/474: write d2/df/d1a/d37/d53/f74 [2684776,11919] 0 2024-04-24T08:25:02.805 INFO:tasks.workunit.client.0.smithi090.stdout:7/587: dwrite db/fd [0,4194304] 0 2024-04-24T08:25:02.805 INFO:tasks.workunit.client.1.smithi149.stdout:3/902: write d0/d5/d53/d34/d3a/d93/fb5 [3669971,130295] 0 2024-04-24T08:25:02.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/740: write d2/d9/df/d2f/d6c/f77 [380136,92178] 0 2024-04-24T08:25:02.809 INFO:tasks.workunit.client.1.smithi149.stdout:4/784: chown d2/da8/fd3 38249508 1 2024-04-24T08:25:02.810 INFO:tasks.workunit.client.0.smithi090.stdout:2/578: mkdir d0/d14/d17/dc2/dc9 0 2024-04-24T08:25:02.814 INFO:tasks.workunit.client.1.smithi149.stdout:2/723: dwrite d0/d6/db/d28/f48 [0,4194304] 0 2024-04-24T08:25:02.817 INFO:tasks.workunit.client.0.smithi090.stdout:3/475: dread d2/f24 [0,4194304] 0 2024-04-24T08:25:02.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/508: dread d0/d19/f1a [0,4194304] 0 2024-04-24T08:25:02.819 INFO:tasks.workunit.client.1.smithi149.stdout:9/849: creat d1/d2/d15/d1c/dca/f136 x:0 0 0 2024-04-24T08:25:02.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/585: rmdir d8/da/d18/d21/d4c 39 2024-04-24T08:25:02.826 INFO:tasks.workunit.client.0.smithi090.stdout:1/554: unlink d3/d5/da/dc/f35 0 2024-04-24T08:25:02.829 INFO:tasks.workunit.client.0.smithi090.stdout:4/438: unlink d3/d38/d6b/f57 0 2024-04-24T08:25:02.835 INFO:tasks.workunit.client.1.smithi149.stdout:9/850: fsync d1/d2/d15/d1c/dca/f136 0 2024-04-24T08:25:02.841 INFO:tasks.workunit.client.0.smithi090.stdout:6/548: getdents d8/d1d/d29 0 2024-04-24T08:25:02.842 INFO:tasks.workunit.client.0.smithi090.stdout:6/549: write d8/f46 [1973752,7165] 0 2024-04-24T08:25:02.845 INFO:tasks.workunit.client.1.smithi149.stdout:5/977: dread db/d12/d22/f33 [4194304,4194304] 0 2024-04-24T08:25:02.845 INFO:tasks.workunit.client.1.smithi149.stdout:4/785: rmdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2 39 2024-04-24T08:25:02.852 INFO:tasks.workunit.client.0.smithi090.stdout:0/535: symlink d1a/d4b/d53/d79/d69/laa 0 2024-04-24T08:25:02.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/897: creat d0/ddf/f122 x:0 0 0 2024-04-24T08:25:02.856 INFO:tasks.workunit.client.1.smithi149.stdout:6/706: dwrite d0/d1a/d28/d31/d4c/d80/d6f/d97/fe3 [0,4194304] 0 2024-04-24T08:25:02.861 INFO:tasks.workunit.client.1.smithi149.stdout:0/741: dread d2/d9/df/d48/f4f [0,4194304] 0 2024-04-24T08:25:02.862 INFO:tasks.workunit.client.1.smithi149.stdout:9/851: fdatasync d1/da/dd/f9e 0 2024-04-24T08:25:02.863 INFO:tasks.workunit.client.1.smithi149.stdout:0/742: write d2/fea [964710,89276] 0 2024-04-24T08:25:02.869 INFO:tasks.workunit.client.1.smithi149.stdout:7/858: truncate d9/dc/f6b 381244 0 2024-04-24T08:25:02.870 INFO:tasks.workunit.client.1.smithi149.stdout:4/786: fsync d2/d24/f39 0 2024-04-24T08:25:02.872 INFO:tasks.workunit.client.1.smithi149.stdout:5/978: mknod db/da4/d120/c14f 0 2024-04-24T08:25:02.873 INFO:tasks.workunit.client.1.smithi149.stdout:4/787: stat d2/d18/db9/dd8/l53 0 2024-04-24T08:25:02.876 INFO:tasks.workunit.client.1.smithi149.stdout:4/788: chown d2/d18/da4/ce2 6472 1 2024-04-24T08:25:02.878 INFO:tasks.workunit.client.1.smithi149.stdout:4/789: readlink d2/d18/d1e/l95 0 2024-04-24T08:25:02.883 INFO:tasks.workunit.client.1.smithi149.stdout:0/743: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fed x:0 0 0 2024-04-24T08:25:02.883 INFO:tasks.workunit.client.1.smithi149.stdout:1/898: write d0/d1b/d4b/f102 [482867,83705] 0 2024-04-24T08:25:02.884 INFO:tasks.workunit.client.1.smithi149.stdout:3/903: creat d0/d5/da/d5a/d63/f130 x:0 0 0 2024-04-24T08:25:02.886 INFO:tasks.workunit.client.1.smithi149.stdout:8/756: getdents d1/de/d18/d32/d65/d8b/d7d/dd1 0 2024-04-24T08:25:02.887 INFO:tasks.workunit.client.1.smithi149.stdout:0/744: write d2/d9/f13 [1275635,27824] 0 2024-04-24T08:25:02.891 INFO:tasks.workunit.client.1.smithi149.stdout:9/852: mkdir d1/d2/d33/dea/d126/d137 0 2024-04-24T08:25:02.892 INFO:tasks.workunit.client.1.smithi149.stdout:8/757: write d1/de/d18/d43/f55 [4903745,128283] 0 2024-04-24T08:25:02.894 INFO:tasks.workunit.client.1.smithi149.stdout:4/790: rename d2/f6c to d2/d18/d1e/d3c/d48/d69/d7d/d8d/def/ffc 0 2024-04-24T08:25:02.894 INFO:tasks.workunit.client.1.smithi149.stdout:9/853: chown d1/de/f22 208613448 1 2024-04-24T08:25:02.902 INFO:tasks.workunit.client.1.smithi149.stdout:1/899: mkdir d0/d1b/db5/d108/dd3/d123 0 2024-04-24T08:25:02.904 INFO:tasks.workunit.client.1.smithi149.stdout:6/707: truncate d0/d1a/d28/d31/d4c/d8a/fb5 1443659 0 2024-04-24T08:25:02.906 INFO:tasks.workunit.client.1.smithi149.stdout:0/745: creat d2/d9/d1a/d36/d85/dcd/fee x:0 0 0 2024-04-24T08:25:02.908 INFO:tasks.workunit.client.1.smithi149.stdout:6/708: write d0/d1a/d28/d31/d4c/d4e/d62/fde [565310,36543] 0 2024-04-24T08:25:02.908 INFO:tasks.workunit.client.1.smithi149.stdout:8/758: rmdir d1/d1f/d25/d7a 39 2024-04-24T08:25:02.910 INFO:tasks.workunit.client.1.smithi149.stdout:5/979: dwrite db/f16 [4194304,4194304] 0 2024-04-24T08:25:02.910 INFO:tasks.workunit.client.1.smithi149.stdout:4/791: stat d2/d18/dd6 0 2024-04-24T08:25:02.912 INFO:tasks.workunit.client.1.smithi149.stdout:6/709: dread - d0/d1a/d28/d31/d4c/d80/fd0 zero size 2024-04-24T08:25:02.913 INFO:tasks.workunit.client.1.smithi149.stdout:8/759: read d1/d1f/d25/f80 [1984047,125714] 0 2024-04-24T08:25:02.914 INFO:tasks.workunit.client.1.smithi149.stdout:9/854: creat d1/d4c/f138 x:0 0 0 2024-04-24T08:25:02.915 INFO:tasks.workunit.client.1.smithi149.stdout:7/859: write d9/dc/d2e/f41 [171619,106225] 0 2024-04-24T08:25:02.916 INFO:tasks.workunit.client.1.smithi149.stdout:4/792: read d2/d24/d2e/d3f/fa9 [4125946,113972] 0 2024-04-24T08:25:02.917 INFO:tasks.workunit.client.1.smithi149.stdout:3/904: symlink d0/d5/da/d5a/d63/d6a/d10e/l131 0 2024-04-24T08:25:02.918 INFO:tasks.workunit.client.1.smithi149.stdout:1/900: creat d0/d1b/d5d/f124 x:0 0 0 2024-04-24T08:25:02.921 INFO:tasks.workunit.client.1.smithi149.stdout:5/980: mknod db/d12/d2f/d74/df6/c150 0 2024-04-24T08:25:02.924 INFO:tasks.workunit.client.1.smithi149.stdout:9/855: creat d1/d2/d15/d1c/dca/f139 x:0 0 0 2024-04-24T08:25:02.924 INFO:tasks.workunit.client.1.smithi149.stdout:1/901: read d0/ddf/da0/db8/dc5/f113 [69586,80952] 0 2024-04-24T08:25:02.928 INFO:tasks.workunit.client.1.smithi149.stdout:0/746: mkdir d2/d9/d1a/d36/d85/db5/def 0 2024-04-24T08:25:02.930 INFO:tasks.workunit.client.1.smithi149.stdout:8/760: symlink d1/de/d18/d32/d65/d8b/d7d/dd1/lf7 0 2024-04-24T08:25:02.932 INFO:tasks.workunit.client.1.smithi149.stdout:0/747: write d2/d9/f89 [855659,113194] 0 2024-04-24T08:25:02.937 INFO:tasks.workunit.client.1.smithi149.stdout:3/905: symlink d0/d5/da/d19/d80/dac/l132 0 2024-04-24T08:25:02.939 INFO:tasks.workunit.client.1.smithi149.stdout:3/906: readlink d0/d5/da/d5e/l58 0 2024-04-24T08:25:02.939 INFO:tasks.workunit.client.1.smithi149.stdout:4/793: mkdir d2/d24/d2e/d3f/d94/dfd 0 2024-04-24T08:25:02.941 INFO:tasks.workunit.client.1.smithi149.stdout:3/907: chown d0/d5/d53/d4c/dca/dfb 66 1 2024-04-24T08:25:02.941 INFO:tasks.workunit.client.1.smithi149.stdout:1/902: read d0/d5/d4f/d60/fc8 [170814,1042] 0 2024-04-24T08:25:02.944 INFO:tasks.workunit.client.1.smithi149.stdout:3/908: readlink d0/d5/d53/d34/d3a/l54 0 2024-04-24T08:25:02.944 INFO:tasks.workunit.client.1.smithi149.stdout:5/981: mkdir db/d12/d60/db2/d83/dc5/d151 0 2024-04-24T08:25:02.947 INFO:tasks.workunit.client.1.smithi149.stdout:3/909: write d0/d5/d53/f3f [3448280,7347] 0 2024-04-24T08:25:02.949 INFO:tasks.workunit.client.1.smithi149.stdout:8/761: creat d1/de/d18/d32/d65/d8b/d7d/ff8 x:0 0 0 2024-04-24T08:25:02.950 INFO:tasks.workunit.client.0.smithi090.stdout:8/463: fsync d2/da/d13/d64/d24/f79 0 2024-04-24T08:25:02.951 INFO:tasks.workunit.client.0.smithi090.stdout:7/588: symlink db/d17/d29/d38/lcc 0 2024-04-24T08:25:02.951 INFO:tasks.workunit.client.1.smithi149.stdout:0/748: mkdir d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/df0 0 2024-04-24T08:25:02.953 INFO:tasks.workunit.client.0.smithi090.stdout:3/476: creat d2/d90/fa1 x:0 0 0 2024-04-24T08:25:02.954 INFO:tasks.workunit.client.0.smithi090.stdout:0/536: truncate d1a/d4b/d53/d79/d35/f55 4388774 0 2024-04-24T08:25:02.957 INFO:tasks.workunit.client.1.smithi149.stdout:5/982: symlink db/d23/l152 0 2024-04-24T08:25:02.958 INFO:tasks.workunit.client.0.smithi090.stdout:4/439: mkdir d3/df/d7c/d81 0 2024-04-24T08:25:02.958 INFO:tasks.workunit.client.0.smithi090.stdout:1/555: rename d3/c1f to d3/d5/da/d94/cc7 0 2024-04-24T08:25:02.959 INFO:tasks.workunit.client.0.smithi090.stdout:3/477: readlink d2/df/d57/d4f/l66 0 2024-04-24T08:25:02.959 INFO:tasks.workunit.client.1.smithi149.stdout:9/856: creat d1/d2/d33/dea/d133/dad/db7/d12d/f13a x:0 0 0 2024-04-24T08:25:02.961 INFO:tasks.workunit.client.0.smithi090.stdout:7/589: write db/d17/d29/d32/d5a/d75/fa4 [175860,77883] 0 2024-04-24T08:25:02.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/860: write d9/dc/d11/d26/f4a [2516936,73350] 0 2024-04-24T08:25:02.962 INFO:tasks.workunit.client.1.smithi149.stdout:0/749: dread d2/d9/df/d2f/d6c/dc1/fcf [0,4194304] 0 2024-04-24T08:25:02.965 INFO:tasks.workunit.client.1.smithi149.stdout:7/861: fsync d9/d43/d64/f73 0 2024-04-24T08:25:02.966 INFO:tasks.workunit.client.1.smithi149.stdout:8/762: fsync d1/d1f/f28 0 2024-04-24T08:25:02.967 INFO:tasks.workunit.client.1.smithi149.stdout:6/710: getdents d0/d1a/d28/d31/d38/d54 0 2024-04-24T08:25:02.967 INFO:tasks.workunit.client.0.smithi090.stdout:2/579: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:25:02.968 INFO:tasks.workunit.client.1.smithi149.stdout:4/794: creat d2/d10/d91/df7/ffe x:0 0 0 2024-04-24T08:25:02.968 INFO:tasks.workunit.client.0.smithi090.stdout:3/478: truncate d2/df/d1a/d37/f85 353187 0 2024-04-24T08:25:02.970 INFO:tasks.workunit.client.1.smithi149.stdout:5/983: write db/d12/d60/db2/f115 [572461,127599] 0 2024-04-24T08:25:02.971 INFO:tasks.workunit.client.0.smithi090.stdout:9/509: dread d0/f1b [0,4194304] 0 2024-04-24T08:25:02.971 INFO:tasks.workunit.client.0.smithi090.stdout:1/556: creat d3/d5/d3e/da3/fc8 x:0 0 0 2024-04-24T08:25:02.973 INFO:tasks.workunit.client.1.smithi149.stdout:5/984: chown db/d12/d2f/d95/fe5 5478 1 2024-04-24T08:25:02.973 INFO:tasks.workunit.client.0.smithi090.stdout:2/580: chown d0/d7/d67/d7f/d31/f41 0 1 2024-04-24T08:25:02.974 INFO:tasks.workunit.client.0.smithi090.stdout:9/510: write d0/dc/f2c [855308,90977] 0 2024-04-24T08:25:02.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/479: rmdir d2 39 2024-04-24T08:25:02.977 INFO:tasks.workunit.client.0.smithi090.stdout:1/557: read d3/d1d/d34/d73/f99 [685129,112885] 0 2024-04-24T08:25:02.979 INFO:tasks.workunit.client.0.smithi090.stdout:6/550: dwrite d8/d1d/d2f/f3c [0,4194304] 0 2024-04-24T08:25:02.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/440: dwrite d3/d18/d5c/f7f [0,4194304] 0 2024-04-24T08:25:02.983 INFO:tasks.workunit.client.0.smithi090.stdout:4/441: fdatasync d3/d38/d49/d6f/f72 0 2024-04-24T08:25:02.985 INFO:tasks.workunit.client.1.smithi149.stdout:0/750: symlink d2/d17/d41/d68/ddc/lf1 0 2024-04-24T08:25:03.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/581: creat d0/d14/d36/fca x:0 0 0 2024-04-24T08:25:03.004 INFO:tasks.workunit.client.1.smithi149.stdout:8/763: dwrite d1/dce/fed [0,4194304] 0 2024-04-24T08:25:03.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/511: mkdir d0/d69/d7d/d5b/d96 0 2024-04-24T08:25:03.007 INFO:tasks.workunit.client.0.smithi090.stdout:6/551: mknod d8/dc/d19/d8e/d81/ca5 0 2024-04-24T08:25:03.012 INFO:tasks.workunit.client.0.smithi090.stdout:8/464: getdents d2/da/d13/d34 0 2024-04-24T08:25:03.014 INFO:tasks.workunit.client.1.smithi149.stdout:5/985: creat db/d35/d123/f153 x:0 0 0 2024-04-24T08:25:03.017 INFO:tasks.workunit.client.0.smithi090.stdout:4/442: unlink d3/d18/c61 0 2024-04-24T08:25:03.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/910: getdents d0/d5/d53 0 2024-04-24T08:25:03.018 INFO:tasks.workunit.client.0.smithi090.stdout:4/443: readlink d3/df/l31 0 2024-04-24T08:25:03.022 INFO:tasks.workunit.client.0.smithi090.stdout:7/590: dwrite db/f11 [0,4194304] 0 2024-04-24T08:25:03.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/582: rmdir d0/d2c 39 2024-04-24T08:25:03.025 INFO:tasks.workunit.client.0.smithi090.stdout:9/512: mknod d0/dc/d24/d2d/d49/d62/c97 0 2024-04-24T08:25:03.028 INFO:tasks.workunit.client.0.smithi090.stdout:2/583: dread - d0/d7/f94 zero size 2024-04-24T08:25:03.031 INFO:tasks.workunit.client.1.smithi149.stdout:9/857: dwrite d1/d2/d15/f30 [0,4194304] 0 2024-04-24T08:25:03.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/552: dwrite d8/d1d/d29/f8d [0,4194304] 0 2024-04-24T08:25:03.032 INFO:tasks.workunit.client.1.smithi149.stdout:6/711: link d0/d1a/d28/d31/d38/l47 d0/d1a/d28/lf2 0 2024-04-24T08:25:03.033 INFO:tasks.workunit.client.0.smithi090.stdout:8/465: fsync d2/da/d13/f52 0 2024-04-24T08:25:03.038 INFO:tasks.workunit.client.1.smithi149.stdout:8/764: creat d1/de/d18/d32/ff9 x:0 0 0 2024-04-24T08:25:03.038 INFO:tasks.workunit.client.0.smithi090.stdout:9/513: creat d0/d2a/f98 x:0 0 0 2024-04-24T08:25:03.042 INFO:tasks.workunit.client.0.smithi090.stdout:2/584: truncate d0/f55 859563 0 2024-04-24T08:25:03.043 INFO:tasks.workunit.client.1.smithi149.stdout:6/712: fsync d0/d1a/d28/d31/d4c/d4e/f96 0 2024-04-24T08:25:03.052 INFO:tasks.workunit.client.1.smithi149.stdout:5/986: write db/d12/f3d [1369223,68189] 0 2024-04-24T08:25:03.053 INFO:tasks.workunit.client.1.smithi149.stdout:8/765: chown d1/l11 34633308 1 2024-04-24T08:25:03.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/480: dwrite d2/fa [0,4194304] 0 2024-04-24T08:25:03.054 INFO:tasks.workunit.client.1.smithi149.stdout:3/911: rmdir d0/d5/da/d19/d80/d117 0 2024-04-24T08:25:03.056 INFO:tasks.workunit.client.0.smithi090.stdout:4/444: dwrite d3/d18/d1d/f4e [0,4194304] 0 2024-04-24T08:25:03.057 INFO:tasks.workunit.client.1.smithi149.stdout:0/751: dwrite d2/d9/df/d2f/d6c/d72/d91/f95 [0,4194304] 0 2024-04-24T08:25:03.060 INFO:tasks.workunit.client.0.smithi090.stdout:9/514: symlink d0/d5e/d78/l99 0 2024-04-24T08:25:03.061 INFO:tasks.workunit.client.0.smithi090.stdout:4/445: write d3/d38/d49/f63 [931923,35601] 0 2024-04-24T08:25:03.066 INFO:tasks.workunit.client.1.smithi149.stdout:5/987: mknod db/d12/d22/d45/d5d/c154 0 2024-04-24T08:25:03.069 INFO:tasks.workunit.client.0.smithi090.stdout:8/466: link d2/da/d13/d64/f8e d2/da/d13/d34/fa0 0 2024-04-24T08:25:03.071 INFO:tasks.workunit.client.1.smithi149.stdout:6/713: mkdir d0/d1a/d28/df3 0 2024-04-24T08:25:03.073 INFO:tasks.workunit.client.0.smithi090.stdout:4/446: creat d3/d38/d6b/f82 x:0 0 0 2024-04-24T08:25:03.075 INFO:tasks.workunit.client.1.smithi149.stdout:0/752: mknod d2/d3c/d57/cf2 0 2024-04-24T08:25:03.075 INFO:tasks.workunit.client.0.smithi090.stdout:4/447: write d3/f32 [621525,62321] 0 2024-04-24T08:25:03.076 INFO:tasks.workunit.client.1.smithi149.stdout:9/858: dwrite d1/da/dd/f32 [4194304,4194304] 0 2024-04-24T08:25:03.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/515: creat d0/dc/d24/d2d/d49/d81/f9a x:0 0 0 2024-04-24T08:25:03.077 INFO:tasks.workunit.client.0.smithi090.stdout:7/591: dread db/f10 [0,4194304] 0 2024-04-24T08:25:03.078 INFO:tasks.workunit.client.0.smithi090.stdout:6/553: creat d8/dc/d19/d8e/fa6 x:0 0 0 2024-04-24T08:25:03.080 INFO:tasks.workunit.client.1.smithi149.stdout:6/714: chown d0/d1a/d28/d31/d4c/d76/fe1 10 1 2024-04-24T08:25:03.080 INFO:tasks.workunit.client.1.smithi149.stdout:5/988: mknod db/d12/d2f/d74/c155 0 2024-04-24T08:25:03.080 INFO:tasks.workunit.client.0.smithi090.stdout:3/481: dread d2/d15/f1d [0,4194304] 0 2024-04-24T08:25:03.081 INFO:tasks.workunit.client.0.smithi090.stdout:4/448: write d3/d38/f4d [4246903,84546] 0 2024-04-24T08:25:03.083 INFO:tasks.workunit.client.1.smithi149.stdout:0/753: fsync d2/d9/df/d2f/d6c/f9a 0 2024-04-24T08:25:03.083 INFO:tasks.workunit.client.1.smithi149.stdout:6/715: fsync d0/f3f 0 2024-04-24T08:25:03.084 INFO:tasks.workunit.client.0.smithi090.stdout:4/449: dread - d3/d24/f44 zero size 2024-04-24T08:25:03.086 INFO:tasks.workunit.client.1.smithi149.stdout:8/766: dwrite d1/d1f/d5e/d9c/fb5 [4194304,4194304] 0 2024-04-24T08:25:03.088 INFO:tasks.workunit.client.1.smithi149.stdout:5/989: write db/d12/d60/db2/d83/dc5/f11a [226656,61256] 0 2024-04-24T08:25:03.091 INFO:tasks.workunit.client.1.smithi149.stdout:5/990: chown db 0 1 2024-04-24T08:25:03.091 INFO:tasks.workunit.client.1.smithi149.stdout:5/991: dread - db/d23/f146 zero size 2024-04-24T08:25:03.093 INFO:tasks.workunit.client.0.smithi090.stdout:8/467: chown d2/da/l31 8 1 2024-04-24T08:25:03.095 INFO:tasks.workunit.client.1.smithi149.stdout:9/859: dread d1/d99/fdf [0,4194304] 0 2024-04-24T08:25:03.096 INFO:tasks.workunit.client.0.smithi090.stdout:8/468: read - d2/da/d13/d34/f66 zero size 2024-04-24T08:25:03.100 INFO:tasks.workunit.client.0.smithi090.stdout:9/516: creat d0/d5e/f9b x:0 0 0 2024-04-24T08:25:03.102 INFO:tasks.workunit.client.1.smithi149.stdout:3/912: getdents d0/d5/da/d19/d1b/d22/dd4 0 2024-04-24T08:25:03.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/585: dwrite d0/d7/d67/f72 [0,4194304] 0 2024-04-24T08:25:03.104 INFO:tasks.workunit.client.1.smithi149.stdout:0/754: dread d2/d9/df/d2f/d6c/dc1/fcf [0,4194304] 0 2024-04-24T08:25:03.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/450: dwrite d3/d38/d49/d6f/f72 [4194304,4194304] 0 2024-04-24T08:25:03.105 INFO:tasks.workunit.client.0.smithi090.stdout:3/482: symlink d2/df/d57/la2 0 2024-04-24T08:25:03.106 INFO:tasks.workunit.client.1.smithi149.stdout:8/767: symlink d1/de/d18/d32/d65/d8b/d7d/lfa 0 2024-04-24T08:25:03.107 INFO:tasks.workunit.client.0.smithi090.stdout:2/586: stat d0/d14/d4d/d50/d96/fa2 0 2024-04-24T08:25:03.109 INFO:tasks.workunit.client.0.smithi090.stdout:2/587: write d0/fb0 [1598244,125324] 0 2024-04-24T08:25:03.111 INFO:tasks.workunit.client.1.smithi149.stdout:5/992: symlink db/d12/d2f/d74/dc6/d113/l156 0 2024-04-24T08:25:03.117 INFO:tasks.workunit.client.1.smithi149.stdout:6/716: mkdir d0/d1a/d28/d31/df4 0 2024-04-24T08:25:03.121 INFO:tasks.workunit.client.0.smithi090.stdout:4/451: creat d3/d38/d49/f83 x:0 0 0 2024-04-24T08:25:03.121 INFO:tasks.workunit.client.0.smithi090.stdout:3/483: creat d2/d84/fa3 x:0 0 0 2024-04-24T08:25:03.123 INFO:tasks.workunit.client.0.smithi090.stdout:3/484: fdatasync d2/d25/f7c 0 2024-04-24T08:25:03.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/517: dwrite d0/dc/d24/d7c/d4d/f84 [0,4194304] 0 2024-04-24T08:25:03.127 INFO:tasks.workunit.client.1.smithi149.stdout:0/755: stat d2/d9/df/d48/c55 0 2024-04-24T08:25:03.127 INFO:tasks.workunit.client.1.smithi149.stdout:5/993: rename db/d12/d2f/c6a to db/d12/d60/db2/d83/dc5/c157 0 2024-04-24T08:25:03.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/469: creat d2/d62/d6b/d9c/fa1 x:0 0 0 2024-04-24T08:25:03.129 INFO:tasks.workunit.client.0.smithi090.stdout:6/554: rename d8/l47 to d8/d1d/d1f/d3e/la7 0 2024-04-24T08:25:03.131 INFO:tasks.workunit.client.0.smithi090.stdout:2/588: dwrite d0/d14/d17/f1b [0,4194304] 0 2024-04-24T08:25:03.132 INFO:tasks.workunit.client.0.smithi090.stdout:8/470: dread - d2/d62/d6b/d9c/fa1 zero size 2024-04-24T08:25:03.134 INFO:tasks.workunit.client.1.smithi149.stdout:3/913: rename d0/d5/da/d19/d1b/d22 to d0/d5/da/d19/d1b/d133 0 2024-04-24T08:25:03.134 INFO:tasks.workunit.client.1.smithi149.stdout:0/756: write d2/d17/f5e [929367,8747] 0 2024-04-24T08:25:03.136 INFO:tasks.workunit.client.1.smithi149.stdout:8/768: creat d1/ffb x:0 0 0 2024-04-24T08:25:03.138 INFO:tasks.workunit.client.0.smithi090.stdout:7/592: rename db/d17/d29/d32/d5a/cbf to db/d17/d29/d32/d5a/d75/ccd 0 2024-04-24T08:25:03.139 INFO:tasks.workunit.client.1.smithi149.stdout:6/717: mknod d0/d1a/d28/d31/d4c/d80/cf5 0 2024-04-24T08:25:03.140 INFO:tasks.workunit.client.0.smithi090.stdout:2/589: chown d0/d7/d67/d7f/d31/d46/d60 333268 1 2024-04-24T08:25:03.143 INFO:tasks.workunit.client.1.smithi149.stdout:5/994: rename db/d12/d1e/d85/d92/f9f to db/d12/d2f/f158 0 2024-04-24T08:25:03.150 INFO:tasks.workunit.client.1.smithi149.stdout:3/914: read d0/d26/d3c/db1/fb6 [531990,45749] 0 2024-04-24T08:25:03.152 INFO:tasks.workunit.client.0.smithi090.stdout:9/518: dwrite d0/dc/d24/d7c/d4d/f5f [0,4194304] 0 2024-04-24T08:25:03.155 INFO:tasks.workunit.client.0.smithi090.stdout:3/485: rename d2/d25/f48 to d2/d15/d36/d6f/d9d/fa4 0 2024-04-24T08:25:03.156 INFO:tasks.workunit.client.1.smithi149.stdout:8/769: rmdir d1/d1f/d22 39 2024-04-24T08:25:03.158 INFO:tasks.workunit.client.1.smithi149.stdout:9/860: write d1/d2/d15/f58 [2124834,79517] 0 2024-04-24T08:25:03.158 INFO:tasks.workunit.client.0.smithi090.stdout:4/452: creat d3/df/f84 x:0 0 0 2024-04-24T08:25:03.159 INFO:tasks.workunit.client.1.smithi149.stdout:8/770: write d1/d1f/f8d [1336149,76160] 0 2024-04-24T08:25:03.160 INFO:tasks.workunit.client.0.smithi090.stdout:7/593: symlink db/d17/d39/d86/dae/lce 0 2024-04-24T08:25:03.162 INFO:tasks.workunit.client.0.smithi090.stdout:2/590: mkdir d0/d14/d17/dc2/dcb 0 2024-04-24T08:25:03.167 INFO:tasks.workunit.client.1.smithi149.stdout:9/861: creat d1/da/d13/d74/f13b x:0 0 0 2024-04-24T08:25:03.168 INFO:tasks.workunit.client.1.smithi149.stdout:8/771: symlink d1/dce/lfc 0 2024-04-24T08:25:03.172 INFO:tasks.workunit.client.1.smithi149.stdout:6/718: rename d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fb6 to d0/d1a/d28/d31/d4c/d80/d6f/dd7/ff6 0 2024-04-24T08:25:03.175 INFO:tasks.workunit.client.1.smithi149.stdout:0/757: getdents d2/d9/df/d2f/d6c/d72 0 2024-04-24T08:25:03.175 INFO:tasks.workunit.client.0.smithi090.stdout:4/453: creat d3/d38/f85 x:0 0 0 2024-04-24T08:25:03.177 INFO:tasks.workunit.client.0.smithi090.stdout:4/454: write d3/d18/d1d/f4e [984079,125615] 0 2024-04-24T08:25:03.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/772: creat d1/de/d18/d32/d63/ffd x:0 0 0 2024-04-24T08:25:03.179 INFO:tasks.workunit.client.1.smithi149.stdout:0/758: chown d2/d9/df/d40/f88 589433 1 2024-04-24T08:25:03.180 INFO:tasks.workunit.client.1.smithi149.stdout:8/773: fdatasync d1/de/d18/d91/fc5 0 2024-04-24T08:25:03.181 INFO:tasks.workunit.client.0.smithi090.stdout:7/594: unlink db/fc 0 2024-04-24T08:25:03.182 INFO:tasks.workunit.client.1.smithi149.stdout:8/774: stat d1/d1f/f28 0 2024-04-24T08:25:03.182 INFO:tasks.workunit.client.1.smithi149.stdout:0/759: chown d2/d9/df/d2f/d6c/dc1/d4e/c84 1433505990 1 2024-04-24T08:25:03.185 INFO:tasks.workunit.client.1.smithi149.stdout:9/862: creat d1/d2/d33/d5d/d83/dbf/dc4/f13c x:0 0 0 2024-04-24T08:25:03.186 INFO:tasks.workunit.client.0.smithi090.stdout:6/555: creat d8/dc/d19/d8e/fa8 x:0 0 0 2024-04-24T08:25:03.188 INFO:tasks.workunit.client.0.smithi090.stdout:8/471: write d2/da/d13/d34/fa0 [264459,58172] 0 2024-04-24T08:25:03.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/915: dread d0/d26/d3c/db1/fd0 [0,4194304] 0 2024-04-24T08:25:03.189 INFO:tasks.workunit.client.0.smithi090.stdout:6/556: truncate d8/d1d/d1f/d3e/d44/f82 145106 0 2024-04-24T08:25:03.190 INFO:tasks.workunit.client.0.smithi090.stdout:2/591: rename d0/d7/f64 to d0/d7/d67/d7f/d31/d46/d60/d65/fcc 0 2024-04-24T08:25:03.195 INFO:tasks.workunit.client.1.smithi149.stdout:6/719: creat d0/d1a/d28/d31/d4c/d80/dbd/ff7 x:0 0 0 2024-04-24T08:25:03.199 INFO:tasks.workunit.client.1.smithi149.stdout:0/760: fsync d2/d9/fe 0 2024-04-24T08:25:03.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/863: getdents d1/de/d9d/dfd/d135 0 2024-04-24T08:25:03.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/557: dwrite d8/d1d/d1f/d3e/f5c [4194304,4194304] 0 2024-04-24T08:25:03.205 INFO:tasks.workunit.client.0.smithi090.stdout:6/558: dread - d8/dc/d19/d8e/fa8 zero size 2024-04-24T08:25:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:8/775: creat d1/d1f/ffe x:0 0 0 2024-04-24T08:25:03.209 INFO:tasks.workunit.client.0.smithi090.stdout:2/592: dread d0/f55 [0,4194304] 0 2024-04-24T08:25:03.212 INFO:tasks.workunit.client.0.smithi090.stdout:9/519: creat d0/dc/f9c x:0 0 0 2024-04-24T08:25:03.212 INFO:tasks.workunit.client.0.smithi090.stdout:5/586: sync 2024-04-24T08:25:03.213 INFO:tasks.workunit.client.1.smithi149.stdout:6/720: dread - d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f88 zero size 2024-04-24T08:25:03.214 INFO:tasks.workunit.client.0.smithi090.stdout:3/486: creat d2/df/fa5 x:0 0 0 2024-04-24T08:25:03.216 INFO:tasks.workunit.client.0.smithi090.stdout:3/487: chown d2/df/d1a/d37/d53/c9c 22517 1 2024-04-24T08:25:03.219 INFO:tasks.workunit.client.1.smithi149.stdout:3/916: truncate d0/d5/ff 7754923 0 2024-04-24T08:25:03.221 INFO:tasks.workunit.client.1.smithi149.stdout:8/776: creat d1/de/d18/d32/d65/fff x:0 0 0 2024-04-24T08:25:03.224 INFO:tasks.workunit.client.1.smithi149.stdout:6/721: mkdir d0/d1a/d28/d31/d4c/df8 0 2024-04-24T08:25:03.224 INFO:tasks.workunit.client.0.smithi090.stdout:2/593: chown d0/d2c/d99/cb3 570288992 1 2024-04-24T08:25:03.226 INFO:tasks.workunit.client.0.smithi090.stdout:5/587: symlink d8/d56/lc0 0 2024-04-24T08:25:03.228 INFO:tasks.workunit.client.0.smithi090.stdout:5/588: chown d8/da/d18/fa5 338722 1 2024-04-24T08:25:03.229 INFO:tasks.workunit.client.0.smithi090.stdout:3/488: dread - d2/df/d1a/d37/d53/f65 zero size 2024-04-24T08:25:03.229 INFO:tasks.workunit.client.0.smithi090.stdout:8/472: dread d2/da/d13/f4b [0,4194304] 0 2024-04-24T08:25:03.230 INFO:tasks.workunit.client.0.smithi090.stdout:8/473: stat d2 0 2024-04-24T08:25:03.230 INFO:tasks.workunit.client.1.smithi149.stdout:9/864: write d1/d99/fdf [166581,113222] 0 2024-04-24T08:25:03.235 INFO:tasks.workunit.client.1.smithi149.stdout:6/722: mkdir d0/d1a/d28/d31/d4c/df8/df9 0 2024-04-24T08:25:03.235 INFO:tasks.workunit.client.1.smithi149.stdout:3/917: write d0/d5/da/d19/d1b/d72/fcd [654491,41761] 0 2024-04-24T08:25:03.237 INFO:tasks.workunit.client.1.smithi149.stdout:9/865: rmdir d1/d4c/d91 39 2024-04-24T08:25:03.237 INFO:tasks.workunit.client.1.smithi149.stdout:6/723: stat d0/d1a/d28/d31/f37 0 2024-04-24T08:25:03.237 INFO:tasks.workunit.client.1.smithi149.stdout:8/777: write d1/de/d18/daf/fcf [4870137,68249] 0 2024-04-24T08:25:03.238 INFO:tasks.workunit.client.0.smithi090.stdout:2/594: truncate d0/d14/f2b 1640812 0 2024-04-24T08:25:03.239 INFO:tasks.workunit.client.0.smithi090.stdout:4/455: getdents d3/d25 0 2024-04-24T08:25:03.239 INFO:tasks.workunit.client.1.smithi149.stdout:6/724: chown d0/d1a/d28/df3 60 1 2024-04-24T08:25:03.240 INFO:tasks.workunit.client.0.smithi090.stdout:7/595: getdents db/d17/d29/d38/d3e/d46 0 2024-04-24T08:25:03.243 INFO:tasks.workunit.client.0.smithi090.stdout:8/474: fsync d2/da/d13/d34/f6f 0 2024-04-24T08:25:03.244 INFO:tasks.workunit.client.0.smithi090.stdout:7/596: write db/d17/d29/d32/f3d [5223993,111826] 0 2024-04-24T08:25:03.246 INFO:tasks.workunit.client.0.smithi090.stdout:9/520: creat d0/d69/f9d x:0 0 0 2024-04-24T08:25:03.249 INFO:tasks.workunit.client.0.smithi090.stdout:8/475: write d2/d62/d6b/d9c/fa1 [327350,47864] 0 2024-04-24T08:25:03.250 INFO:tasks.workunit.client.1.smithi149.stdout:9/866: write d1/d2/f2b [1464484,29438] 0 2024-04-24T08:25:03.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/559: dwrite d8/d1d/f1e [4194304,4194304] 0 2024-04-24T08:25:03.252 INFO:tasks.workunit.client.0.smithi090.stdout:4/456: creat d3/d38/d49/f86 x:0 0 0 2024-04-24T08:25:03.255 INFO:tasks.workunit.client.0.smithi090.stdout:3/489: mkdir d2/d15/da6 0 2024-04-24T08:25:03.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/725: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 [0,4194304] 0 2024-04-24T08:25:03.262 INFO:tasks.workunit.client.0.smithi090.stdout:7/597: fdatasync db/d17/d29/f55 0 2024-04-24T08:25:03.262 INFO:tasks.workunit.client.0.smithi090.stdout:9/521: mknod d0/d19/c9e 0 2024-04-24T08:25:03.264 INFO:tasks.workunit.client.1.smithi149.stdout:6/726: readlink d0/d1a/d28/d31/d4c/l50 0 2024-04-24T08:25:03.265 INFO:tasks.workunit.client.0.smithi090.stdout:2/595: truncate d0/d7/fd 1788557 0 2024-04-24T08:25:03.266 INFO:tasks.workunit.client.1.smithi149.stdout:8/778: link d1/d1f/d22/l24 d1/d1f/d22/l100 0 2024-04-24T08:25:03.266 INFO:tasks.workunit.client.0.smithi090.stdout:8/476: rename d2/da/l68 to d2/d62/la2 0 2024-04-24T08:25:03.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/560: rmdir d8 39 2024-04-24T08:25:03.268 INFO:tasks.workunit.client.0.smithi090.stdout:4/457: mkdir d3/d87 0 2024-04-24T08:25:03.271 INFO:tasks.workunit.client.1.smithi149.stdout:3/918: dwrite d0/d5/da/d19/d1b/d72/f107 [0,4194304] 0 2024-04-24T08:25:03.272 INFO:tasks.workunit.client.1.smithi149.stdout:8/779: creat d1/d1f/d22/f101 x:0 0 0 2024-04-24T08:25:03.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:02 smithi149 ceph-mon[31282]: pgmap v24: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 90 MiB/s rd, 274 MiB/s wr, 531 op/s 2024-04-24T08:25:03.277 INFO:tasks.workunit.client.0.smithi090.stdout:8/477: mkdir d2/d62/d83/da3 0 2024-04-24T08:25:03.283 INFO:tasks.workunit.client.1.smithi149.stdout:9/867: write d1/da/f27 [8395730,44496] 0 2024-04-24T08:25:03.285 INFO:tasks.workunit.client.0.smithi090.stdout:5/589: truncate d8/da/d3a/d2d/d37/d93/d7a/d50/f6c 786851 0 2024-04-24T08:25:03.287 INFO:tasks.workunit.client.0.smithi090.stdout:2/596: mkdir d0/d14/d4d/db4/dcd 0 2024-04-24T08:25:03.289 INFO:tasks.workunit.client.0.smithi090.stdout:8/478: mknod d2/d62/d6b/ca4 0 2024-04-24T08:25:03.291 INFO:tasks.workunit.client.0.smithi090.stdout:4/458: symlink d3/df/d69/l88 0 2024-04-24T08:25:03.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/558: sync 2024-04-24T08:25:03.292 INFO:tasks.workunit.client.0.smithi090.stdout:0/537: sync 2024-04-24T08:25:03.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:02 smithi090 ceph-mon[25016]: pgmap v24: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 90 MiB/s rd, 274 MiB/s wr, 531 op/s 2024-04-24T08:25:03.294 INFO:tasks.workunit.client.0.smithi090.stdout:4/459: readlink d3/d24/l5f 0 2024-04-24T08:25:03.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/795: sync 2024-04-24T08:25:03.295 INFO:tasks.workunit.client.0.smithi090.stdout:0/538: chown f18 27002786 1 2024-04-24T08:25:03.296 INFO:tasks.workunit.client.1.smithi149.stdout:2/724: sync 2024-04-24T08:25:03.296 INFO:tasks.workunit.client.1.smithi149.stdout:1/903: sync 2024-04-24T08:25:03.296 INFO:tasks.workunit.client.1.smithi149.stdout:7/862: sync 2024-04-24T08:25:03.298 INFO:tasks.workunit.client.1.smithi149.stdout:8/780: getdents d1 0 2024-04-24T08:25:03.304 INFO:tasks.workunit.client.1.smithi149.stdout:8/781: write d1/d1f/d31/f84 [1958117,80461] 0 2024-04-24T08:25:03.304 INFO:tasks.workunit.client.1.smithi149.stdout:7/863: mkdir d9/dc/d11/d83/da6/d67/d69/d11c 0 2024-04-24T08:25:03.310 INFO:tasks.workunit.client.1.smithi149.stdout:2/725: mkdir d0/d6/db/d27/d64/d66/db1/ddb 0 2024-04-24T08:25:03.314 INFO:tasks.workunit.client.0.smithi090.stdout:3/490: rename d2/d25/f95 to d2/fa7 0 2024-04-24T08:25:03.315 INFO:tasks.workunit.client.1.smithi149.stdout:3/919: dwrite d0/d26/d3c/db1/fd0 [8388608,4194304] 0 2024-04-24T08:25:03.316 INFO:tasks.workunit.client.0.smithi090.stdout:7/598: truncate db/d17/d39/f65 2984452 0 2024-04-24T08:25:03.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/561: truncate d8/dc/d19/f64 3071151 0 2024-04-24T08:25:03.319 INFO:tasks.workunit.client.0.smithi090.stdout:6/562: dread - d8/dc/d19/d8e/fa8 zero size 2024-04-24T08:25:03.321 INFO:tasks.workunit.client.1.smithi149.stdout:1/904: unlink d0/d1b/db5/c54 0 2024-04-24T08:25:03.329 INFO:tasks.workunit.client.1.smithi149.stdout:9/868: dwrite d1/da/dd/d79/db6/ffa [0,4194304] 0 2024-04-24T08:25:03.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/920: creat d0/d5/da/d5a/d9c/f134 x:0 0 0 2024-04-24T08:25:03.338 INFO:tasks.workunit.client.1.smithi149.stdout:8/782: symlink d1/de/d18/d32/d65/d8b/l102 0 2024-04-24T08:25:03.339 INFO:tasks.workunit.client.1.smithi149.stdout:1/905: dread - d0/ddd/fa2 zero size 2024-04-24T08:25:03.342 INFO:tasks.workunit.client.1.smithi149.stdout:7/864: dwrite d9/d43/d4f/da5/f10e [0,4194304] 0 2024-04-24T08:25:03.345 INFO:tasks.workunit.client.1.smithi149.stdout:8/783: write d1/de/d18/d32/d63/d9e/fd6 [1751371,48829] 0 2024-04-24T08:25:03.347 INFO:tasks.workunit.client.1.smithi149.stdout:9/869: symlink d1/d2/d33/dea/d133/l13d 0 2024-04-24T08:25:03.351 INFO:tasks.workunit.client.0.smithi090.stdout:0/539: dread d1a/d4b/f6f [0,4194304] 0 2024-04-24T08:25:03.352 INFO:tasks.workunit.client.1.smithi149.stdout:1/906: dread - d0/d5/d4f/ff7 zero size 2024-04-24T08:25:03.353 INFO:tasks.workunit.client.1.smithi149.stdout:3/921: mkdir d0/d5/d135 0 2024-04-24T08:25:03.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/870: truncate d1/d4c/f138 51902 0 2024-04-24T08:25:03.358 INFO:tasks.workunit.client.1.smithi149.stdout:7/865: fdatasync d9/dc/de/f12 0 2024-04-24T08:25:03.375 INFO:tasks.workunit.client.0.smithi090.stdout:1/559: truncate d3/d5/da/f82 1880930 0 2024-04-24T08:25:03.377 INFO:tasks.workunit.client.1.smithi149.stdout:1/907: rename d0/d5/db/d63/f6e to d0/ddf/da0/db8/dc5/f125 0 2024-04-24T08:25:03.378 INFO:tasks.workunit.client.1.smithi149.stdout:2/726: dread d0/f21 [0,4194304] 0 2024-04-24T08:25:03.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/871: dwrite d1/da/f8 [0,4194304] 0 2024-04-24T08:25:03.387 INFO:tasks.workunit.client.0.smithi090.stdout:9/522: rename d0/d5e to d0/dc/d24/d7c/d4d/d32/d9f 0 2024-04-24T08:25:03.394 INFO:tasks.workunit.client.0.smithi090.stdout:1/560: dwrite d3/d1d/f55 [0,4194304] 0 2024-04-24T08:25:03.395 INFO:tasks.workunit.client.0.smithi090.stdout:3/491: fsync d2/df/d5b/f8e 0 2024-04-24T08:25:03.397 INFO:tasks.workunit.client.1.smithi149.stdout:3/922: write d0/d5/ff [916510,113350] 0 2024-04-24T08:25:03.398 INFO:tasks.workunit.client.1.smithi149.stdout:9/872: creat d1/d2/d15/d1c/d42/dae/f13e x:0 0 0 2024-04-24T08:25:03.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/563: fdatasync d8/dc/d19/d8e/f4b 0 2024-04-24T08:25:03.404 INFO:tasks.workunit.client.0.smithi090.stdout:2/597: write d0/d14/d17/f62 [300753,46941] 0 2024-04-24T08:25:03.405 INFO:tasks.workunit.client.0.smithi090.stdout:1/561: truncate d3/d5/da/d97/fbf 411642 0 2024-04-24T08:25:03.405 INFO:tasks.workunit.client.1.smithi149.stdout:8/784: getdents d1/de/d18/d32/d63/d76 0 2024-04-24T08:25:03.406 INFO:tasks.workunit.client.1.smithi149.stdout:7/866: write d9/dc/de/d44/fbe [4255582,8853] 0 2024-04-24T08:25:03.406 INFO:tasks.workunit.client.0.smithi090.stdout:5/590: write d8/da/d3a/d2d/f82 [1473438,22116] 0 2024-04-24T08:25:03.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/598: write d0/d14/d36/fca [792609,71738] 0 2024-04-24T08:25:03.409 INFO:tasks.workunit.client.1.smithi149.stdout:7/867: write d9/dc/d2e/d3c/ffa [408684,58126] 0 2024-04-24T08:25:03.410 INFO:tasks.workunit.client.0.smithi090.stdout:0/540: unlink d1a/d4b/d53/d79/d35/d62/f7a 0 2024-04-24T08:25:03.412 INFO:tasks.workunit.client.0.smithi090.stdout:2/599: stat d0/d7/d67/d7f/f87 0 2024-04-24T08:25:03.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/460: link d3/d38/f4d d3/df/d77/f89 0 2024-04-24T08:25:03.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/908: write d0/d5/db/d18/d4d/f90 [992221,49764] 0 2024-04-24T08:25:03.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/599: rename db/d17/d29/d32/l4d to db/d17/d29/d38/d3e/lcf 0 2024-04-24T08:25:03.425 INFO:tasks.workunit.client.0.smithi090.stdout:3/492: fdatasync d2/d15/f16 0 2024-04-24T08:25:03.428 INFO:tasks.workunit.client.0.smithi090.stdout:3/493: write d2/df/d1a/d37/d53/f6b [34317,88542] 0 2024-04-24T08:25:03.431 INFO:tasks.workunit.client.0.smithi090.stdout:5/591: symlink d8/da/d3a/d2d/lc1 0 2024-04-24T08:25:03.434 INFO:tasks.workunit.client.0.smithi090.stdout:2/600: unlink d0/d14/d4d/d50/c88 0 2024-04-24T08:25:03.434 INFO:tasks.workunit.client.0.smithi090.stdout:8/479: sync 2024-04-24T08:25:03.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/541: creat d1a/d4b/d53/d79/d35/d62/fab x:0 0 0 2024-04-24T08:25:03.439 INFO:tasks.workunit.client.1.smithi149.stdout:7/868: dread d9/d43/f7e [0,4194304] 0 2024-04-24T08:25:03.439 INFO:tasks.workunit.client.0.smithi090.stdout:9/523: write d0/dc/d24/d2d/f5d [4267575,38401] 0 2024-04-24T08:25:03.439 INFO:tasks.workunit.client.1.smithi149.stdout:7/869: stat d9/dc/d11 0 2024-04-24T08:25:03.440 INFO:tasks.workunit.client.0.smithi090.stdout:6/564: rename d8/dc/d19/d8e/f54 to d8/d1d/d1f/d3e/d44/d90/fa9 0 2024-04-24T08:25:03.442 INFO:tasks.workunit.client.0.smithi090.stdout:9/524: chown d0/d19/f25 31 1 2024-04-24T08:25:03.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/601: rmdir d0/d2c/d99 39 2024-04-24T08:25:03.454 INFO:tasks.workunit.client.0.smithi090.stdout:9/525: truncate d0/dc/d24/d2d/d49/d81/f9a 863911 0 2024-04-24T08:25:03.456 INFO:tasks.workunit.client.0.smithi090.stdout:5/592: dwrite d8/d76/fb2 [0,4194304] 0 2024-04-24T08:25:03.467 INFO:tasks.workunit.client.0.smithi090.stdout:7/600: dwrite db/d17/d29/d32/d5a/d75/f26 [0,4194304] 0 2024-04-24T08:25:03.469 INFO:tasks.workunit.client.0.smithi090.stdout:6/565: mkdir d8/d1d/d1f/d3e/d4f/d7e/daa 0 2024-04-24T08:25:03.470 INFO:tasks.workunit.client.0.smithi090.stdout:3/494: write d2/df/d1a/d37/f55 [959057,120004] 0 2024-04-24T08:25:03.472 INFO:tasks.workunit.client.0.smithi090.stdout:1/562: creat d3/d1d/fc9 x:0 0 0 2024-04-24T08:25:03.475 INFO:tasks.workunit.client.0.smithi090.stdout:4/461: rmdir d3/df/d7c/d81 0 2024-04-24T08:25:03.475 INFO:tasks.workunit.client.0.smithi090.stdout:5/593: mkdir d8/da/d14/d2f/dc2 0 2024-04-24T08:25:03.478 INFO:tasks.workunit.client.0.smithi090.stdout:9/526: dread d0/dc/d24/d7c/d4d/f5f [0,4194304] 0 2024-04-24T08:25:03.484 INFO:tasks.workunit.client.0.smithi090.stdout:6/566: mknod d8/dc/d19/d8e/d81/d96/cab 0 2024-04-24T08:25:03.486 INFO:tasks.workunit.client.1.smithi149.stdout:2/727: link d0/d6/db/d39/d78/fac d0/d6/db/d27/d81/fdc 0 2024-04-24T08:25:03.490 INFO:tasks.workunit.client.0.smithi090.stdout:4/462: rmdir d3/d38/d49 39 2024-04-24T08:25:03.491 INFO:tasks.workunit.client.1.smithi149.stdout:1/909: unlink d0/d5/db/d18/d2f/f3d 0 2024-04-24T08:25:03.492 INFO:tasks.workunit.client.1.smithi149.stdout:1/910: chown d0/d1b/db5/d108 35298 1 2024-04-24T08:25:03.493 INFO:tasks.workunit.client.0.smithi090.stdout:2/602: dwrite d0/d14/d17/f18 [0,4194304] 0 2024-04-24T08:25:03.495 INFO:tasks.workunit.client.1.smithi149.stdout:7/870: creat d9/d43/d4f/d66/f11d x:0 0 0 2024-04-24T08:25:03.495 INFO:tasks.workunit.client.1.smithi149.stdout:9/873: link d1/d2/d15/d1c/l53 d1/de/l13f 0 2024-04-24T08:25:03.497 INFO:tasks.workunit.client.0.smithi090.stdout:5/594: symlink d8/da/d3a/d2d/d37/d93/d7a/d50/d9f/lc3 0 2024-04-24T08:25:03.498 INFO:tasks.workunit.client.0.smithi090.stdout:0/542: dwrite d1a/d4b/d53/f8b [0,4194304] 0 2024-04-24T08:25:03.498 INFO:tasks.workunit.client.0.smithi090.stdout:6/567: truncate d8/dc/d19/d8e/f7b 906461 0 2024-04-24T08:25:03.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/463: chown d3/d38/l55 10597 1 2024-04-24T08:25:03.499 INFO:tasks.workunit.client.1.smithi149.stdout:1/911: unlink d0/d1b/d4b/da1/cb3 0 2024-04-24T08:25:03.504 INFO:tasks.workunit.client.0.smithi090.stdout:0/543: write d1a/d4b/d53/d79/d69/d8a/fa2 [174388,99613] 0 2024-04-24T08:25:03.511 INFO:tasks.workunit.client.0.smithi090.stdout:3/495: dread d2/d15/d36/d6f/d7a/f8f [0,4194304] 0 2024-04-24T08:25:03.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/595: mknod d8/da/d3a/d96/cc4 0 2024-04-24T08:25:03.517 INFO:tasks.workunit.client.0.smithi090.stdout:8/480: dread d2/da/d13/d64/d24/f79 [0,4194304] 0 2024-04-24T08:25:03.521 INFO:tasks.workunit.client.0.smithi090.stdout:1/563: rename d3/d5/da/d94/l93 to d3/d1d/lca 0 2024-04-24T08:25:03.525 INFO:tasks.workunit.client.0.smithi090.stdout:4/464: symlink d3/d87/l8a 0 2024-04-24T08:25:03.528 INFO:tasks.workunit.client.1.smithi149.stdout:3/923: truncate d0/d5/d53/f3f 2377284 0 2024-04-24T08:25:03.531 INFO:tasks.workunit.client.1.smithi149.stdout:2/728: write d0/d6/db/d27/d64/f6e [4691337,34093] 0 2024-04-24T08:25:03.534 INFO:tasks.workunit.client.0.smithi090.stdout:3/496: dread d2/f24 [0,4194304] 0 2024-04-24T08:25:03.539 INFO:tasks.workunit.client.0.smithi090.stdout:8/481: creat d2/d62/d83/da3/fa5 x:0 0 0 2024-04-24T08:25:03.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/544: dread d1a/d4b/d53/f6a [0,4194304] 0 2024-04-24T08:25:03.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/482: chown d2/f12 1574 1 2024-04-24T08:25:03.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/912: write d0/d5/db/d63/f6b [1417083,677] 0 2024-04-24T08:25:03.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/913: readlink d0/d1b/l10e 0 2024-04-24T08:25:03.543 INFO:tasks.workunit.client.0.smithi090.stdout:2/603: rename d0/d2c/d99 to d0/d14/d36/dce 0 2024-04-24T08:25:03.543 INFO:tasks.workunit.client.1.smithi149.stdout:9/874: dread d1/da/dd/d79/db6/feb [0,4194304] 0 2024-04-24T08:25:03.545 INFO:tasks.workunit.client.1.smithi149.stdout:2/729: creat d0/d6/db/d27/d64/db3/fdd x:0 0 0 2024-04-24T08:25:03.546 INFO:tasks.workunit.client.1.smithi149.stdout:8/785: dread d1/de/d18/f45 [0,4194304] 0 2024-04-24T08:25:03.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/483: write d2/da/d13/d64/d50/f5d [699875,60350] 0 2024-04-24T08:25:03.547 INFO:tasks.workunit.client.0.smithi090.stdout:6/568: write d8/fa [3925868,5343] 0 2024-04-24T08:25:03.551 INFO:tasks.workunit.client.0.smithi090.stdout:3/497: rename d2/d15/f16 to d2/df/d1a/d37/d4b/fa8 0 2024-04-24T08:25:03.551 INFO:tasks.workunit.client.1.smithi149.stdout:7/871: dwrite d9/f8b [0,4194304] 0 2024-04-24T08:25:03.552 INFO:tasks.workunit.client.0.smithi090.stdout:2/604: creat d0/d14/d4d/d50/d96/fcf x:0 0 0 2024-04-24T08:25:03.554 INFO:tasks.workunit.client.0.smithi090.stdout:5/596: dread d8/da/d3a/d2d/d37/d93/d7a/d50/f71 [0,4194304] 0 2024-04-24T08:25:03.556 INFO:tasks.workunit.client.1.smithi149.stdout:1/914: rmdir d0/ddf/d53/d61 39 2024-04-24T08:25:03.557 INFO:tasks.workunit.client.0.smithi090.stdout:4/465: dwrite d3/d38/d49/d6f/f51 [0,4194304] 0 2024-04-24T08:25:03.557 INFO:tasks.workunit.client.1.smithi149.stdout:3/924: getdents d0/d5/da/d5a/d63/d6a 0 2024-04-24T08:25:03.560 INFO:tasks.workunit.client.1.smithi149.stdout:2/730: mknod d0/d6/db/d28/db0/cde 0 2024-04-24T08:25:03.564 INFO:tasks.workunit.client.1.smithi149.stdout:2/731: chown d0/d6/db/d39/d78/d8a 15 1 2024-04-24T08:25:03.564 INFO:tasks.workunit.client.0.smithi090.stdout:0/545: rename d1a/d4b/d53/d79/d35/c73 to d1a/d4b/d53/d79/d58/cac 0 2024-04-24T08:25:03.566 INFO:tasks.workunit.client.1.smithi149.stdout:8/786: rename d1/d1f/d5e/le8 to d1/d1f/d22/l103 0 2024-04-24T08:25:03.568 INFO:tasks.workunit.client.0.smithi090.stdout:8/484: symlink d2/da/d13/d64/d50/la6 0 2024-04-24T08:25:03.568 INFO:tasks.workunit.client.1.smithi149.stdout:9/875: mknod d1/da/c140 0 2024-04-24T08:25:03.570 INFO:tasks.workunit.client.1.smithi149.stdout:0/761: sync 2024-04-24T08:25:03.570 INFO:tasks.workunit.client.1.smithi149.stdout:5/995: sync 2024-04-24T08:25:03.570 INFO:tasks.workunit.client.1.smithi149.stdout:6/727: sync 2024-04-24T08:25:03.570 INFO:tasks.workunit.client.1.smithi149.stdout:4/796: sync 2024-04-24T08:25:03.571 INFO:tasks.workunit.client.0.smithi090.stdout:6/569: dwrite d8/d1d/d1f/d3e/d4f/d7e/f80 [0,4194304] 0 2024-04-24T08:25:03.574 INFO:tasks.workunit.client.1.smithi149.stdout:5/996: fdatasync db/d12/d1e/d12d/f148 0 2024-04-24T08:25:03.581 INFO:tasks.workunit.client.0.smithi090.stdout:4/466: dwrite d3/d18/f70 [0,4194304] 0 2024-04-24T08:25:03.582 INFO:tasks.workunit.client.0.smithi090.stdout:2/605: symlink d0/d14/d36/d77/ld0 0 2024-04-24T08:25:03.583 INFO:tasks.workunit.client.0.smithi090.stdout:1/564: dwrite d3/d5/da/f6a [0,4194304] 0 2024-04-24T08:25:03.584 INFO:tasks.workunit.client.1.smithi149.stdout:3/925: creat d0/d5/da/d5e/f136 x:0 0 0 2024-04-24T08:25:03.586 INFO:tasks.workunit.client.1.smithi149.stdout:2/732: fdatasync d0/d6/db/d39/d3d/fb5 0 2024-04-24T08:25:03.586 INFO:tasks.workunit.client.0.smithi090.stdout:1/565: fdatasync d3/d5/da/dc/f87 0 2024-04-24T08:25:03.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/762: chown d2/d9/df/f1d 1411649654 1 2024-04-24T08:25:03.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/485: creat d2/d62/d71/fa7 x:0 0 0 2024-04-24T08:25:03.590 INFO:tasks.workunit.client.0.smithi090.stdout:6/570: creat d8/d1d/d1f/fac x:0 0 0 2024-04-24T08:25:03.594 INFO:tasks.workunit.client.0.smithi090.stdout:3/498: creat d2/d15/d36/d6f/d92/fa9 x:0 0 0 2024-04-24T08:25:03.595 INFO:tasks.workunit.client.0.smithi090.stdout:6/571: stat d8/d1d/d1f/d3e/d8f 0 2024-04-24T08:25:03.595 INFO:tasks.workunit.client.1.smithi149.stdout:5/997: symlink db/d12/d1e/d65/de8/dfa/l159 0 2024-04-24T08:25:03.596 INFO:tasks.workunit.client.0.smithi090.stdout:5/597: creat d8/d9c/fc5 x:0 0 0 2024-04-24T08:25:03.598 INFO:tasks.workunit.client.1.smithi149.stdout:8/787: creat d1/dce/de9/f104 x:0 0 0 2024-04-24T08:25:03.600 INFO:tasks.workunit.client.0.smithi090.stdout:5/598: write d8/da/d18/fa5 [100239,86195] 0 2024-04-24T08:25:03.601 INFO:tasks.workunit.client.0.smithi090.stdout:5/599: fsync d8/f1c 0 2024-04-24T08:25:03.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/486: fsync d2/da/d13/d64/d24/f79 0 2024-04-24T08:25:03.616 INFO:tasks.workunit.client.1.smithi149.stdout:8/788: symlink d1/l105 0 2024-04-24T08:25:03.619 INFO:tasks.workunit.client.1.smithi149.stdout:1/915: write d0/d5/db/d82/f9b [1079755,126160] 0 2024-04-24T08:25:03.621 INFO:tasks.workunit.client.1.smithi149.stdout:2/733: dread d0/d6/db/d39/d3d/f51 [0,4194304] 0 2024-04-24T08:25:03.622 INFO:tasks.workunit.client.1.smithi149.stdout:5/998: dwrite db/d12/d2f/d74/dc6/d113/f10d [0,4194304] 0 2024-04-24T08:25:03.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/872: rename d9/dc/d11/d38 to d9/dc/d2e/d11e 0 2024-04-24T08:25:03.624 INFO:tasks.workunit.client.1.smithi149.stdout:2/734: chown d0/d6/l2f 2 1 2024-04-24T08:25:03.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/999: chown db/d12/fb0 7408 1 2024-04-24T08:25:03.627 INFO:tasks.workunit.client.0.smithi090.stdout:3/499: rename d2/df/d1a/d37/d59/l5d to d2/d25/d33/laa 0 2024-04-24T08:25:03.630 INFO:tasks.workunit.client.1.smithi149.stdout:0/763: creat d2/d9/d1a/d36/d85/db5/def/ff3 x:0 0 0 2024-04-24T08:25:03.635 INFO:tasks.workunit.client.1.smithi149.stdout:6/728: rename d0/d1a/d28/d31/d4c/d4e/d62/c8b to d0/d1a/d28/d31/d38/d54/cfa 0 2024-04-24T08:25:03.636 INFO:tasks.workunit.client.1.smithi149.stdout:0/764: chown d2/d9/df/d2f/d6c/f77 2965977 1 2024-04-24T08:25:03.638 INFO:tasks.workunit.client.0.smithi090.stdout:0/546: rmdir d1a/d4b/d53/d79/d83 0 2024-04-24T08:25:03.640 INFO:tasks.workunit.client.1.smithi149.stdout:2/735: fdatasync d0/d6/db/d39/d3d/fb6 0 2024-04-24T08:25:03.644 INFO:tasks.workunit.client.0.smithi090.stdout:8/487: stat d2/da/d13/f28 0 2024-04-24T08:25:03.644 INFO:tasks.workunit.client.1.smithi149.stdout:3/926: dread d0/d88/f8e [0,4194304] 0 2024-04-24T08:25:03.645 INFO:tasks.workunit.client.1.smithi149.stdout:0/765: creat d2/d9/d1a/d36/d85/dcd/dda/ff4 x:0 0 0 2024-04-24T08:25:03.647 INFO:tasks.workunit.client.1.smithi149.stdout:6/729: dread - d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/f94 zero size 2024-04-24T08:25:03.649 INFO:tasks.workunit.client.0.smithi090.stdout:2/606: write d0/d7/d67/d7f/f78 [645260,64878] 0 2024-04-24T08:25:03.652 INFO:tasks.workunit.client.0.smithi090.stdout:5/600: rename d8/da/f35 to d8/d7c/da9/fc6 0 2024-04-24T08:25:03.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/797: truncate d2/d10/f27 2897852 0 2024-04-24T08:25:03.658 INFO:tasks.workunit.client.0.smithi090.stdout:0/547: creat d1a/d4b/d53/d79/d3c/fad x:0 0 0 2024-04-24T08:25:03.661 INFO:tasks.workunit.client.1.smithi149.stdout:4/798: read d2/d24/d2e/d38/f5c [129024,108436] 0 2024-04-24T08:25:03.662 INFO:tasks.workunit.client.1.smithi149.stdout:0/766: rename d2/d9/d1a/d36/l3b to d2/d9/d1a/d36/d85/db5/lf5 0 2024-04-24T08:25:03.663 INFO:tasks.workunit.client.1.smithi149.stdout:3/927: dwrite d0/d26/d3c/db1/f12e [0,4194304] 0 2024-04-24T08:25:03.664 INFO:tasks.workunit.client.0.smithi090.stdout:8/488: symlink d2/da/d13/d60/la8 0 2024-04-24T08:25:03.666 INFO:tasks.workunit.client.1.smithi149.stdout:7/873: link d9/dc/d2e/d3c/db8/le6 d9/dc/d11/d26/l11f 0 2024-04-24T08:25:03.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/789: chown d1/de/f38 0 1 2024-04-24T08:25:03.672 INFO:tasks.workunit.client.0.smithi090.stdout:4/467: getdents d3/d18/d5c 0 2024-04-24T08:25:03.675 INFO:tasks.workunit.client.1.smithi149.stdout:6/730: rename d0/d1a/d28/d31/d4c/d4e/lbb to d0/d1a/d28/d31/d38/lfb 0 2024-04-24T08:25:03.677 INFO:tasks.workunit.client.1.smithi149.stdout:0/767: creat d2/d9/d1a/d36/dcc/ff6 x:0 0 0 2024-04-24T08:25:03.677 INFO:tasks.workunit.client.0.smithi090.stdout:0/548: symlink d1a/d4b/d53/d79/d69/lae 0 2024-04-24T08:25:03.679 INFO:tasks.workunit.client.0.smithi090.stdout:8/489: mknod d2/da/d13/d60/ca9 0 2024-04-24T08:25:03.679 INFO:tasks.workunit.client.1.smithi149.stdout:2/736: dwrite d0/d6/db/d28/d7f/d8f/fcc [0,4194304] 0 2024-04-24T08:25:03.681 INFO:tasks.workunit.client.0.smithi090.stdout:6/572: getdents d8/dc/d19/d8e/d81 0 2024-04-24T08:25:03.682 INFO:tasks.workunit.client.1.smithi149.stdout:3/928: mknod d0/d5/da/d5e/c137 0 2024-04-24T08:25:03.683 INFO:tasks.workunit.client.1.smithi149.stdout:3/929: stat d0/d5/laa 0 2024-04-24T08:25:03.684 INFO:tasks.workunit.client.0.smithi090.stdout:5/601: dread f6 [0,4194304] 0 2024-04-24T08:25:03.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/573: truncate d8/fa 4426195 0 2024-04-24T08:25:03.684 INFO:tasks.workunit.client.0.smithi090.stdout:4/468: mkdir d3/df/d69/d8b 0 2024-04-24T08:25:03.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/500: dread d2/d15/f31 [0,4194304] 0 2024-04-24T08:25:03.686 INFO:tasks.workunit.client.0.smithi090.stdout:0/549: creat d1a/d4b/d53/d57/faf x:0 0 0 2024-04-24T08:25:03.692 INFO:tasks.workunit.client.0.smithi090.stdout:8/490: mknod d2/d62/d71/caa 0 2024-04-24T08:25:03.693 INFO:tasks.workunit.client.0.smithi090.stdout:3/501: truncate d2/d15/d36/d6f/f9a 465610 0 2024-04-24T08:25:03.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/502: chown d2/d84/fa3 15643 1 2024-04-24T08:25:03.697 INFO:tasks.workunit.client.1.smithi149.stdout:3/930: dread d0/d5/f16 [0,4194304] 0 2024-04-24T08:25:03.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/550: rmdir d1a/d4b/d53/d79/d82 39 2024-04-24T08:25:03.703 INFO:tasks.workunit.client.1.smithi149.stdout:8/790: symlink d1/de/d18/d32/l106 0 2024-04-24T08:25:03.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/799: creat d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/fff x:0 0 0 2024-04-24T08:25:03.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/731: symlink d0/d1a/d28/df3/lfc 0 2024-04-24T08:25:03.712 INFO:tasks.workunit.client.1.smithi149.stdout:0/768: creat d2/d9/df/d48/dba/ff7 x:0 0 0 2024-04-24T08:25:03.713 INFO:tasks.workunit.client.1.smithi149.stdout:3/931: read - d0/d5/da/d19/f9f zero size 2024-04-24T08:25:03.715 INFO:tasks.workunit.client.1.smithi149.stdout:2/737: mkdir d0/d6/db/d28/d7f/d8f/db9/ddf 0 2024-04-24T08:25:03.717 INFO:tasks.workunit.client.0.smithi090.stdout:6/574: creat d8/dc/fad x:0 0 0 2024-04-24T08:25:03.717 INFO:tasks.workunit.client.1.smithi149.stdout:2/738: chown d0/d6/db/d27/d64/d66/db1 7955609 1 2024-04-24T08:25:03.720 INFO:tasks.workunit.client.0.smithi090.stdout:0/551: creat d1a/d4b/d53/d79/d69/d78/d9b/fb0 x:0 0 0 2024-04-24T08:25:03.720 INFO:tasks.workunit.client.0.smithi090.stdout:6/575: read d8/f46 [2871589,125318] 0 2024-04-24T08:25:03.721 INFO:tasks.workunit.client.1.smithi149.stdout:2/739: read d0/d6/fae [741263,38077] 0 2024-04-24T08:25:03.722 INFO:tasks.workunit.client.1.smithi149.stdout:6/732: write d0/d1a/d28/d31/d4c/d8a/fb5 [2080864,44410] 0 2024-04-24T08:25:03.723 INFO:tasks.workunit.client.0.smithi090.stdout:5/602: rename d8/da/d3a/d2d/d37/d93 to d8/d59/dc7 0 2024-04-24T08:25:03.725 INFO:tasks.workunit.client.1.smithi149.stdout:7/874: dwrite d9/dc/d11/d83/da6/f52 [0,4194304] 0 2024-04-24T08:25:03.726 INFO:tasks.workunit.client.0.smithi090.stdout:3/503: creat d2/d15/fab x:0 0 0 2024-04-24T08:25:03.727 INFO:tasks.workunit.client.1.smithi149.stdout:4/800: rmdir d2/d18/d1e/d3c 39 2024-04-24T08:25:03.729 INFO:tasks.workunit.client.0.smithi090.stdout:4/469: getdents d3/d38/d49/d6f 0 2024-04-24T08:25:03.732 INFO:tasks.workunit.client.1.smithi149.stdout:8/791: truncate d1/d1f/f2c 882848 0 2024-04-24T08:25:03.735 INFO:tasks.workunit.client.1.smithi149.stdout:2/740: mkdir d0/d6/db/d27/d64/db3/de0 0 2024-04-24T08:25:03.737 INFO:tasks.workunit.client.1.smithi149.stdout:2/741: dread - d0/d6/db/d27/d64/fa4 zero size 2024-04-24T08:25:03.738 INFO:tasks.workunit.client.1.smithi149.stdout:0/769: mknod d2/d9/d1a/cf8 0 2024-04-24T08:25:03.738 INFO:tasks.workunit.client.0.smithi090.stdout:0/552: chown d1a/d4b/d53/d79/d58/cac 368293839 1 2024-04-24T08:25:03.739 INFO:tasks.workunit.client.1.smithi149.stdout:7/875: dread d9/f8b [0,4194304] 0 2024-04-24T08:25:03.740 INFO:tasks.workunit.client.0.smithi090.stdout:8/491: rename d2/da/d13/d64/d24/l56 to d2/da/d13/d60/d91/lab 0 2024-04-24T08:25:03.743 INFO:tasks.workunit.client.1.smithi149.stdout:6/733: creat d0/d1a/d28/d31/d4c/df8/ffd x:0 0 0 2024-04-24T08:25:03.745 INFO:tasks.workunit.client.1.smithi149.stdout:6/734: chown d0/d1a/d28/d31/d38/l8f 2097003 1 2024-04-24T08:25:03.747 INFO:tasks.workunit.client.1.smithi149.stdout:4/801: mknod d2/d24/d2e/d3f/c100 0 2024-04-24T08:25:03.748 INFO:tasks.workunit.client.0.smithi090.stdout:3/504: creat d2/d15/d36/d6f/d92/fac x:0 0 0 2024-04-24T08:25:03.749 INFO:tasks.workunit.client.1.smithi149.stdout:8/792: rename d1/de/d18/daf/lcc to d1/de/d18/dd3/l107 0 2024-04-24T08:25:03.750 INFO:tasks.workunit.client.0.smithi090.stdout:4/470: creat d3/d38/d6b/f8c x:0 0 0 2024-04-24T08:25:03.752 INFO:tasks.workunit.client.1.smithi149.stdout:3/932: dwrite d0/d5/d53/d6d/fea [0,4194304] 0 2024-04-24T08:25:03.754 INFO:tasks.workunit.client.0.smithi090.stdout:0/553: readlink d1a/l45 0 2024-04-24T08:25:03.755 INFO:tasks.workunit.client.0.smithi090.stdout:0/554: dread - d1a/d4b/d53/d79/d35/d62/fab zero size 2024-04-24T08:25:03.758 INFO:tasks.workunit.client.1.smithi149.stdout:7/876: creat d9/dc/d7b/dc2/f120 x:0 0 0 2024-04-24T08:25:03.761 INFO:tasks.workunit.client.1.smithi149.stdout:7/877: dread - d9/d43/d4f/d66/f11d zero size 2024-04-24T08:25:03.765 INFO:tasks.workunit.client.1.smithi149.stdout:4/802: readlink d2/lb0 0 2024-04-24T08:25:03.766 INFO:tasks.workunit.client.0.smithi090.stdout:5/603: link d8/d76/fb2 d8/d25/fc8 0 2024-04-24T08:25:03.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/742: rename d0/d6/db/d28/d7f/d9b/la1 to d0/d6/db/d39/d3d/d44/d8b/le1 0 2024-04-24T08:25:03.773 INFO:tasks.workunit.client.1.smithi149.stdout:2/743: dread - d0/d6/db/d27/d64/db3/fdd zero size 2024-04-24T08:25:03.774 INFO:tasks.workunit.client.1.smithi149.stdout:0/770: unlink d2/d17/c83 0 2024-04-24T08:25:03.776 INFO:tasks.workunit.client.1.smithi149.stdout:7/878: rmdir d9/dc/d11/d84/da7 39 2024-04-24T08:25:03.779 INFO:tasks.workunit.client.1.smithi149.stdout:2/744: read d0/d6/db/d39/d78/d7c/f97 [980873,78693] 0 2024-04-24T08:25:03.779 INFO:tasks.workunit.client.0.smithi090.stdout:6/576: truncate d8/d1d/d1f/d3e/f5c 5973280 0 2024-04-24T08:25:03.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/735: dwrite d0/d1a/d28/d4a/f72 [0,4194304] 0 2024-04-24T08:25:03.783 INFO:tasks.workunit.client.0.smithi090.stdout:0/555: creat d1a/d4b/d53/d79/d58/d96/fb1 x:0 0 0 2024-04-24T08:25:03.784 INFO:tasks.workunit.client.1.smithi149.stdout:6/736: stat d0/d1a/d28/d31/c5b 0 2024-04-24T08:25:03.786 INFO:tasks.workunit.client.0.smithi090.stdout:5/604: write d8/da/d14/d2f/d44/f97 [113675,53353] 0 2024-04-24T08:25:03.787 INFO:tasks.workunit.client.1.smithi149.stdout:8/793: rename d1/d1f/d5e/f8c to d1/d1f/d22/f108 0 2024-04-24T08:25:03.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/471: rename d3/d24/f6a to d3/d38/f8d 0 2024-04-24T08:25:03.792 INFO:tasks.workunit.client.0.smithi090.stdout:6/577: creat d8/dc/d19/d8e/d81/d96/d97/fae x:0 0 0 2024-04-24T08:25:03.793 INFO:tasks.workunit.client.1.smithi149.stdout:7/879: creat d9/dc/d2e/d3c/db8/f121 x:0 0 0 2024-04-24T08:25:03.797 INFO:tasks.workunit.client.1.smithi149.stdout:0/771: creat d2/d9/de6/ff9 x:0 0 0 2024-04-24T08:25:03.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/492: getdents d2/d62/d6b 0 2024-04-24T08:25:03.801 INFO:tasks.workunit.client.1.smithi149.stdout:8/794: rename d1/de/d18/d32/ff9 to d1/d1f/d5e/d92/f109 0 2024-04-24T08:25:03.803 INFO:tasks.workunit.client.1.smithi149.stdout:7/880: symlink d9/dc/d11/d83/l122 0 2024-04-24T08:25:03.805 INFO:tasks.workunit.client.0.smithi090.stdout:4/472: mkdir d3/d8e 0 2024-04-24T08:25:03.807 INFO:tasks.workunit.client.0.smithi090.stdout:0/556: dwrite d1a/d4b/d53/f7f [0,4194304] 0 2024-04-24T08:25:03.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/772: creat d2/d9/d1a/d36/d85/ffa x:0 0 0 2024-04-24T08:25:03.809 INFO:tasks.workunit.client.0.smithi090.stdout:4/473: write d3/d38/d49/f65 [3417398,62552] 0 2024-04-24T08:25:03.810 INFO:tasks.workunit.client.0.smithi090.stdout:5/605: dwrite d8/da/d14/d2f/f66 [0,4194304] 0 2024-04-24T08:25:03.812 INFO:tasks.workunit.client.0.smithi090.stdout:0/557: chown d1a/d4b/d53/d79/d69/ca8 76451696 1 2024-04-24T08:25:03.814 INFO:tasks.workunit.client.1.smithi149.stdout:7/881: rename d9/dcd to d9/dc/d11/d83/da6/d67/d69/d123 0 2024-04-24T08:25:03.816 INFO:tasks.workunit.client.1.smithi149.stdout:4/803: dwrite d2/d10/f27 [0,4194304] 0 2024-04-24T08:25:03.821 INFO:tasks.workunit.client.0.smithi090.stdout:4/474: mknod d3/d18/d5c/c8f 0 2024-04-24T08:25:03.822 INFO:tasks.workunit.client.1.smithi149.stdout:8/795: creat d1/de/d18/d32/f10a x:0 0 0 2024-04-24T08:25:03.823 INFO:tasks.workunit.client.0.smithi090.stdout:5/606: creat d8/d59/dc7/d7a/da1/dae/fc9 x:0 0 0 2024-04-24T08:25:03.824 INFO:tasks.workunit.client.0.smithi090.stdout:8/493: dwrite d2/d62/d6b/d9c/fa1 [0,4194304] 0 2024-04-24T08:25:03.828 INFO:tasks.workunit.client.1.smithi149.stdout:8/796: creat d1/de/d18/d32/d65/d8b/dc9/f10b x:0 0 0 2024-04-24T08:25:03.838 INFO:tasks.workunit.client.0.smithi090.stdout:5/607: creat d8/da/d3a/d96/da2/fca x:0 0 0 2024-04-24T08:25:03.839 INFO:tasks.workunit.client.0.smithi090.stdout:6/578: dwrite d8/d1d/d2f/f83 [8388608,4194304] 0 2024-04-24T08:25:03.840 INFO:tasks.workunit.client.0.smithi090.stdout:8/494: symlink d2/da/d13/d64/d50/lac 0 2024-04-24T08:25:03.841 INFO:tasks.workunit.client.0.smithi090.stdout:5/608: readlink d8/d59/l9b 0 2024-04-24T08:25:03.844 INFO:tasks.workunit.client.0.smithi090.stdout:5/609: chown d8/da/d18/d21/l29 1 1 2024-04-24T08:25:03.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/475: getdents d3/d38/d49 0 2024-04-24T08:25:03.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/495: dread d2/da/d13/d34/f40 [0,4194304] 0 2024-04-24T08:25:03.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/527: sync 2024-04-24T08:25:03.851 INFO:tasks.workunit.client.0.smithi090.stdout:8/496: chown d2/f3 0 1 2024-04-24T08:25:03.851 INFO:tasks.workunit.client.0.smithi090.stdout:7/601: sync 2024-04-24T08:25:03.853 INFO:tasks.workunit.client.0.smithi090.stdout:0/558: dwrite d1a/d4b/f61 [0,4194304] 0 2024-04-24T08:25:03.855 INFO:tasks.workunit.client.0.smithi090.stdout:8/497: chown d2/da/d13/d34/fa0 21827 1 2024-04-24T08:25:03.856 INFO:tasks.workunit.client.0.smithi090.stdout:6/579: creat d8/dc/d19/d8e/d38/faf x:0 0 0 2024-04-24T08:25:03.856 INFO:tasks.workunit.client.1.smithi149.stdout:0/773: dwrite d2/d17/fb9 [0,4194304] 0 2024-04-24T08:25:03.858 INFO:tasks.workunit.client.0.smithi090.stdout:0/559: chown d1a/d4b/d53/d79/d3c/l4f 0 1 2024-04-24T08:25:03.860 INFO:tasks.workunit.client.0.smithi090.stdout:9/528: creat d0/d19/fa0 x:0 0 0 2024-04-24T08:25:03.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/498: mkdir d2/da/d13/d64/d24/dad 0 2024-04-24T08:25:03.862 INFO:tasks.workunit.client.1.smithi149.stdout:8/797: write d1/de/d18/f45 [4267973,8692] 0 2024-04-24T08:25:03.863 INFO:tasks.workunit.client.0.smithi090.stdout:4/476: symlink d3/d8e/l90 0 2024-04-24T08:25:03.863 INFO:tasks.workunit.client.0.smithi090.stdout:9/529: readlink d0/dc/d24/d7c/d4d/d32/d9f/l8f 0 2024-04-24T08:25:03.865 INFO:tasks.workunit.client.0.smithi090.stdout:6/580: mknod d8/d1d/d1f/d3e/d44/cb0 0 2024-04-24T08:25:03.865 INFO:tasks.workunit.client.0.smithi090.stdout:7/602: creat db/dbd/fd0 x:0 0 0 2024-04-24T08:25:03.867 INFO:tasks.workunit.client.0.smithi090.stdout:4/477: write d3/d18/d5c/f74 [159936,60372] 0 2024-04-24T08:25:03.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/581: chown d8/d1d/d1f/d3e/d44/d90/fa9 410879631 1 2024-04-24T08:25:03.871 INFO:tasks.workunit.client.0.smithi090.stdout:6/582: chown d8/dc/c93 25937 1 2024-04-24T08:25:03.871 INFO:tasks.workunit.client.0.smithi090.stdout:7/603: symlink db/d17/d39/d59/ld1 0 2024-04-24T08:25:03.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/560: rename d1a/d4b/d53/d57/f59 to d1a/d4b/d53/d79/fb2 0 2024-04-24T08:25:03.874 INFO:tasks.workunit.client.0.smithi090.stdout:6/583: chown d8/d1d/d1f/d3e/f6a 188149 1 2024-04-24T08:25:03.875 INFO:tasks.workunit.client.1.smithi149.stdout:7/882: getdents d9 0 2024-04-24T08:25:03.876 INFO:tasks.workunit.client.1.smithi149.stdout:0/774: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/ffb x:0 0 0 2024-04-24T08:25:03.876 INFO:tasks.workunit.client.1.smithi149.stdout:4/804: dwrite d2/d18/d1e/d3c/f84 [0,4194304] 0 2024-04-24T08:25:03.876 INFO:tasks.workunit.client.0.smithi090.stdout:0/561: fdatasync d1a/d4b/d53/d79/fa9 0 2024-04-24T08:25:03.877 INFO:tasks.workunit.client.0.smithi090.stdout:7/604: write db/d17/d29/d32/d5a/d75/f8a [725463,66029] 0 2024-04-24T08:25:03.881 INFO:tasks.workunit.client.0.smithi090.stdout:7/605: readlink db/d17/d29/d32/d5a/d75/l1e 0 2024-04-24T08:25:03.889 INFO:tasks.workunit.client.1.smithi149.stdout:9/876: sync 2024-04-24T08:25:03.892 INFO:tasks.workunit.client.0.smithi090.stdout:5/610: dread d8/da/d18/d21/d4c/f9a [0,4194304] 0 2024-04-24T08:25:03.905 INFO:tasks.workunit.client.1.smithi149.stdout:9/877: mkdir d1/d2/d33/d60/d10b/d141 0 2024-04-24T08:25:03.908 INFO:tasks.workunit.client.0.smithi090.stdout:4/478: write d3/d18/f5b [384249,40317] 0 2024-04-24T08:25:03.908 INFO:tasks.workunit.client.0.smithi090.stdout:0/562: symlink d1a/d4b/d53/d79/d35/d68/lb3 0 2024-04-24T08:25:03.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/479: chown d3/c21 1836 1 2024-04-24T08:25:03.910 INFO:tasks.workunit.client.1.smithi149.stdout:8/798: getdents d1/d1f/d31 0 2024-04-24T08:25:03.910 INFO:tasks.workunit.client.0.smithi090.stdout:5/611: dwrite d8/da/d14/d2f/d44/f49 [0,4194304] 0 2024-04-24T08:25:03.923 INFO:tasks.workunit.client.1.smithi149.stdout:0/775: write d2/d9/df/d2f/d6c/dc1/d4e/f6d [2930126,47269] 0 2024-04-24T08:25:03.926 INFO:tasks.workunit.client.0.smithi090.stdout:8/499: rename d2/da/d13/d34/f4d to d2/da/d13/fae 0 2024-04-24T08:25:03.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/584: dwrite d8/d1d/d1f/d3e/f6a [0,4194304] 0 2024-04-24T08:25:03.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/606: mknod db/d17/d29/cd2 0 2024-04-24T08:25:03.933 INFO:tasks.workunit.client.1.smithi149.stdout:7/883: dwrite d9/dc/d7b/fc1 [0,4194304] 0 2024-04-24T08:25:03.934 INFO:tasks.workunit.client.0.smithi090.stdout:7/607: read - db/d17/d39/d59/fc2 zero size 2024-04-24T08:25:03.936 INFO:tasks.workunit.client.1.smithi149.stdout:7/884: dread - d9/dc/d11/d83/da6/ff6 zero size 2024-04-24T08:25:03.938 INFO:tasks.workunit.client.1.smithi149.stdout:4/805: dwrite d2/d24/d2e/f5b [0,4194304] 0 2024-04-24T08:25:03.939 INFO:tasks.workunit.client.0.smithi090.stdout:7/608: dread - db/d17/d29/d32/d5a/d75/fc4 zero size 2024-04-24T08:25:03.941 INFO:tasks.workunit.client.1.smithi149.stdout:8/799: creat d1/de/d18/d32/d65/d8b/d7d/dd1/f10c x:0 0 0 2024-04-24T08:25:03.943 INFO:tasks.workunit.client.1.smithi149.stdout:8/800: chown d1/d1f/d5e/d92 1333 1 2024-04-24T08:25:03.944 INFO:tasks.workunit.client.0.smithi090.stdout:8/500: dwrite d2/da/d13/f84 [0,4194304] 0 2024-04-24T08:25:03.947 INFO:tasks.workunit.client.0.smithi090.stdout:8/501: write d2/da/d13/d34/fa0 [209999,66094] 0 2024-04-24T08:25:03.952 INFO:tasks.workunit.client.1.smithi149.stdout:0/776: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/ffc x:0 0 0 2024-04-24T08:25:03.952 INFO:tasks.workunit.client.0.smithi090.stdout:4/480: fdatasync d3/d18/d1d/f27 0 2024-04-24T08:25:03.954 INFO:tasks.workunit.client.1.smithi149.stdout:0/777: fsync d2/f7f 0 2024-04-24T08:25:03.956 INFO:tasks.workunit.client.0.smithi090.stdout:1/566: sync 2024-04-24T08:25:03.956 INFO:tasks.workunit.client.0.smithi090.stdout:2/607: sync 2024-04-24T08:25:03.956 INFO:tasks.workunit.client.0.smithi090.stdout:3/505: sync 2024-04-24T08:25:03.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/608: readlink d0/d7/l61 0 2024-04-24T08:25:03.961 INFO:tasks.workunit.client.1.smithi149.stdout:8/801: rename d1/de/d18/d32/d65/d8b/l102 to d1/de/d18/d32/d65/d8b/d7d/ddc/l10d 0 2024-04-24T08:25:03.965 INFO:tasks.workunit.client.0.smithi090.stdout:5/612: dread d8/d25/f8f [0,4194304] 0 2024-04-24T08:25:03.965 INFO:tasks.workunit.client.1.smithi149.stdout:4/806: mkdir d2/d24/d7f/d101 0 2024-04-24T08:25:03.966 INFO:tasks.workunit.client.0.smithi090.stdout:8/502: chown d2/l5 3865369 1 2024-04-24T08:25:03.972 INFO:tasks.workunit.client.1.smithi149.stdout:8/802: mkdir d1/de/d18/d43/d10e 0 2024-04-24T08:25:03.978 INFO:tasks.workunit.client.1.smithi149.stdout:4/807: fdatasync d2/d10/d1a/f3e 0 2024-04-24T08:25:03.984 INFO:tasks.workunit.client.1.smithi149.stdout:9/878: write d1/d2/d15/d1c/dca/fe2 [72085,51083] 0 2024-04-24T08:25:03.988 INFO:tasks.workunit.client.1.smithi149.stdout:4/808: creat d2/d18/da4/f102 x:0 0 0 2024-04-24T08:25:03.989 INFO:tasks.workunit.client.0.smithi090.stdout:8/503: rename d2/da/d13/d64/d50/f6c to d2/d62/d83/faf 0 2024-04-24T08:25:03.993 INFO:tasks.workunit.client.1.smithi149.stdout:9/879: read d1/da/f3e [3528292,23724] 0 2024-04-24T08:25:03.997 INFO:tasks.workunit.client.0.smithi090.stdout:3/506: mknod d2/df/d1a/d37/d53/d60/d9b/cad 0 2024-04-24T08:25:03.999 INFO:tasks.workunit.client.0.smithi090.stdout:0/563: dread d1a/f9a [0,4194304] 0 2024-04-24T08:25:04.005 INFO:tasks.workunit.client.0.smithi090.stdout:8/504: unlink d2/l8 0 2024-04-24T08:25:04.011 INFO:tasks.workunit.client.0.smithi090.stdout:3/507: rename d2/d25/f50 to d2/d15/d36/d6f/d92/fae 0 2024-04-24T08:25:04.013 INFO:tasks.workunit.client.0.smithi090.stdout:0/564: mkdir d1a/d4b/d53/d79/d58/db4 0 2024-04-24T08:25:04.014 INFO:tasks.workunit.client.1.smithi149.stdout:4/809: link d2/d24/f39 d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/f103 0 2024-04-24T08:25:04.018 INFO:tasks.workunit.client.0.smithi090.stdout:4/481: getdents d3 0 2024-04-24T08:25:04.018 INFO:tasks.workunit.client.0.smithi090.stdout:0/565: truncate d1a/d4b/d53/d79/d3c/fad 788698 0 2024-04-24T08:25:04.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/505: mkdir d2/da/db0 0 2024-04-24T08:25:04.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/609: write db/d17/d29/f30 [3815624,35873] 0 2024-04-24T08:25:04.021 INFO:tasks.workunit.client.1.smithi149.stdout:4/810: fdatasync d2/d10/d91/fcb 0 2024-04-24T08:25:04.025 INFO:tasks.workunit.client.1.smithi149.stdout:0/778: dwrite d2/d9/df/d48/fb7 [0,4194304] 0 2024-04-24T08:25:04.025 INFO:tasks.workunit.client.0.smithi090.stdout:3/508: symlink d2/df/d1a/d37/d59/d87/laf 0 2024-04-24T08:25:04.027 INFO:tasks.workunit.client.0.smithi090.stdout:6/585: dwrite d8/dc/f72 [0,4194304] 0 2024-04-24T08:25:04.027 INFO:tasks.workunit.client.0.smithi090.stdout:8/506: truncate d2/da/d13/d34/fa0 631337 0 2024-04-24T08:25:04.030 INFO:tasks.workunit.client.1.smithi149.stdout:4/811: unlink d2/d24/d2e/d3f/f4d 0 2024-04-24T08:25:04.030 INFO:tasks.workunit.client.1.smithi149.stdout:8/803: write d1/d1f/d31/f67 [1095174,58617] 0 2024-04-24T08:25:04.035 INFO:tasks.workunit.client.0.smithi090.stdout:9/530: sync 2024-04-24T08:25:04.036 INFO:tasks.workunit.client.1.smithi149.stdout:0/779: creat d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/ffd x:0 0 0 2024-04-24T08:25:04.040 INFO:tasks.workunit.client.1.smithi149.stdout:0/780: dread d2/d9/df/d40/f70 [0,4194304] 0 2024-04-24T08:25:04.042 INFO:tasks.workunit.client.1.smithi149.stdout:8/804: dread - d1/f61 zero size 2024-04-24T08:25:04.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/509: mkdir d2/df/d1a/d37/d59/d87/db0 0 2024-04-24T08:25:04.043 INFO:tasks.workunit.client.1.smithi149.stdout:4/812: dread - d2/d24/d2e/d3f/fcc zero size 2024-04-24T08:25:04.043 INFO:tasks.workunit.client.0.smithi090.stdout:7/610: fdatasync db/d17/d29/f2f 0 2024-04-24T08:25:04.046 INFO:tasks.workunit.client.0.smithi090.stdout:6/586: chown d8/d1d/d1f/d3e/d4f/c70 9284 1 2024-04-24T08:25:04.047 INFO:tasks.workunit.client.1.smithi149.stdout:8/805: readlink d1/de/d18/l60 0 2024-04-24T08:25:04.052 INFO:tasks.workunit.client.0.smithi090.stdout:8/507: creat d2/da/d13/d60/fb1 x:0 0 0 2024-04-24T08:25:04.053 INFO:tasks.workunit.client.0.smithi090.stdout:4/482: dwrite d3/d38/f85 [0,4194304] 0 2024-04-24T08:25:04.055 INFO:tasks.workunit.client.1.smithi149.stdout:9/880: dwrite d1/de/f22 [0,4194304] 0 2024-04-24T08:25:04.055 INFO:tasks.workunit.client.0.smithi090.stdout:0/566: rename d1a/d4b/d53/d79/d82/fa6 to d1a/d4b/d53/d79/d58/db4/fb5 0 2024-04-24T08:25:04.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/609: dwrite d0/d14/d4d/d50/d96/f98 [0,4194304] 0 2024-04-24T08:25:04.063 INFO:tasks.workunit.client.0.smithi090.stdout:5/613: dwrite d8/da/d18/f2b [0,4194304] 0 2024-04-24T08:25:04.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/510: fsync d2/df/f3c 0 2024-04-24T08:25:04.071 INFO:tasks.workunit.client.1.smithi149.stdout:4/813: rename d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/lc3 to d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/l104 0 2024-04-24T08:25:04.073 INFO:tasks.workunit.client.1.smithi149.stdout:4/814: dread - d2/d18/ff4 zero size 2024-04-24T08:25:04.074 INFO:tasks.workunit.client.0.smithi090.stdout:1/567: dwrite d3/d68/f8f [0,4194304] 0 2024-04-24T08:25:04.075 INFO:tasks.workunit.client.0.smithi090.stdout:0/567: chown d1a/d4b/d53/d79/d35/d68/lb3 2141 1 2024-04-24T08:25:04.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/587: write d8/d1d/d1f/d3e/d4f/d7e/f80 [4451576,21310] 0 2024-04-24T08:25:04.076 INFO:tasks.workunit.client.1.smithi149.stdout:1/916: sync 2024-04-24T08:25:04.082 INFO:tasks.workunit.client.0.smithi090.stdout:3/511: dread - d2/d25/d33/f80 zero size 2024-04-24T08:25:04.082 INFO:tasks.workunit.client.0.smithi090.stdout:0/568: chown d1a/d4b/d53/d79/d35 1784922626 1 2024-04-24T08:25:04.082 INFO:tasks.workunit.client.0.smithi090.stdout:6/588: write d8/d1d/d1f/fac [269258,104620] 0 2024-04-24T08:25:04.085 INFO:tasks.workunit.client.0.smithi090.stdout:7/611: mknod db/d17/d39/d59/cd3 0 2024-04-24T08:25:04.087 INFO:tasks.workunit.client.1.smithi149.stdout:0/781: write d2/d9/df/d56/f5f [1323600,71203] 0 2024-04-24T08:25:04.099 INFO:tasks.workunit.client.1.smithi149.stdout:9/881: truncate d1/de/fdb 1010169 0 2024-04-24T08:25:04.100 INFO:tasks.workunit.client.0.smithi090.stdout:5/614: unlink d8/da/d3a/d96/da2/fca 0 2024-04-24T08:25:04.102 INFO:tasks.workunit.client.0.smithi090.stdout:8/508: mkdir d2/da/d13/d64/db2 0 2024-04-24T08:25:04.102 INFO:tasks.workunit.client.0.smithi090.stdout:1/568: symlink d3/d68/d7f/lcb 0 2024-04-24T08:25:04.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/483: mkdir d3/df/d69/d8b/d91 0 2024-04-24T08:25:04.107 INFO:tasks.workunit.client.0.smithi090.stdout:1/569: chown d3/d5/d3e 3 1 2024-04-24T08:25:04.108 INFO:tasks.workunit.client.1.smithi149.stdout:9/882: creat d1/da/d13/f142 x:0 0 0 2024-04-24T08:25:04.109 INFO:tasks.workunit.client.0.smithi090.stdout:7/612: mkdir db/d17/d29/d32/dd4 0 2024-04-24T08:25:04.111 INFO:tasks.workunit.client.0.smithi090.stdout:4/484: readlink d3/d38/d49/l67 0 2024-04-24T08:25:04.111 INFO:tasks.workunit.client.1.smithi149.stdout:4/815: write d2/d10/d91/fcb [3670829,50242] 0 2024-04-24T08:25:04.113 INFO:tasks.workunit.client.0.smithi090.stdout:1/570: dread - d3/d1d/fc9 zero size 2024-04-24T08:25:04.113 INFO:tasks.workunit.client.1.smithi149.stdout:9/883: unlink d1/l38 0 2024-04-24T08:25:04.114 INFO:tasks.workunit.client.0.smithi090.stdout:3/512: dwrite d2/d15/d36/d6f/d92/fa9 [0,4194304] 0 2024-04-24T08:25:04.116 INFO:tasks.workunit.client.1.smithi149.stdout:9/884: chown d1/d2/f97 326755260 1 2024-04-24T08:25:04.116 INFO:tasks.workunit.client.1.smithi149.stdout:0/782: dwrite d2/d9/d1a/d36/d85/ffa [0,4194304] 0 2024-04-24T08:25:04.116 INFO:tasks.workunit.client.1.smithi149.stdout:1/917: link d0/d5/df/fca d0/ddf/d53/d61/f126 0 2024-04-24T08:25:04.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/816: read d2/d18/db9/dd8/f42 [2542630,48698] 0 2024-04-24T08:25:04.121 INFO:tasks.workunit.client.0.smithi090.stdout:0/569: truncate d1a/f52 1111162 0 2024-04-24T08:25:04.121 INFO:tasks.workunit.client.0.smithi090.stdout:9/531: dread d0/fb [4194304,4194304] 0 2024-04-24T08:25:04.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/615: rename d8/da/d18 to d8/da/d3a/d96/dcb 0 2024-04-24T08:25:04.124 INFO:tasks.workunit.client.1.smithi149.stdout:1/918: mkdir d0/d1b/d5d/dd6/ddb/d127 0 2024-04-24T08:25:04.127 INFO:tasks.workunit.client.0.smithi090.stdout:8/509: mkdir d2/da/d13/d64/d24/db3 0 2024-04-24T08:25:04.130 INFO:tasks.workunit.client.0.smithi090.stdout:8/510: truncate d2/d62/f7a 4905965 0 2024-04-24T08:25:04.130 INFO:tasks.workunit.client.0.smithi090.stdout:9/532: write d0/dc/d24/d2d/f47 [8481961,64350] 0 2024-04-24T08:25:04.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/513: rmdir d2/d15/d36/d6f 39 2024-04-24T08:25:04.134 INFO:tasks.workunit.client.1.smithi149.stdout:4/817: dread d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf [0,4194304] 0 2024-04-24T08:25:04.139 INFO:tasks.workunit.client.0.smithi090.stdout:1/571: getdents d3/d5/d3e 0 2024-04-24T08:25:04.142 INFO:tasks.workunit.client.1.smithi149.stdout:4/818: symlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/def/l105 0 2024-04-24T08:25:04.143 INFO:tasks.workunit.client.0.smithi090.stdout:2/610: dread d0/d7/d6f/f29 [0,4194304] 0 2024-04-24T08:25:04.143 INFO:tasks.workunit.client.1.smithi149.stdout:0/783: dwrite d2/d9/df/d2f/d6c/d72/d91/f95 [0,4194304] 0 2024-04-24T08:25:04.144 INFO:tasks.workunit.client.0.smithi090.stdout:1/572: chown d3/d25/d30/d9d/da2/fb5 16597733 1 2024-04-24T08:25:04.144 INFO:tasks.workunit.client.0.smithi090.stdout:6/589: dread d8/dc/fa1 [0,4194304] 0 2024-04-24T08:25:04.145 INFO:tasks.workunit.client.1.smithi149.stdout:0/784: readlink d2/d3c/l6e 0 2024-04-24T08:25:04.146 INFO:tasks.workunit.client.0.smithi090.stdout:7/613: rename db/d17/d29/l33 to db/d17/d29/ld5 0 2024-04-24T08:25:04.149 INFO:tasks.workunit.client.1.smithi149.stdout:4/819: rmdir d2/d18/d1e/d3c 39 2024-04-24T08:25:04.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/611: stat d0/d14/d36/l42 0 2024-04-24T08:25:04.149 INFO:tasks.workunit.client.0.smithi090.stdout:5/616: dread d8/d59/dc7/d7b/f80 [0,4194304] 0 2024-04-24T08:25:04.150 INFO:tasks.workunit.client.1.smithi149.stdout:0/785: dread - d2/d9/d1a/d36/dcc/ff6 zero size 2024-04-24T08:25:04.157 INFO:tasks.workunit.client.0.smithi090.stdout:3/514: mknod d2/df/d1a/d37/d59/d98/cb1 0 2024-04-24T08:25:04.157 INFO:tasks.workunit.client.0.smithi090.stdout:0/570: dwrite ff [8388608,4194304] 0 2024-04-24T08:25:04.161 INFO:tasks.workunit.client.0.smithi090.stdout:8/511: mkdir d2/da/d13/d64/d24/db3/db4 0 2024-04-24T08:25:04.161 INFO:tasks.workunit.client.0.smithi090.stdout:7/614: mknod db/dbd/cd6 0 2024-04-24T08:25:04.165 INFO:tasks.workunit.client.0.smithi090.stdout:6/590: rename d8/dc/d19/d8e to d8/dc/d19/d8e/db1 22 2024-04-24T08:25:04.167 INFO:tasks.workunit.client.0.smithi090.stdout:1/573: mkdir d3/d5/da/d97/d9c/dc4/dcc 0 2024-04-24T08:25:04.172 INFO:tasks.workunit.client.0.smithi090.stdout:8/512: getdents d2/d97 0 2024-04-24T08:25:04.176 INFO:tasks.workunit.client.0.smithi090.stdout:6/591: creat d8/dc/d19/d8e/d38/fb2 x:0 0 0 2024-04-24T08:25:04.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/615: fdatasync db/d17/f28 0 2024-04-24T08:25:04.178 INFO:tasks.workunit.client.0.smithi090.stdout:1/574: fsync d3/d1d/d54/f74 0 2024-04-24T08:25:04.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/515: link d2/d25/d33/f44 d2/d25/d2c/fb2 0 2024-04-24T08:25:04.182 INFO:tasks.workunit.client.0.smithi090.stdout:9/533: dwrite d0/dc/f1f [0,4194304] 0 2024-04-24T08:25:04.182 INFO:tasks.workunit.client.0.smithi090.stdout:0/571: dread d1a/d4b/f6f [0,4194304] 0 2024-04-24T08:25:04.189 INFO:tasks.workunit.client.0.smithi090.stdout:9/534: stat d0/dc/d24/d7c/d4d/d32/f95 0 2024-04-24T08:25:04.190 INFO:tasks.workunit.client.0.smithi090.stdout:7/616: write db/d17/d29/d32/d5a/d75/fc4 [818196,55988] 0 2024-04-24T08:25:04.190 INFO:tasks.workunit.client.0.smithi090.stdout:1/575: mknod d3/d5/d3e/da3/ccd 0 2024-04-24T08:25:04.193 INFO:tasks.workunit.client.0.smithi090.stdout:5/617: dread d8/da/d3a/d96/dcb/f1e [0,4194304] 0 2024-04-24T08:25:04.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/820: truncate d2/d18/d1e/d3c/f43 3294316 0 2024-04-24T08:25:04.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/576: symlink d3/d5/d58/lce 0 2024-04-24T08:25:04.196 INFO:tasks.workunit.client.1.smithi149.stdout:9/885: dwrite d1/d2/d33/d5d/d83/dbf/dc4/ff5 [0,4194304] 0 2024-04-24T08:25:04.200 INFO:tasks.workunit.client.0.smithi090.stdout:0/572: creat d1a/d4b/d53/d79/fb6 x:0 0 0 2024-04-24T08:25:04.204 INFO:tasks.workunit.client.0.smithi090.stdout:4/485: dread d3/d38/d49/f65 [0,4194304] 0 2024-04-24T08:25:04.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/516: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:25:04.211 INFO:tasks.workunit.client.0.smithi090.stdout:9/535: creat d0/d69/d7d/d5b/d96/fa1 x:0 0 0 2024-04-24T08:25:04.213 INFO:tasks.workunit.client.0.smithi090.stdout:4/486: write d3/d38/d6b/f64 [883240,20731] 0 2024-04-24T08:25:04.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/617: truncate db/d17/d29/d38/d3e/fb7 319260 0 2024-04-24T08:25:04.215 INFO:tasks.workunit.client.1.smithi149.stdout:9/886: rename d1/da/dd/f69 to d1/d2/d15/f143 0 2024-04-24T08:25:04.220 INFO:tasks.workunit.client.0.smithi090.stdout:6/592: dwrite d8/dc/d19/d8e/d81/d96/d97/fae [0,4194304] 0 2024-04-24T08:25:04.221 INFO:tasks.workunit.client.0.smithi090.stdout:4/487: readlink d3/l45 0 2024-04-24T08:25:04.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/887: symlink d1/d2/d33/dea/d126/l144 0 2024-04-24T08:25:04.224 INFO:tasks.workunit.client.0.smithi090.stdout:3/517: write d2/df/d57/d4f/f72 [475315,9709] 0 2024-04-24T08:25:04.225 INFO:tasks.workunit.client.0.smithi090.stdout:8/513: dwrite d2/da/d13/f84 [0,4194304] 0 2024-04-24T08:25:04.226 INFO:tasks.workunit.client.0.smithi090.stdout:0/573: rmdir d1a/d4b/d53/d79/d69/d78 39 2024-04-24T08:25:04.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/612: dread d0/d7/f33 [0,4194304] 0 2024-04-24T08:25:04.230 INFO:tasks.workunit.client.0.smithi090.stdout:7/618: creat db/d17/d29/d32/fd7 x:0 0 0 2024-04-24T08:25:04.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/514: stat d2/da/d13/d34/f3c 0 2024-04-24T08:25:04.236 INFO:tasks.workunit.client.0.smithi090.stdout:9/536: mknod d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/ca2 0 2024-04-24T08:25:04.238 INFO:tasks.workunit.client.0.smithi090.stdout:2/613: truncate d0/d7/d28/fbf 54251 0 2024-04-24T08:25:04.239 INFO:tasks.workunit.client.0.smithi090.stdout:3/518: symlink d2/lb3 0 2024-04-24T08:25:04.243 INFO:tasks.workunit.client.0.smithi090.stdout:0/574: fsync d1a/d4b/d53/d79/d3c/f5b 0 2024-04-24T08:25:04.243 INFO:tasks.workunit.client.0.smithi090.stdout:8/515: chown d2/d62/d71/l95 7636558 1 2024-04-24T08:25:04.243 INFO:tasks.workunit.client.1.smithi149.stdout:0/786: dwrite d2/d3c/f61 [0,4194304] 0 2024-04-24T08:25:04.245 INFO:tasks.workunit.client.1.smithi149.stdout:9/888: symlink d1/d2/d33/dea/d133/dad/db7/d12d/l145 0 2024-04-24T08:25:04.245 INFO:tasks.workunit.client.0.smithi090.stdout:1/577: dwrite d3/d5/d58/fb3 [0,4194304] 0 2024-04-24T08:25:04.253 INFO:tasks.workunit.client.0.smithi090.stdout:5/618: dread d8/d59/dc7/d7a/d50/f79 [0,4194304] 0 2024-04-24T08:25:04.257 INFO:tasks.workunit.client.0.smithi090.stdout:6/593: creat d8/d1d/d2f/d85/fb3 x:0 0 0 2024-04-24T08:25:04.258 INFO:tasks.workunit.client.1.smithi149.stdout:6/737: sync 2024-04-24T08:25:04.258 INFO:tasks.workunit.client.1.smithi149.stdout:2/745: sync 2024-04-24T08:25:04.258 INFO:tasks.workunit.client.1.smithi149.stdout:3/933: sync 2024-04-24T08:25:04.270 INFO:tasks.workunit.client.0.smithi090.stdout:3/519: write d2/df/d57/d4f/f86 [5027201,117238] 0 2024-04-24T08:25:04.271 INFO:tasks.workunit.client.1.smithi149.stdout:4/821: dwrite d2/d18/db9/dd8/fa0 [0,4194304] 0 2024-04-24T08:25:04.280 INFO:tasks.workunit.client.1.smithi149.stdout:9/889: creat d1/d2/d33/d60/d10b/d141/f146 x:0 0 0 2024-04-24T08:25:04.280 INFO:tasks.workunit.client.0.smithi090.stdout:0/575: mknod d1a/d4b/d53/d79/d35/d8e/cb7 0 2024-04-24T08:25:04.287 INFO:tasks.workunit.client.1.smithi149.stdout:3/934: rename d0/d5/da/d5a/d63/d6a/df5 to d0/d5/da/d138 0 2024-04-24T08:25:04.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/822: creat d2/d10/d91/d9f/f106 x:0 0 0 2024-04-24T08:25:04.296 INFO:tasks.workunit.client.0.smithi090.stdout:2/614: creat d0/d14/d4d/db4/dcd/fd1 x:0 0 0 2024-04-24T08:25:04.298 INFO:tasks.workunit.client.1.smithi149.stdout:6/738: dread d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 [0,4194304] 0 2024-04-24T08:25:04.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/890: truncate d1/d2/d34/f35 747188 0 2024-04-24T08:25:04.300 INFO:tasks.workunit.client.0.smithi090.stdout:5/619: dwrite d8/d99/f9d [0,4194304] 0 2024-04-24T08:25:04.306 INFO:tasks.workunit.client.0.smithi090.stdout:0/576: fdatasync d1a/f2c 0 2024-04-24T08:25:04.306 INFO:tasks.workunit.client.0.smithi090.stdout:9/537: dwrite d0/dc/d24/d7c/f57 [0,4194304] 0 2024-04-24T08:25:04.306 INFO:tasks.workunit.client.1.smithi149.stdout:2/746: link d0/d6/db/d39/d3d/d58/d98/faf d0/d6/db/d28/d7f/d9b/fe2 0 2024-04-24T08:25:04.307 INFO:tasks.workunit.client.0.smithi090.stdout:8/516: write d2/d62/d6b/f7d [532888,97767] 0 2024-04-24T08:25:04.308 INFO:tasks.workunit.client.0.smithi090.stdout:1/578: creat d3/d25/d30/dc5/fcf x:0 0 0 2024-04-24T08:25:04.310 INFO:tasks.workunit.client.1.smithi149.stdout:0/787: dwrite d2/d9/d1a/f6b [0,4194304] 0 2024-04-24T08:25:04.315 INFO:tasks.workunit.client.1.smithi149.stdout:2/747: truncate d0/d6/db/d39/d3d/d58/d98/faf 1074722 0 2024-04-24T08:25:04.316 INFO:tasks.workunit.client.0.smithi090.stdout:6/594: write d8/dc/f53 [4448262,42738] 0 2024-04-24T08:25:04.321 INFO:tasks.workunit.client.1.smithi149.stdout:2/748: readlink d0/d6/db/d28/db0/ld0 0 2024-04-24T08:25:04.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/517: dwrite d2/da/d13/d64/f86 [0,4194304] 0 2024-04-24T08:25:04.331 INFO:tasks.workunit.client.0.smithi090.stdout:2/615: truncate d0/d7/f32 4709352 0 2024-04-24T08:25:04.332 INFO:tasks.workunit.client.1.smithi149.stdout:0/788: mknod d2/d9/df/d48/dba/cfe 0 2024-04-24T08:25:04.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/616: write d0/d14/d36/fca [528734,100491] 0 2024-04-24T08:25:04.335 INFO:tasks.workunit.client.1.smithi149.stdout:4/823: write d2/d10/d1a/fda [230129,80388] 0 2024-04-24T08:25:04.339 INFO:tasks.workunit.client.1.smithi149.stdout:4/824: write d2/d18/db9/dd8/f79 [2972420,88218] 0 2024-04-24T08:25:04.341 INFO:tasks.workunit.client.0.smithi090.stdout:8/518: dwrite d2/da/d13/d34/fa0 [0,4194304] 0 2024-04-24T08:25:04.341 INFO:tasks.workunit.client.1.smithi149.stdout:6/739: unlink d0/d1a/l41 0 2024-04-24T08:25:04.343 INFO:tasks.workunit.client.0.smithi090.stdout:8/519: chown d2/da/d13/d64/f3f 107613 1 2024-04-24T08:25:04.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/749: fsync d0/d6/db/d39/d3d/d44/d75/f79 0 2024-04-24T08:25:04.350 INFO:tasks.workunit.client.1.smithi149.stdout:1/919: sync 2024-04-24T08:25:04.350 INFO:tasks.workunit.client.1.smithi149.stdout:8/806: sync 2024-04-24T08:25:04.351 INFO:tasks.workunit.client.1.smithi149.stdout:7/885: sync 2024-04-24T08:25:04.353 INFO:tasks.workunit.client.1.smithi149.stdout:2/750: truncate d0/d6/db/d27/d81/fc7 797185 0 2024-04-24T08:25:04.356 INFO:tasks.workunit.client.1.smithi149.stdout:2/751: dread - d0/d6/db/d39/d3d/d44/d57/f9d zero size 2024-04-24T08:25:04.368 INFO:tasks.workunit.client.1.smithi149.stdout:8/807: mknod d1/de/d18/d32/c10f 0 2024-04-24T08:25:04.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/752: dread - d0/d6/db/d28/db0/fc0 zero size 2024-04-24T08:25:04.375 INFO:tasks.workunit.client.1.smithi149.stdout:7/886: rename d9/dc/d7b/cdf to d9/d43/d95/c124 0 2024-04-24T08:25:04.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/825: creat d2/f107 x:0 0 0 2024-04-24T08:25:04.377 INFO:tasks.workunit.client.1.smithi149.stdout:6/740: creat d0/d1a/d28/ffe x:0 0 0 2024-04-24T08:25:04.380 INFO:tasks.workunit.client.1.smithi149.stdout:8/808: symlink d1/de/d18/d32/d65/d8b/dc9/l110 0 2024-04-24T08:25:04.386 INFO:tasks.workunit.client.1.smithi149.stdout:8/809: dread - d1/d1f/d5e/fa0 zero size 2024-04-24T08:25:04.387 INFO:tasks.workunit.client.1.smithi149.stdout:7/887: mknod d9/dc/d2e/d3c/c125 0 2024-04-24T08:25:04.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/810: chown d1/de/f38 814930 1 2024-04-24T08:25:04.392 INFO:tasks.workunit.client.0.smithi090.stdout:5/620: rmdir d8/da/d3a/d96/dcb/d21/d4c 39 2024-04-24T08:25:04.394 INFO:tasks.workunit.client.1.smithi149.stdout:7/888: readlink d9/dc/d11/d26/l11f 0 2024-04-24T08:25:04.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/621: truncate d8/da/d3a/f43 4885371 0 2024-04-24T08:25:04.396 INFO:tasks.workunit.client.0.smithi090.stdout:9/538: creat d0/d19/fa3 x:0 0 0 2024-04-24T08:25:04.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/539: write d0/d19/fa3 [597117,51764] 0 2024-04-24T08:25:04.400 INFO:tasks.workunit.client.0.smithi090.stdout:0/577: mkdir d1a/d4b/d53/d79/d35/db8 0 2024-04-24T08:25:04.402 INFO:tasks.workunit.client.1.smithi149.stdout:3/935: write d0/d5/d1a/d70/da0/fc4 [2026598,79354] 0 2024-04-24T08:25:04.403 INFO:tasks.workunit.client.0.smithi090.stdout:1/579: creat d3/d5/d3e/da3/fd0 x:0 0 0 2024-04-24T08:25:04.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/540: write d0/dc/d24/d7c/f36 [713863,76952] 0 2024-04-24T08:25:04.406 INFO:tasks.workunit.client.0.smithi090.stdout:4/488: getdents d3/df/d77 0 2024-04-24T08:25:04.413 INFO:tasks.workunit.client.1.smithi149.stdout:9/891: dwrite d1/d2/d15/d1c/d42/dae/dd5/fde [0,4194304] 0 2024-04-24T08:25:04.414 INFO:tasks.workunit.client.1.smithi149.stdout:6/741: dwrite d0/d1a/d28/d31/d4c/d80/dbd/ff7 [0,4194304] 0 2024-04-24T08:25:04.420 INFO:tasks.workunit.client.0.smithi090.stdout:2/617: symlink d0/d7/d67/d7f/d31/d46/d60/ld2 0 2024-04-24T08:25:04.437 INFO:tasks.workunit.client.1.smithi149.stdout:9/892: dread d1/da/dd/d79/db6/ffa [0,4194304] 0 2024-04-24T08:25:04.439 INFO:tasks.workunit.client.0.smithi090.stdout:4/489: mkdir d3/d24/d92 0 2024-04-24T08:25:04.442 INFO:tasks.workunit.client.0.smithi090.stdout:7/619: sync 2024-04-24T08:25:04.442 INFO:tasks.workunit.client.0.smithi090.stdout:3/520: sync 2024-04-24T08:25:04.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/595: unlink d8/d1d/d1f/d3e/f5c 0 2024-04-24T08:25:04.443 INFO:tasks.workunit.client.1.smithi149.stdout:7/889: write d9/dc/d2e/f6f [2613816,2865] 0 2024-04-24T08:25:04.444 INFO:tasks.workunit.client.0.smithi090.stdout:8/520: symlink d2/da/db0/lb5 0 2024-04-24T08:25:04.447 INFO:tasks.workunit.client.0.smithi090.stdout:5/622: creat d8/da/d14/d2f/d81/da6/fcc x:0 0 0 2024-04-24T08:25:04.449 INFO:tasks.workunit.client.1.smithi149.stdout:2/753: dwrite d0/d6/db/d28/f73 [0,4194304] 0 2024-04-24T08:25:04.450 INFO:tasks.workunit.client.0.smithi090.stdout:4/490: chown d3/d18/d5c/f47 0 1 2024-04-24T08:25:04.451 INFO:tasks.workunit.client.0.smithi090.stdout:0/578: creat d1a/d4b/d53/d79/d69/d78/d9b/fb9 x:0 0 0 2024-04-24T08:25:04.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/618: dwrite d0/d7/d67/d7f/d31/d46/d60/fa4 [0,4194304] 0 2024-04-24T08:25:04.453 INFO:tasks.workunit.client.1.smithi149.stdout:4/826: dwrite d2/d18/db9/dd8/ff3 [0,4194304] 0 2024-04-24T08:25:04.458 INFO:tasks.workunit.client.0.smithi090.stdout:9/541: mkdir d0/da4 0 2024-04-24T08:25:04.460 INFO:tasks.workunit.client.0.smithi090.stdout:3/521: fsync d2/d25/d33/f43 0 2024-04-24T08:25:04.462 INFO:tasks.workunit.client.0.smithi090.stdout:6/596: mknod d8/d1d/d1f/d3e/d7a/cb4 0 2024-04-24T08:25:04.470 INFO:tasks.workunit.client.1.smithi149.stdout:9/893: dread d1/de/f22 [0,4194304] 0 2024-04-24T08:25:04.473 INFO:tasks.workunit.client.1.smithi149.stdout:9/894: chown d1/d2/d33/d5d/c132 235 1 2024-04-24T08:25:04.476 INFO:tasks.workunit.client.0.smithi090.stdout:8/521: mkdir d2/da/d13/d64/db2/db6 0 2024-04-24T08:25:04.476 INFO:tasks.workunit.client.1.smithi149.stdout:8/811: mkdir d1/d1f/d25/d111 0 2024-04-24T08:25:04.481 INFO:tasks.workunit.client.1.smithi149.stdout:8/812: chown d1/de/d18/d32/d65/d8b/d7d/ff8 51 1 2024-04-24T08:25:04.483 INFO:tasks.workunit.client.0.smithi090.stdout:4/491: dwrite d3/df/f68 [0,4194304] 0 2024-04-24T08:25:04.487 INFO:tasks.workunit.client.0.smithi090.stdout:2/619: dread d0/d7/d6f/f29 [0,4194304] 0 2024-04-24T08:25:04.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/580: dread d3/d5/f28 [0,4194304] 0 2024-04-24T08:25:04.493 INFO:tasks.workunit.client.1.smithi149.stdout:6/742: mknod d0/d1a/d28/d31/d4c/d8a/cff 0 2024-04-24T08:25:04.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/522: link d2/d25/d33/l89 d2/df/d57/d4f/lb4 0 2024-04-24T08:25:04.494 INFO:tasks.workunit.client.0.smithi090.stdout:2/620: truncate d0/d7/d67/d7f/d31/d46/d60/d65/fc0 721056 0 2024-04-24T08:25:04.496 INFO:tasks.workunit.client.1.smithi149.stdout:0/789: sync 2024-04-24T08:25:04.502 INFO:tasks.workunit.client.0.smithi090.stdout:1/581: symlink d3/d5/da/d97/d9c/dc4/ld1 0 2024-04-24T08:25:04.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/621: creat d0/d7/d67/d7f/d31/d46/d60/d65/fd3 x:0 0 0 2024-04-24T08:25:04.506 INFO:tasks.workunit.client.1.smithi149.stdout:7/890: mkdir d9/dc/d7b/d126 0 2024-04-24T08:25:04.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/582: read d3/f2e [512477,100967] 0 2024-04-24T08:25:04.510 INFO:tasks.workunit.client.0.smithi090.stdout:2/622: mknod d0/d7/d28/daa/db1/cd4 0 2024-04-24T08:25:04.512 INFO:tasks.workunit.client.1.smithi149.stdout:7/891: dread d9/dc/d2e/d3c/ffa [0,4194304] 0 2024-04-24T08:25:04.513 INFO:tasks.workunit.client.1.smithi149.stdout:9/895: mkdir d1/d2/d33/dea/d133/d147 0 2024-04-24T08:25:04.513 INFO:tasks.workunit.client.0.smithi090.stdout:6/597: dread d8/dc/d19/f34 [0,4194304] 0 2024-04-24T08:25:04.514 INFO:tasks.workunit.client.0.smithi090.stdout:6/598: fsync d8/f3f 0 2024-04-24T08:25:04.517 INFO:tasks.workunit.client.0.smithi090.stdout:8/522: getdents d2/d62/d83 0 2024-04-24T08:25:04.520 INFO:tasks.workunit.client.1.smithi149.stdout:8/813: mknod d1/de/d18/d32/d65/d8b/d7d/c112 0 2024-04-24T08:25:04.522 INFO:tasks.workunit.client.0.smithi090.stdout:1/583: creat d3/d5/da/d97/d9c/dc4/fd2 x:0 0 0 2024-04-24T08:25:04.524 INFO:tasks.workunit.client.0.smithi090.stdout:2/623: truncate d0/d7/f33 3511959 0 2024-04-24T08:25:04.525 INFO:tasks.workunit.client.1.smithi149.stdout:0/790: mknod d2/d9/d1a/d36/d85/cff 0 2024-04-24T08:25:04.526 INFO:tasks.workunit.client.0.smithi090.stdout:8/523: dread d2/da/d13/f84 [0,4194304] 0 2024-04-24T08:25:04.527 INFO:tasks.workunit.client.1.smithi149.stdout:0/791: chown d2/d9/df/d2f/d6c/d72/db0 1038045 1 2024-04-24T08:25:04.531 INFO:tasks.workunit.client.0.smithi090.stdout:5/623: dwrite d8/f9e [0,4194304] 0 2024-04-24T08:25:04.536 INFO:tasks.workunit.client.0.smithi090.stdout:7/620: write db/d54/d83/f8f [2683434,20839] 0 2024-04-24T08:25:04.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/621: chown db/d17/d29/d32/d5a/f8d 20735488 1 2024-04-24T08:25:04.541 INFO:tasks.workunit.client.1.smithi149.stdout:2/754: mkdir d0/d6/db/d28/d7f/d8f/db9/ddf/de3 0 2024-04-24T08:25:04.546 INFO:tasks.workunit.client.1.smithi149.stdout:9/896: rename d1/d2/d15/d1c/d42/dae/f13e to d1/d2/d33/dea/f148 0 2024-04-24T08:25:04.551 INFO:tasks.workunit.client.0.smithi090.stdout:8/524: dread d2/d62/f6e [0,4194304] 0 2024-04-24T08:25:04.553 INFO:tasks.workunit.client.0.smithi090.stdout:8/525: dread - d2/da/d13/d34/f72 zero size 2024-04-24T08:25:04.554 INFO:tasks.workunit.client.0.smithi090.stdout:5/624: rmdir d8/d59/dc7/d7a 39 2024-04-24T08:25:04.558 INFO:tasks.workunit.client.0.smithi090.stdout:2/624: rename d0/d2e/cbd to d0/d14/d17/cd5 0 2024-04-24T08:25:04.558 INFO:tasks.workunit.client.0.smithi090.stdout:6/599: dread d8/dc/d19/d8e/f4b [0,4194304] 0 2024-04-24T08:25:04.561 INFO:tasks.workunit.client.0.smithi090.stdout:7/622: creat db/dab/fd8 x:0 0 0 2024-04-24T08:25:04.562 INFO:tasks.workunit.client.0.smithi090.stdout:2/625: truncate d0/d7/d6f/fc6 34476 0 2024-04-24T08:25:04.563 INFO:tasks.workunit.client.1.smithi149.stdout:4/827: rmdir d2/d18/dd6/de6 0 2024-04-24T08:25:04.563 INFO:tasks.workunit.client.0.smithi090.stdout:1/584: link d3/d1d/f86 d3/d25/d30/d9d/fd3 0 2024-04-24T08:25:04.564 INFO:tasks.workunit.client.1.smithi149.stdout:4/828: chown d2/fc2 13 1 2024-04-24T08:25:04.572 INFO:tasks.workunit.client.1.smithi149.stdout:6/743: link d0/d1a/d28/d31/d4c/d4e/d62/f91 d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/f100 0 2024-04-24T08:25:04.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/526: dread d2/da/d13/d34/f6f [0,4194304] 0 2024-04-24T08:25:04.577 INFO:tasks.workunit.client.1.smithi149.stdout:3/936: rename d0/d5/da/d5a/d63/f130 to d0/d5/da/d19/d1b/d133/dd4/f139 0 2024-04-24T08:25:04.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/937: readlink d0/d5/d53/d34/l6c 0 2024-04-24T08:25:04.579 INFO:tasks.workunit.client.0.smithi090.stdout:1/585: rename d3/d5/da/dc/f27 to d3/d25/fd4 0 2024-04-24T08:25:04.582 INFO:tasks.workunit.client.0.smithi090.stdout:6/600: dread d8/fa [0,4194304] 0 2024-04-24T08:25:04.585 INFO:tasks.workunit.client.1.smithi149.stdout:2/755: mkdir d0/d6/db/de4 0 2024-04-24T08:25:04.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/623: symlink db/d17/d39/ld9 0 2024-04-24T08:25:04.588 INFO:tasks.workunit.client.0.smithi090.stdout:7/624: chown db/d17/d29/d32/f61 208175504 1 2024-04-24T08:25:04.588 INFO:tasks.workunit.client.1.smithi149.stdout:8/814: dwrite d1/d1f/d25/f80 [0,4194304] 0 2024-04-24T08:25:04.588 INFO:tasks.workunit.client.0.smithi090.stdout:8/527: fdatasync d2/da/d13/f52 0 2024-04-24T08:25:04.592 INFO:tasks.workunit.client.1.smithi149.stdout:9/897: mkdir d1/da/dd/d79/db6/d115/d149 0 2024-04-24T08:25:04.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/586: rmdir d3/d1d/d34/d73 39 2024-04-24T08:25:04.597 INFO:tasks.workunit.client.1.smithi149.stdout:0/792: dwrite d2/d9/d69/fa7 [0,4194304] 0 2024-04-24T08:25:04.597 INFO:tasks.workunit.client.1.smithi149.stdout:8/815: write d1/d1f/d25/f42 [5039919,76145] 0 2024-04-24T08:25:04.598 INFO:tasks.workunit.client.1.smithi149.stdout:3/938: rmdir d0/d5/da/d5a/d63 39 2024-04-24T08:25:04.600 INFO:tasks.workunit.client.1.smithi149.stdout:8/816: chown d1/d1f/d25/f34 91320388 1 2024-04-24T08:25:04.603 INFO:tasks.workunit.client.0.smithi090.stdout:5/625: write d8/d59/dc7/d7a/f4e [908315,129931] 0 2024-04-24T08:25:04.604 INFO:tasks.workunit.client.0.smithi090.stdout:2/626: dwrite d0/d7/d67/fc7 [4194304,4194304] 0 2024-04-24T08:25:04.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/829: getdents d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/dbd 0 2024-04-24T08:25:04.611 INFO:tasks.workunit.client.1.smithi149.stdout:6/744: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d101 0 2024-04-24T08:25:04.615 INFO:tasks.workunit.client.1.smithi149.stdout:6/745: chown d0/d1a/d28/d31/d4c/df8/ffd 115 1 2024-04-24T08:25:04.617 INFO:tasks.workunit.client.1.smithi149.stdout:3/939: fsync d0/d5/d53/d34/f95 0 2024-04-24T08:25:04.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/817: rename d1/de/cb9 to d1/d1f/d25/d111/c113 0 2024-04-24T08:25:04.629 INFO:tasks.workunit.client.1.smithi149.stdout:2/756: symlink d0/d6/db/d39/d3d/d44/le5 0 2024-04-24T08:25:04.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/818: read d1/de/d18/f45 [3684721,81969] 0 2024-04-24T08:25:04.630 INFO:tasks.workunit.client.1.smithi149.stdout:3/940: write d0/d5/da/d5a/d9c/f134 [260002,78821] 0 2024-04-24T08:25:04.633 INFO:tasks.workunit.client.1.smithi149.stdout:6/746: truncate d0/f11 3734939 0 2024-04-24T08:25:04.636 INFO:tasks.workunit.client.1.smithi149.stdout:2/757: write d0/d6/db/d28/f73 [542228,71914] 0 2024-04-24T08:25:04.636 INFO:tasks.workunit.client.1.smithi149.stdout:6/747: chown d0/d1a/d28/d31/d4c/d76/la3 2 1 2024-04-24T08:25:04.637 INFO:tasks.workunit.client.1.smithi149.stdout:9/898: link d1/d2/d15/d1c/d42/dae/fd8 d1/d2/d33/dea/d126/d137/f14a 0 2024-04-24T08:25:04.643 INFO:tasks.workunit.client.1.smithi149.stdout:0/793: getdents d2/d9/df/d2f/d6c/d72/d91 0 2024-04-24T08:25:04.646 INFO:tasks.workunit.client.1.smithi149.stdout:2/758: creat d0/d6/db/d39/d78/d7c/fe6 x:0 0 0 2024-04-24T08:25:04.648 INFO:tasks.workunit.client.1.smithi149.stdout:8/819: symlink d1/d1f/l114 0 2024-04-24T08:25:04.653 INFO:tasks.workunit.client.1.smithi149.stdout:2/759: mknod d0/d6/db/d28/d7f/d9b/ce7 0 2024-04-24T08:25:04.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/830: dread d2/d24/d2e/f66 [0,4194304] 0 2024-04-24T08:25:04.654 INFO:tasks.workunit.client.1.smithi149.stdout:6/748: dread d0/d1a/d28/d31/d33/f90 [0,4194304] 0 2024-04-24T08:25:04.655 INFO:tasks.workunit.client.1.smithi149.stdout:4/831: stat d2/d18/d1e/d3c/d48 0 2024-04-24T08:25:04.660 INFO:tasks.workunit.client.1.smithi149.stdout:4/832: truncate d2/d18/da4/f102 10605 0 2024-04-24T08:25:04.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/920: sync 2024-04-24T08:25:04.668 INFO:tasks.workunit.client.0.smithi090.stdout:6/601: mkdir d8/db5 0 2024-04-24T08:25:04.669 INFO:tasks.workunit.client.0.smithi090.stdout:7/625: symlink db/d17/lda 0 2024-04-24T08:25:04.670 INFO:tasks.workunit.client.0.smithi090.stdout:8/528: dwrite d2/da/f35 [0,4194304] 0 2024-04-24T08:25:04.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/627: creat d0/d7/d57/d79/fd6 x:0 0 0 2024-04-24T08:25:04.670 INFO:tasks.workunit.client.1.smithi149.stdout:6/749: getdents d0/d1a/d28/d4a 0 2024-04-24T08:25:04.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/833: symlink d2/l108 0 2024-04-24T08:25:04.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/587: symlink d3/d5/ld5 0 2024-04-24T08:25:04.682 INFO:tasks.workunit.client.1.smithi149.stdout:9/899: dread d1/d2/f2b [4194304,4194304] 0 2024-04-24T08:25:04.692 INFO:tasks.workunit.client.1.smithi149.stdout:1/921: write d0/d1b/d4b/f58 [1448882,46591] 0 2024-04-24T08:25:04.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/529: creat d2/da/d13/d64/d50/fb7 x:0 0 0 2024-04-24T08:25:04.696 INFO:tasks.workunit.client.0.smithi090.stdout:6/602: truncate d8/d1d/d1f/d3e/d44/d90/fa9 990126 0 2024-04-24T08:25:04.696 INFO:tasks.workunit.client.1.smithi149.stdout:3/941: dread d0/d5/da/f97 [0,4194304] 0 2024-04-24T08:25:04.697 INFO:tasks.workunit.client.0.smithi090.stdout:1/588: creat d3/d25/d30/d9d/da2/fd6 x:0 0 0 2024-04-24T08:25:04.699 INFO:tasks.workunit.client.0.smithi090.stdout:6/603: stat d8/d1d/d2f/d85/d9f 0 2024-04-24T08:25:04.701 INFO:tasks.workunit.client.0.smithi090.stdout:7/626: mkdir db/d54/ddb 0 2024-04-24T08:25:04.703 INFO:tasks.workunit.client.0.smithi090.stdout:6/604: write d8/d1d/d2f/f83 [2482712,74198] 0 2024-04-24T08:25:04.707 INFO:tasks.workunit.client.0.smithi090.stdout:1/589: fsync d3/d5/fa5 0 2024-04-24T08:25:04.708 INFO:tasks.workunit.client.0.smithi090.stdout:8/530: read - d2/da/d13/f7e zero size 2024-04-24T08:25:04.710 INFO:tasks.workunit.client.0.smithi090.stdout:6/605: unlink d8/dc/d19/d8e/d38/faf 0 2024-04-24T08:25:04.711 INFO:tasks.workunit.client.1.smithi149.stdout:3/942: dread d0/d5/da/d19/d1b/d133/d38/f64 [0,4194304] 0 2024-04-24T08:25:04.714 INFO:tasks.workunit.client.1.smithi149.stdout:6/750: dwrite d0/d1a/f69 [0,4194304] 0 2024-04-24T08:25:04.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/626: truncate d8/d59/dc7/d7a/d50/f71 7414731 0 2024-04-24T08:25:04.723 INFO:tasks.workunit.client.1.smithi149.stdout:4/834: dwrite d2/d18/d1e/d3c/f80 [4194304,4194304] 0 2024-04-24T08:25:04.725 INFO:tasks.workunit.client.0.smithi090.stdout:6/606: link d8/dc/d19/f64 d8/d1d/d1f/fb6 0 2024-04-24T08:25:04.727 INFO:tasks.workunit.client.0.smithi090.stdout:2/628: dwrite d0/d7/f33 [0,4194304] 0 2024-04-24T08:25:04.728 INFO:tasks.workunit.client.0.smithi090.stdout:5/627: symlink d8/d76/dad/lcd 0 2024-04-24T08:25:04.730 INFO:tasks.workunit.client.0.smithi090.stdout:5/628: chown d8/d59 201836 1 2024-04-24T08:25:04.730 INFO:tasks.workunit.client.0.smithi090.stdout:2/629: chown d0/d14/d36/db9 77888 1 2024-04-24T08:25:04.732 INFO:tasks.workunit.client.0.smithi090.stdout:9/542: sync 2024-04-24T08:25:04.732 INFO:tasks.workunit.client.0.smithi090.stdout:0/579: sync 2024-04-24T08:25:04.733 INFO:tasks.workunit.client.1.smithi149.stdout:9/900: dwrite d1/d2/d34/f10a [0,4194304] 0 2024-04-24T08:25:04.736 INFO:tasks.workunit.client.0.smithi090.stdout:6/607: mknod d8/dc/d19/d8e/d38/cb7 0 2024-04-24T08:25:04.741 INFO:tasks.workunit.client.1.smithi149.stdout:1/922: dwrite d0/da/fa3 [4194304,4194304] 0 2024-04-24T08:25:04.742 INFO:tasks.workunit.client.0.smithi090.stdout:2/630: rmdir d0 39 2024-04-24T08:25:04.742 INFO:tasks.workunit.client.1.smithi149.stdout:6/751: mkdir d0/d1a/d28/d31/d4c/d102 0 2024-04-24T08:25:04.743 INFO:tasks.workunit.client.1.smithi149.stdout:3/943: mkdir d0/d5/da/d5a/d63/d6a/d129/d13a 0 2024-04-24T08:25:04.744 INFO:tasks.workunit.client.0.smithi090.stdout:1/590: getdents d3/d1d/d34/d5a 0 2024-04-24T08:25:04.746 INFO:tasks.workunit.client.0.smithi090.stdout:5/629: dwrite d8/d25/f3d [0,4194304] 0 2024-04-24T08:25:04.750 INFO:tasks.workunit.client.0.smithi090.stdout:1/591: stat d3/d5/da/d97/d9c/dc4/ld1 0 2024-04-24T08:25:04.750 INFO:tasks.workunit.client.0.smithi090.stdout:7/627: dread db/f11 [0,4194304] 0 2024-04-24T08:25:04.753 INFO:tasks.workunit.client.1.smithi149.stdout:3/944: write d0/d5/ff [4850550,60992] 0 2024-04-24T08:25:04.759 INFO:tasks.workunit.client.1.smithi149.stdout:6/752: read d0/d1a/d28/d31/f4b [457451,59569] 0 2024-04-24T08:25:04.767 INFO:tasks.workunit.client.0.smithi090.stdout:2/631: write d0/d7/d67/f72 [17652,69687] 0 2024-04-24T08:25:04.767 INFO:tasks.workunit.client.0.smithi090.stdout:9/543: mkdir d0/da5 0 2024-04-24T08:25:04.768 INFO:tasks.workunit.client.0.smithi090.stdout:5/630: creat d8/d59/dc7/d7a/da1/dae/fce x:0 0 0 2024-04-24T08:25:04.771 INFO:tasks.workunit.client.0.smithi090.stdout:7/628: dwrite db/d17/d39/f97 [0,4194304] 0 2024-04-24T08:25:04.773 INFO:tasks.workunit.client.1.smithi149.stdout:4/835: creat d2/d18/d1e/d3c/df0/f109 x:0 0 0 2024-04-24T08:25:04.780 INFO:tasks.workunit.client.1.smithi149.stdout:6/753: rename d0/d1a/f32 to d0/d1a/d28/d31/d4c/d4e/d62/d7a/f103 0 2024-04-24T08:25:04.781 INFO:tasks.workunit.client.1.smithi149.stdout:4/836: chown d2/d24/f9b 53 1 2024-04-24T08:25:04.786 INFO:tasks.workunit.client.0.smithi090.stdout:7/629: dwrite db/d17/d39/d86/dae/fb0 [0,4194304] 0 2024-04-24T08:25:04.793 INFO:tasks.workunit.client.0.smithi090.stdout:2/632: creat d0/d7/d67/d7f/fd7 x:0 0 0 2024-04-24T08:25:04.796 INFO:tasks.workunit.client.1.smithi149.stdout:3/945: dwrite d0/d5/d53/d4c/fa8 [0,4194304] 0 2024-04-24T08:25:04.799 INFO:tasks.workunit.client.1.smithi149.stdout:6/754: mkdir d0/d1a/d28/d31/d38/d104 0 2024-04-24T08:25:04.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/608: rename d8/d1d/d1f/d3e/d4f/c6e to d8/d1d/d1f/d3e/cb8 0 2024-04-24T08:25:04.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/946: readlink d0/d5/d53/dd6/le4 0 2024-04-24T08:25:04.802 INFO:tasks.workunit.client.0.smithi090.stdout:7/630: dwrite db/d17/d29/d32/d5a/d75/f81 [0,4194304] 0 2024-04-24T08:25:04.804 INFO:tasks.workunit.client.0.smithi090.stdout:9/544: dread d0/f18 [0,4194304] 0 2024-04-24T08:25:04.807 INFO:tasks.workunit.client.0.smithi090.stdout:2/633: dread - d0/d14/d4d/f8d zero size 2024-04-24T08:25:04.810 INFO:tasks.workunit.client.0.smithi090.stdout:9/545: fsync d0/dc/d24/d2d/f47 0 2024-04-24T08:25:04.811 INFO:tasks.workunit.client.0.smithi090.stdout:6/609: mknod d8/d1d/d1f/d3e/d44/d90/cb9 0 2024-04-24T08:25:04.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/631: rmdir d8/d59/dc7/d7a/db6 0 2024-04-24T08:25:04.819 INFO:tasks.workunit.client.0.smithi090.stdout:0/580: dwrite d1a/d4b/d53/d79/d35/f3e [0,4194304] 0 2024-04-24T08:25:04.819 INFO:tasks.workunit.client.0.smithi090.stdout:5/632: chown d8/d9c/fc5 2812 1 2024-04-24T08:25:04.821 INFO:tasks.workunit.client.0.smithi090.stdout:2/634: readlink d0/d7/d28/l69 0 2024-04-24T08:25:04.822 INFO:tasks.workunit.client.1.smithi149.stdout:3/947: fsync d0/d5/da/d59/fcf 0 2024-04-24T08:25:04.824 INFO:tasks.workunit.client.0.smithi090.stdout:5/633: fdatasync d8/da/d3a/d96/dcb/f1f 0 2024-04-24T08:25:04.825 INFO:tasks.workunit.client.0.smithi090.stdout:7/631: dwrite db/d17/d29/d38/d3e/fc8 [0,4194304] 0 2024-04-24T08:25:04.828 INFO:tasks.workunit.client.1.smithi149.stdout:9/901: dwrite d1/d2/d33/dea/d133/dad/fc2 [0,4194304] 0 2024-04-24T08:25:04.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/632: chown db/d17/d29/d32/fa3 63600632 1 2024-04-24T08:25:04.830 INFO:tasks.workunit.client.0.smithi090.stdout:7/633: readlink db/d17/lb5 0 2024-04-24T08:25:04.831 INFO:tasks.workunit.client.1.smithi149.stdout:1/923: dwrite d0/d1b/db5/fb1 [0,4194304] 0 2024-04-24T08:25:04.832 INFO:tasks.workunit.client.0.smithi090.stdout:9/546: creat d0/d69/d7d/d5b/fa6 x:0 0 0 2024-04-24T08:25:04.835 INFO:tasks.workunit.client.0.smithi090.stdout:6/610: dwrite d8/d1d/d1f/f5d [0,4194304] 0 2024-04-24T08:25:04.840 INFO:tasks.workunit.client.0.smithi090.stdout:2/635: dwrite d0/d7/d67/d7f/d31/d90/f9d [0,4194304] 0 2024-04-24T08:25:04.842 INFO:tasks.workunit.client.1.smithi149.stdout:4/837: dread d2/d24/d2e/d38/f5e [0,4194304] 0 2024-04-24T08:25:04.851 INFO:tasks.workunit.client.1.smithi149.stdout:4/838: chown d2/d18/d1e/d3c/d48/d69/ldb 1 1 2024-04-24T08:25:04.858 INFO:tasks.workunit.client.0.smithi090.stdout:0/581: unlink d1a/d4b/d53/d79/d3c/c4c 0 2024-04-24T08:25:04.871 INFO:tasks.workunit.client.1.smithi149.stdout:4/839: unlink d2/d10/d1a/d34/f4c 0 2024-04-24T08:25:04.873 INFO:tasks.workunit.client.1.smithi149.stdout:9/902: getdents d1/da/d13/d67/de4 0 2024-04-24T08:25:04.873 INFO:tasks.workunit.client.0.smithi090.stdout:6/611: mknod d8/d1d/d1f/d3e/d44/d90/cba 0 2024-04-24T08:25:04.880 INFO:tasks.workunit.client.1.smithi149.stdout:7/892: sync 2024-04-24T08:25:04.880 INFO:tasks.workunit.client.1.smithi149.stdout:2/760: sync 2024-04-24T08:25:04.880 INFO:tasks.workunit.client.1.smithi149.stdout:8/820: sync 2024-04-24T08:25:04.881 INFO:tasks.workunit.client.1.smithi149.stdout:0/794: sync 2024-04-24T08:25:04.885 INFO:tasks.workunit.client.1.smithi149.stdout:8/821: stat d1/de/d18/d32/cda 0 2024-04-24T08:25:04.886 INFO:tasks.workunit.client.1.smithi149.stdout:3/948: creat d0/d5/d53/d4c/f13b x:0 0 0 2024-04-24T08:25:04.886 INFO:tasks.workunit.client.0.smithi090.stdout:4/492: sync 2024-04-24T08:25:04.886 INFO:tasks.workunit.client.0.smithi090.stdout:3/523: sync 2024-04-24T08:25:04.889 INFO:tasks.workunit.client.0.smithi090.stdout:6/612: dwrite d8/dc/d19/d8e/d81/d96/d97/f9b [0,4194304] 0 2024-04-24T08:25:04.889 INFO:tasks.workunit.client.0.smithi090.stdout:4/493: truncate d3/df/f36 4999689 0 2024-04-24T08:25:04.890 INFO:tasks.workunit.client.0.smithi090.stdout:2/636: mknod d0/d7/d67/d7f/d31/d46/d60/d65/cd8 0 2024-04-24T08:25:04.892 INFO:tasks.workunit.client.0.smithi090.stdout:6/613: dread d8/f3f [0,4194304] 0 2024-04-24T08:25:04.902 INFO:tasks.workunit.client.0.smithi090.stdout:0/582: rename d1a/d4b/d53/d79/d69/f6d to d1a/d4b/d53/d79/d58/d96/fba 0 2024-04-24T08:25:04.904 INFO:tasks.workunit.client.1.smithi149.stdout:9/903: rename d1/d99/l11e to d1/d2/d15/d1c/dca/l14b 0 2024-04-24T08:25:04.905 INFO:tasks.workunit.client.1.smithi149.stdout:4/840: dread d2/d18/ff4 [0,4194304] 0 2024-04-24T08:25:04.905 INFO:tasks.workunit.client.1.smithi149.stdout:2/761: rmdir d0/d6/db/d39/d78/d7c 39 2024-04-24T08:25:04.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/583: read d1a/d4b/d53/f6a [1803604,85422] 0 2024-04-24T08:25:04.913 INFO:tasks.workunit.client.1.smithi149.stdout:6/755: dread d0/f1 [4194304,4194304] 0 2024-04-24T08:25:04.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/822: creat d1/d1f/d22/f115 x:0 0 0 2024-04-24T08:25:04.919 INFO:tasks.workunit.client.0.smithi090.stdout:8/531: sync 2024-04-24T08:25:04.919 INFO:tasks.workunit.client.0.smithi090.stdout:1/592: sync 2024-04-24T08:25:04.920 INFO:tasks.workunit.client.0.smithi090.stdout:1/593: chown d3/d5/fa5 3210453 1 2024-04-24T08:25:04.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/893: dread d9/d43/f7e [0,4194304] 0 2024-04-24T08:25:04.921 INFO:tasks.workunit.client.1.smithi149.stdout:3/949: creat d0/d5/d53/d4c/dca/f13c x:0 0 0 2024-04-24T08:25:04.923 INFO:tasks.workunit.client.0.smithi090.stdout:4/494: creat d3/d38/d6b/f93 x:0 0 0 2024-04-24T08:25:04.927 INFO:tasks.workunit.client.0.smithi090.stdout:2/637: mkdir d0/d7/d67/d7f/d31/d46/d60/dd9 0 2024-04-24T08:25:04.929 INFO:tasks.workunit.client.0.smithi090.stdout:5/634: write d8/da/d3a/d96/dcb/f1e [1705469,104841] 0 2024-04-24T08:25:04.931 INFO:tasks.workunit.client.1.smithi149.stdout:1/924: getdents d0/d1b/d10d 0 2024-04-24T08:25:04.933 INFO:tasks.workunit.client.0.smithi090.stdout:6/614: write d8/d1d/d1f/d3e/d4d/f99 [487685,19463] 0 2024-04-24T08:25:04.936 INFO:tasks.workunit.client.0.smithi090.stdout:8/532: dwrite d2/d62/d83/da3/fa5 [0,4194304] 0 2024-04-24T08:25:04.937 INFO:tasks.workunit.client.1.smithi149.stdout:0/795: mkdir d2/d9/df/d56/dca/d100 0 2024-04-24T08:25:04.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/533: stat d2/d62 0 2024-04-24T08:25:04.943 INFO:tasks.workunit.client.1.smithi149.stdout:8/823: rmdir d1/d1f/d5e 39 2024-04-24T08:25:04.946 INFO:tasks.workunit.client.0.smithi090.stdout:3/524: mkdir d2/df/db5 0 2024-04-24T08:25:04.946 INFO:tasks.workunit.client.1.smithi149.stdout:6/756: rmdir d0/d1a/d28/d31/d4c/d4e/d62/d66 39 2024-04-24T08:25:04.950 INFO:tasks.workunit.client.0.smithi090.stdout:9/547: dwrite d0/d2a/f38 [0,4194304] 0 2024-04-24T08:25:04.950 INFO:tasks.workunit.client.0.smithi090.stdout:3/525: fdatasync d2/df/d1a/d37/f55 0 2024-04-24T08:25:04.952 INFO:tasks.workunit.client.1.smithi149.stdout:9/904: mknod d1/da/c14c 0 2024-04-24T08:25:04.953 INFO:tasks.workunit.client.0.smithi090.stdout:7/634: getdents db/d54 0 2024-04-24T08:25:04.954 INFO:tasks.workunit.client.0.smithi090.stdout:8/534: dread d2/da/d13/d34/f3d [0,4194304] 0 2024-04-24T08:25:04.961 INFO:tasks.workunit.client.0.smithi090.stdout:6/615: dwrite d8/dc/d19/d8e/fa8 [0,4194304] 0 2024-04-24T08:25:04.965 INFO:tasks.workunit.client.1.smithi149.stdout:0/796: unlink d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fcb 0 2024-04-24T08:25:04.969 INFO:tasks.workunit.client.1.smithi149.stdout:0/797: chown d2/d17/d41/d68/ddc/lf1 339834 1 2024-04-24T08:25:04.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/548: symlink d0/d19/la7 0 2024-04-24T08:25:04.972 INFO:tasks.workunit.client.0.smithi090.stdout:2/638: mkdir d0/d7/dda 0 2024-04-24T08:25:04.978 INFO:tasks.workunit.client.0.smithi090.stdout:8/535: mknod d2/d97/cb8 0 2024-04-24T08:25:04.984 INFO:tasks.workunit.client.0.smithi090.stdout:4/495: rmdir d3/d38/d6b 39 2024-04-24T08:25:04.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/584: creat d1a/d4b/d53/fbb x:0 0 0 2024-04-24T08:25:04.986 INFO:tasks.workunit.client.0.smithi090.stdout:6/616: dwrite d8/d1d/d1f/d3e/d4f/d7e/f80 [0,4194304] 0 2024-04-24T08:25:04.987 INFO:tasks.workunit.client.1.smithi149.stdout:6/757: creat d0/d1a/d28/d31/d4c/d102/f105 x:0 0 0 2024-04-24T08:25:04.990 INFO:tasks.workunit.client.0.smithi090.stdout:6/617: fdatasync d8/d1d/d1f/d3e/d44/f82 0 2024-04-24T08:25:04.994 INFO:tasks.workunit.client.0.smithi090.stdout:2/639: dwrite d0/d5e/f63 [0,4194304] 0 2024-04-24T08:25:04.995 INFO:tasks.workunit.client.0.smithi090.stdout:6/618: fdatasync d8/d1d/d1f/d3e/d44/f91 0 2024-04-24T08:25:04.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/640: chown d0/d7/dda 231763 1 2024-04-24T08:25:04.998 INFO:tasks.workunit.client.1.smithi149.stdout:1/925: rename d0/da/c47 to d0/d5/db/d63/c128 0 2024-04-24T08:25:04.999 INFO:tasks.workunit.client.0.smithi090.stdout:6/619: chown d8/d1d/d2f/c92 1759 1 2024-04-24T08:25:05.002 INFO:tasks.workunit.client.1.smithi149.stdout:7/894: dread d9/dc/fb7 [0,4194304] 0 2024-04-24T08:25:05.003 INFO:tasks.workunit.client.1.smithi149.stdout:2/762: link d0/d6/db/d39/d3d/fb5 d0/d6/db/d39/d78/d7c/fe8 0 2024-04-24T08:25:05.006 INFO:tasks.workunit.client.0.smithi090.stdout:4/496: dwrite d3/d38/d49/d6f/f51 [0,4194304] 0 2024-04-24T08:25:05.008 INFO:tasks.workunit.client.0.smithi090.stdout:8/536: dread - d2/da/d13/f5c zero size 2024-04-24T08:25:05.009 INFO:tasks.workunit.client.1.smithi149.stdout:6/758: mkdir d0/d1a/d28/d31/d38/d106 0 2024-04-24T08:25:05.012 INFO:tasks.workunit.client.1.smithi149.stdout:3/950: getdents d0/d5/d1a/d70/da3 0 2024-04-24T08:25:05.013 INFO:tasks.workunit.client.0.smithi090.stdout:0/585: fsync f18 0 2024-04-24T08:25:05.014 INFO:tasks.workunit.client.1.smithi149.stdout:3/951: chown d0/d88/l8d 530 1 2024-04-24T08:25:05.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/586: write d1a/f1d [1504928,70584] 0 2024-04-24T08:25:05.017 INFO:tasks.workunit.client.1.smithi149.stdout:0/798: write d2/d3c/fc6 [1704756,15180] 0 2024-04-24T08:25:05.019 INFO:tasks.workunit.client.0.smithi090.stdout:1/594: dwrite d3/d25/d30/d9d/fd3 [0,4194304] 0 2024-04-24T08:25:05.020 INFO:tasks.workunit.client.1.smithi149.stdout:9/905: rename d1/d2/d15/de7/l121 to d1/da/dd/d79/l14d 0 2024-04-24T08:25:05.021 INFO:tasks.workunit.client.0.smithi090.stdout:3/526: dwrite d2/df/d1a/d37/d53/f65 [0,4194304] 0 2024-04-24T08:25:05.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/635: link db/d17/d29/d32/d5a/d75/fbe db/d54/fdc 0 2024-04-24T08:25:05.024 INFO:tasks.workunit.client.1.smithi149.stdout:1/926: creat d0/d5/d4f/f129 x:0 0 0 2024-04-24T08:25:05.026 INFO:tasks.workunit.client.1.smithi149.stdout:0/799: read d2/d9/f2d [4113351,52033] 0 2024-04-24T08:25:05.028 INFO:tasks.workunit.client.0.smithi090.stdout:9/549: dwrite d0/dc/d24/d7c/d4d/f5f [0,4194304] 0 2024-04-24T08:25:05.032 INFO:tasks.workunit.client.1.smithi149.stdout:8/824: link d1/ffb d1/d1f/d25/d7a/f116 0 2024-04-24T08:25:05.035 INFO:tasks.workunit.client.0.smithi090.stdout:6/620: creat d8/dc/d19/d8e/d81/d96/fbb x:0 0 0 2024-04-24T08:25:05.035 INFO:tasks.workunit.client.1.smithi149.stdout:6/759: read - d0/d1a/d28/f9c zero size 2024-04-24T08:25:05.037 INFO:tasks.workunit.client.0.smithi090.stdout:6/621: chown d8/d1d/d29/f3b 178591483 1 2024-04-24T08:25:05.037 INFO:tasks.workunit.client.1.smithi149.stdout:3/952: rename d0/d5/d53/c4d to d0/d5/d53/d39/c13d 0 2024-04-24T08:25:05.044 INFO:tasks.workunit.client.0.smithi090.stdout:1/595: dwrite d3/f41 [0,4194304] 0 2024-04-24T08:25:05.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/537: dread d2/da/f82 [0,4194304] 0 2024-04-24T08:25:05.050 INFO:tasks.workunit.client.0.smithi090.stdout:7/636: symlink db/d17/d39/d59/ldd 0 2024-04-24T08:25:05.052 INFO:tasks.workunit.client.1.smithi149.stdout:2/763: creat d0/d6/db/d39/d78/fe9 x:0 0 0 2024-04-24T08:25:05.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/622: dwrite d8/d1d/d2f/f88 [0,4194304] 0 2024-04-24T08:25:05.056 INFO:tasks.workunit.client.0.smithi090.stdout:2/641: mkdir d0/d7/d67/d7f/d31/ddb 0 2024-04-24T08:25:05.058 INFO:tasks.workunit.client.0.smithi090.stdout:2/642: write d0/d14/f34 [1913976,99890] 0 2024-04-24T08:25:05.058 INFO:tasks.workunit.client.1.smithi149.stdout:8/825: creat d1/de/d18/d43/d10e/f117 x:0 0 0 2024-04-24T08:25:05.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/643: fsync d0/f21 0 2024-04-24T08:25:05.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/644: stat d0/d7 0 2024-04-24T08:25:05.063 INFO:tasks.workunit.client.1.smithi149.stdout:6/760: unlink d0/d1a/d28/d31/d4c/d4e/d62/d66/f7b 0 2024-04-24T08:25:05.063 INFO:tasks.workunit.client.1.smithi149.stdout:1/927: link d0/d5/db/d18/fb9 d0/d1b/d10d/d121/f12a 0 2024-04-24T08:25:05.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/761: stat d0/d1a/d28/f95 0 2024-04-24T08:25:05.066 INFO:tasks.workunit.client.0.smithi090.stdout:1/596: creat d3/d5/da/d97/fd7 x:0 0 0 2024-04-24T08:25:05.066 INFO:tasks.workunit.client.1.smithi149.stdout:2/764: truncate d0/d6/db/d28/f6f 4675649 0 2024-04-24T08:25:05.066 INFO:tasks.workunit.client.1.smithi149.stdout:1/928: stat d0/d1b/d5d/f110 0 2024-04-24T08:25:05.067 INFO:tasks.workunit.client.0.smithi090.stdout:0/587: mkdir d1a/d4b/d53/d79/d69/d90/dbc 0 2024-04-24T08:25:05.070 INFO:tasks.workunit.client.1.smithi149.stdout:8/826: chown d1/de/d18/dd3/l107 7981 1 2024-04-24T08:25:05.072 INFO:tasks.workunit.client.1.smithi149.stdout:2/765: readlink d0/d6/l19 0 2024-04-24T08:25:05.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/623: creat d8/d1d/d2f/d85/fbc x:0 0 0 2024-04-24T08:25:05.074 INFO:tasks.workunit.client.0.smithi090.stdout:4/497: creat d3/d18/f94 x:0 0 0 2024-04-24T08:25:05.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/624: write d8/dc/d19/d8e/d81/d96/d97/fae [4984606,90151] 0 2024-04-24T08:25:05.075 INFO:tasks.workunit.client.1.smithi149.stdout:8/827: readlink d1/de/d18/d32/d63/d9e/le3 0 2024-04-24T08:25:05.076 INFO:tasks.workunit.client.1.smithi149.stdout:6/762: read d0/d1a/d28/d31/d4c/d4e/d62/fbc [199175,65986] 0 2024-04-24T08:25:05.077 INFO:tasks.workunit.client.0.smithi090.stdout:4/498: stat d3/d38/d49/d6f/f7d 0 2024-04-24T08:25:05.078 INFO:tasks.workunit.client.0.smithi090.stdout:5/635: sync 2024-04-24T08:25:05.082 INFO:tasks.workunit.client.0.smithi090.stdout:2/645: mkdir d0/d7/d28/daa/ddc 0 2024-04-24T08:25:05.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/766: rmdir d0/d6/db/d39/d78/d7c 39 2024-04-24T08:25:05.086 INFO:tasks.workunit.client.0.smithi090.stdout:1/597: rmdir d3/d1d 39 2024-04-24T08:25:05.091 INFO:tasks.workunit.client.1.smithi149.stdout:1/929: symlink d0/ddf/l12b 0 2024-04-24T08:25:05.091 INFO:tasks.workunit.client.0.smithi090.stdout:0/588: write d1a/f3b [739373,4879] 0 2024-04-24T08:25:05.095 INFO:tasks.workunit.client.1.smithi149.stdout:6/763: chown d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/f100 20 1 2024-04-24T08:25:05.096 INFO:tasks.workunit.client.1.smithi149.stdout:9/906: dread d1/d2/d33/f80 [0,4194304] 0 2024-04-24T08:25:05.097 INFO:tasks.workunit.client.0.smithi090.stdout:5/636: creat d8/d76/fcf x:0 0 0 2024-04-24T08:25:05.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/646: chown d0/d7/d67/d7f/d31/d46/d60/d65/l6c 4792125 1 2024-04-24T08:25:05.108 INFO:tasks.workunit.client.1.smithi149.stdout:6/764: mkdir d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/d107 0 2024-04-24T08:25:05.112 INFO:tasks.workunit.client.0.smithi090.stdout:6/625: dread d8/f46 [0,4194304] 0 2024-04-24T08:25:05.113 INFO:tasks.workunit.client.1.smithi149.stdout:1/930: creat d0/d5/df/f12c x:0 0 0 2024-04-24T08:25:05.121 INFO:tasks.workunit.client.1.smithi149.stdout:9/907: rename d1/da/d13/d74/d11a/l125 to d1/d2/d33/d5d/d83/df4/l14e 0 2024-04-24T08:25:05.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/767: dread d0/d6c/fa8 [0,4194304] 0 2024-04-24T08:25:05.126 INFO:tasks.workunit.client.1.smithi149.stdout:0/800: dread d2/d9/df/d2f/d6c/dc1/d4e/fa0 [0,4194304] 0 2024-04-24T08:25:05.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/538: link d2/d62/d6b/c77 d2/da/d13/d64/d24/db3/db4/cb9 0 2024-04-24T08:25:05.129 INFO:tasks.workunit.client.1.smithi149.stdout:8/828: dwrite d1/dce/fed [0,4194304] 0 2024-04-24T08:25:05.129 INFO:tasks.workunit.client.0.smithi090.stdout:1/598: truncate d3/d5/f17 4725456 0 2024-04-24T08:25:05.131 INFO:tasks.workunit.client.1.smithi149.stdout:8/829: readlink d1/de/d18/l1e 0 2024-04-24T08:25:05.131 INFO:tasks.workunit.client.1.smithi149.stdout:4/841: sync 2024-04-24T08:25:05.134 INFO:tasks.workunit.client.1.smithi149.stdout:8/830: read d1/de/d18/f6a [63902,15238] 0 2024-04-24T08:25:05.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/931: mkdir d0/d1b/d5d/dd6/ddb/d127/d12d 0 2024-04-24T08:25:05.135 INFO:tasks.workunit.client.1.smithi149.stdout:4/842: fsync d2/d10/d91/d9f/f106 0 2024-04-24T08:25:05.140 INFO:tasks.workunit.client.1.smithi149.stdout:4/843: write d2/d10/d91/f9d [8541802,12482] 0 2024-04-24T08:25:05.141 INFO:tasks.workunit.client.1.smithi149.stdout:9/908: mknod d1/da/de8/c14f 0 2024-04-24T08:25:05.145 INFO:tasks.workunit.client.0.smithi090.stdout:7/637: getdents db/d17/d39/d59 0 2024-04-24T08:25:05.145 INFO:tasks.workunit.client.1.smithi149.stdout:9/909: chown d1/d2/d33/dea/d133/dad/db7 0 1 2024-04-24T08:25:05.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/801: mknod d2/d9/df/d40/c101 0 2024-04-24T08:25:05.147 INFO:tasks.workunit.client.1.smithi149.stdout:6/765: dread d0/d1a/d28/d31/d4c/d80/dbd/ff7 [0,4194304] 0 2024-04-24T08:25:05.148 INFO:tasks.workunit.client.1.smithi149.stdout:3/953: dwrite d0/d5/da/d19/faf [0,4194304] 0 2024-04-24T08:25:05.148 INFO:tasks.workunit.client.1.smithi149.stdout:4/844: truncate d2/d10/d1a/fda 957600 0 2024-04-24T08:25:05.149 INFO:tasks.workunit.client.0.smithi090.stdout:9/550: write d0/dc/f29 [2737673,6409] 0 2024-04-24T08:25:05.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/647: mknod d0/d7/d67/d7f/d31/cdd 0 2024-04-24T08:25:05.151 INFO:tasks.workunit.client.0.smithi090.stdout:2/648: stat d0/d14/la6 0 2024-04-24T08:25:05.153 INFO:tasks.workunit.client.0.smithi090.stdout:2/649: stat d0/d14/l20 0 2024-04-24T08:25:05.154 INFO:tasks.workunit.client.0.smithi090.stdout:8/539: fsync d2/da/d13/f5c 0 2024-04-24T08:25:05.155 INFO:tasks.workunit.client.1.smithi149.stdout:6/766: dread - d0/d1a/d28/d31/d4c/df8/ffd zero size 2024-04-24T08:25:05.157 INFO:tasks.workunit.client.1.smithi149.stdout:4/845: chown d2/d18/d1e/d3c/d48/d69/d7d/de9/ff6 604 1 2024-04-24T08:25:05.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/540: read d2/da/d13/f28 [181940,10668] 0 2024-04-24T08:25:05.162 INFO:tasks.workunit.client.1.smithi149.stdout:1/932: rmdir d0/d1b/d5d/de5/d101 39 2024-04-24T08:25:05.167 INFO:tasks.workunit.client.1.smithi149.stdout:9/910: unlink d1/d2/d15/f143 0 2024-04-24T08:25:05.168 INFO:tasks.workunit.client.0.smithi090.stdout:9/551: dwrite d0/dc/d24/d7c/d4d/f5f [4194304,4194304] 0 2024-04-24T08:25:05.169 INFO:tasks.workunit.client.0.smithi090.stdout:3/527: truncate d2/f24 2292359 0 2024-04-24T08:25:05.178 INFO:tasks.workunit.client.1.smithi149.stdout:0/802: symlink d2/d9/df/d2f/d6c/d72/l102 0 2024-04-24T08:25:05.183 INFO:tasks.workunit.client.1.smithi149.stdout:6/767: unlink d0/d1a/d28/lae 0 2024-04-24T08:25:05.183 INFO:tasks.workunit.client.1.smithi149.stdout:6/768: readlink d0/d1a/d7e/la5 0 2024-04-24T08:25:05.184 INFO:tasks.workunit.client.0.smithi090.stdout:8/541: creat d2/d62/d6b/fba x:0 0 0 2024-04-24T08:25:05.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/589: write d1a/d4b/d53/d79/d3c/f91 [799469,49190] 0 2024-04-24T08:25:05.195 INFO:tasks.workunit.client.1.smithi149.stdout:9/911: chown d1/d2/d15/d1c/l53 116 1 2024-04-24T08:25:05.196 INFO:tasks.workunit.client.0.smithi090.stdout:5/637: write d8/da/d3a/d2d/f4d [426802,25421] 0 2024-04-24T08:25:05.199 INFO:tasks.workunit.client.0.smithi090.stdout:4/499: dwrite d3/d38/d6b/f6d [0,4194304] 0 2024-04-24T08:25:05.202 INFO:tasks.workunit.client.0.smithi090.stdout:8/542: fsync d2/f12 0 2024-04-24T08:25:05.204 INFO:tasks.workunit.client.1.smithi149.stdout:0/803: dwrite d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fed [0,4194304] 0 2024-04-24T08:25:05.204 INFO:tasks.workunit.client.1.smithi149.stdout:6/769: unlink d0/fb 0 2024-04-24T08:25:05.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/552: symlink d0/dc/d24/d2d/d49/la8 0 2024-04-24T08:25:05.205 INFO:tasks.workunit.client.0.smithi090.stdout:8/543: fdatasync d2/da/d13/d64/f8f 0 2024-04-24T08:25:05.205 INFO:tasks.workunit.client.0.smithi090.stdout:9/553: chown d0/da5 116102155 1 2024-04-24T08:25:05.206 INFO:tasks.workunit.client.0.smithi090.stdout:8/544: fsync d2/d62/f93 0 2024-04-24T08:25:05.207 INFO:tasks.workunit.client.0.smithi090.stdout:9/554: read - d0/dc/d24/d7c/f86 zero size 2024-04-24T08:25:05.213 INFO:tasks.workunit.client.1.smithi149.stdout:2/768: getdents d0/d6/db/d39/d3d/d44 0 2024-04-24T08:25:05.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/638: link db/d17/d39/ld9 db/d17/d29/d38/d99/lde 0 2024-04-24T08:25:05.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/638: dwrite d8/da/d14/d2f/f66 [0,4194304] 0 2024-04-24T08:25:05.214 INFO:tasks.workunit.client.0.smithi090.stdout:1/599: write d3/d5/da/dc/f81 [1588552,111090] 0 2024-04-24T08:25:05.217 INFO:tasks.workunit.client.1.smithi149.stdout:7/895: sync 2024-04-24T08:25:05.221 INFO:tasks.workunit.client.0.smithi090.stdout:6/626: truncate d8/d1d/d1f/f58 4043401 0 2024-04-24T08:25:05.225 INFO:tasks.workunit.client.0.smithi090.stdout:9/555: dwrite d0/dc/d24/d2d/f93 [0,4194304] 0 2024-04-24T08:25:05.233 INFO:tasks.workunit.client.0.smithi090.stdout:9/556: chown d0/dc/d24/d7c/d4d/d32/f3c 92747757 1 2024-04-24T08:25:05.233 INFO:tasks.workunit.client.1.smithi149.stdout:3/954: write d0/d5/da/d59/d79/f98 [3371430,33113] 0 2024-04-24T08:25:05.233 INFO:tasks.workunit.client.1.smithi149.stdout:8/831: dwrite d1/de/d18/f3e [0,4194304] 0 2024-04-24T08:25:05.237 INFO:tasks.workunit.client.1.smithi149.stdout:4/846: write d2/d24/d2e/d38/f5e [2062865,56168] 0 2024-04-24T08:25:05.238 INFO:tasks.workunit.client.1.smithi149.stdout:6/770: rename d0/d1a/d28/l45 to d0/d1a/d28/d31/d4c/d8a/l108 0 2024-04-24T08:25:05.242 INFO:tasks.workunit.client.1.smithi149.stdout:3/955: chown d0/d5/da/d19/d1b/d133/d38/f50 61 1 2024-04-24T08:25:05.244 INFO:tasks.workunit.client.1.smithi149.stdout:7/896: symlink d9/dc/d7b/l127 0 2024-04-24T08:25:05.245 INFO:tasks.workunit.client.1.smithi149.stdout:3/956: chown d0/d88/l8d 1 1 2024-04-24T08:25:05.246 INFO:tasks.workunit.client.1.smithi149.stdout:6/771: dread - d0/d1a/d28/d31/d4c/fe4 zero size 2024-04-24T08:25:05.249 INFO:tasks.workunit.client.0.smithi090.stdout:8/545: mkdir d2/da/d13/d64/db2/dbb 0 2024-04-24T08:25:05.254 INFO:tasks.workunit.client.0.smithi090.stdout:7/639: mknod db/d17/d29/d38/d99/cdf 0 2024-04-24T08:25:05.255 INFO:tasks.workunit.client.0.smithi090.stdout:7/640: write db/d17/d29/f30 [1001948,26877] 0 2024-04-24T08:25:05.258 INFO:tasks.workunit.client.1.smithi149.stdout:8/832: fsync d1/de/d18/d91/d9f/fc6 0 2024-04-24T08:25:05.258 INFO:tasks.workunit.client.1.smithi149.stdout:7/897: creat d9/dc/d11/d83/f128 x:0 0 0 2024-04-24T08:25:05.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/639: dread - d8/da/d3a/d2d/d37/f63 zero size 2024-04-24T08:25:05.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/640: read d8/d99/f9d [3460136,34833] 0 2024-04-24T08:25:05.265 INFO:tasks.workunit.client.1.smithi149.stdout:1/933: write d0/d1b/db5/f2b [3403960,1333] 0 2024-04-24T08:25:05.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/650: link d0/lc d0/d7/dda/lde 0 2024-04-24T08:25:05.267 INFO:tasks.workunit.client.0.smithi090.stdout:2/651: stat d0/d2e/ca5 0 2024-04-24T08:25:05.272 INFO:tasks.workunit.client.0.smithi090.stdout:9/557: mknod d0/d69/d7d/d5b/ca9 0 2024-04-24T08:25:05.275 INFO:tasks.workunit.client.0.smithi090.stdout:9/558: write d0/d2a/f8e [333738,2309] 0 2024-04-24T08:25:05.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:05.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:25:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:25:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:05.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:04 smithi149 ceph-mon[31282]: pgmap v25: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 65 MiB/s rd, 177 MiB/s wr, 359 op/s 2024-04-24T08:25:05.280 INFO:tasks.workunit.client.0.smithi090.stdout:3/528: getdents d2/df/d1a/d37/d4b 0 2024-04-24T08:25:05.282 INFO:tasks.workunit.client.1.smithi149.stdout:9/912: getdents d1/d2/d33/d5d/d83/df4 0 2024-04-24T08:25:05.286 INFO:tasks.workunit.client.1.smithi149.stdout:8/833: symlink d1/de/d18/d32/d63/d9e/l118 0 2024-04-24T08:25:05.287 INFO:tasks.workunit.client.1.smithi149.stdout:9/913: write d1/da/f8 [608021,49596] 0 2024-04-24T08:25:05.288 INFO:tasks.workunit.client.0.smithi090.stdout:8/546: creat d2/d62/fbc x:0 0 0 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:04 smithi090 ceph-mon[25016]: pgmap v25: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 65 MiB/s rd, 177 MiB/s wr, 359 op/s 2024-04-24T08:25:05.293 INFO:tasks.workunit.client.1.smithi149.stdout:8/834: read d1/d1f/f28 [3453539,108810] 0 2024-04-24T08:25:05.294 INFO:tasks.workunit.client.1.smithi149.stdout:6/772: dread d0/f4 [0,4194304] 0 2024-04-24T08:25:05.295 INFO:tasks.workunit.client.1.smithi149.stdout:4/847: write d2/d10/d1a/da2/fa3 [3169570,5570] 0 2024-04-24T08:25:05.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/500: dwrite d3/d18/d5c/f4a [0,4194304] 0 2024-04-24T08:25:05.299 INFO:tasks.workunit.client.0.smithi090.stdout:7/641: creat db/dbd/fe0 x:0 0 0 2024-04-24T08:25:05.300 INFO:tasks.workunit.client.1.smithi149.stdout:0/804: dwrite d2/d17/d41/d68/f82 [0,4194304] 0 2024-04-24T08:25:05.303 INFO:tasks.workunit.client.1.smithi149.stdout:1/934: creat d0/d5/db/d18/d4d/de8/dfb/f12e x:0 0 0 2024-04-24T08:25:05.303 INFO:tasks.workunit.client.0.smithi090.stdout:7/642: dread db/d17/d29/d38/d3e/fb7 [0,4194304] 0 2024-04-24T08:25:05.304 INFO:tasks.workunit.client.1.smithi149.stdout:3/957: symlink d0/d5/d53/d4c/d5f/l13e 0 2024-04-24T08:25:05.306 INFO:tasks.workunit.client.1.smithi149.stdout:4/848: rmdir d2/d24/d2e 39 2024-04-24T08:25:05.309 INFO:tasks.workunit.client.0.smithi090.stdout:5/641: rename d8/d59/dc7/d7a/f6f to d8/d56/fd0 0 2024-04-24T08:25:05.309 INFO:tasks.workunit.client.0.smithi090.stdout:2/652: rmdir d0/d7/dda 39 2024-04-24T08:25:05.310 INFO:tasks.workunit.client.1.smithi149.stdout:8/835: write d1/d1f/d31/f84 [799192,94013] 0 2024-04-24T08:25:05.312 INFO:tasks.workunit.client.0.smithi090.stdout:5/642: chown d8/d59/dc7/d7a/d50/lb1 3 1 2024-04-24T08:25:05.313 INFO:tasks.workunit.client.1.smithi149.stdout:0/805: truncate d2/d9/df/fec 762752 0 2024-04-24T08:25:05.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/590: getdents d1a/d4b/d53/d79/d69/d8a 0 2024-04-24T08:25:05.318 INFO:tasks.workunit.client.1.smithi149.stdout:9/914: rename d1/d2/d33/f66 to d1/da/d13/d74/dbb/f150 0 2024-04-24T08:25:05.318 INFO:tasks.workunit.client.0.smithi090.stdout:8/547: creat d2/d62/d83/da3/fbd x:0 0 0 2024-04-24T08:25:05.319 INFO:tasks.workunit.client.0.smithi090.stdout:2/653: read d0/d14/d36/fca [523059,35026] 0 2024-04-24T08:25:05.320 INFO:tasks.workunit.client.1.smithi149.stdout:2/769: dwrite d0/d6/db/d39/d3d/d44/d57/f71 [0,4194304] 0 2024-04-24T08:25:05.320 INFO:tasks.workunit.client.1.smithi149.stdout:1/935: creat d0/d1b/d5d/dd6/f12f x:0 0 0 2024-04-24T08:25:05.320 INFO:tasks.workunit.client.0.smithi090.stdout:8/548: chown d2/d62/d71/c80 332452 1 2024-04-24T08:25:05.322 INFO:tasks.workunit.client.1.smithi149.stdout:3/958: dread - d0/d26/d3c/f10c zero size 2024-04-24T08:25:05.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/501: creat d3/d18/d5c/f95 x:0 0 0 2024-04-24T08:25:05.327 INFO:tasks.workunit.client.1.smithi149.stdout:2/770: readlink d0/d6/db/d27/d81/d9f/ld1 0 2024-04-24T08:25:05.332 INFO:tasks.workunit.client.0.smithi090.stdout:6/627: rmdir d8/db5 0 2024-04-24T08:25:05.333 INFO:tasks.workunit.client.1.smithi149.stdout:2/771: fsync d0/d6/db/d28/f48 0 2024-04-24T08:25:05.334 INFO:tasks.workunit.client.1.smithi149.stdout:8/836: mkdir d1/de/d18/d43/d10e/d119 0 2024-04-24T08:25:05.342 INFO:tasks.workunit.client.1.smithi149.stdout:0/806: unlink d2/d9/df/d40/c5b 0 2024-04-24T08:25:05.344 INFO:tasks.workunit.client.1.smithi149.stdout:3/959: mkdir d0/d5/da/d19/d1b/d133/d38/d13f 0 2024-04-24T08:25:05.344 INFO:tasks.workunit.client.1.smithi149.stdout:0/807: stat d2/d3c/dc7/le9 0 2024-04-24T08:25:05.350 INFO:tasks.workunit.client.1.smithi149.stdout:2/772: rename d0/d6/db/d27/d81/db4/ld8 to d0/d6c/lea 0 2024-04-24T08:25:05.353 INFO:tasks.workunit.client.1.smithi149.stdout:0/808: write d2/d9/df/fec [1103038,17628] 0 2024-04-24T08:25:05.359 INFO:tasks.workunit.client.1.smithi149.stdout:7/898: dread d9/dc/d11/f2b [0,4194304] 0 2024-04-24T08:25:05.368 INFO:tasks.workunit.client.1.smithi149.stdout:7/899: fdatasync d9/dc/de/d44/fd5 0 2024-04-24T08:25:05.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/809: dwrite d2/d9/df/d56/f5f [0,4194304] 0 2024-04-24T08:25:05.382 INFO:tasks.workunit.client.1.smithi149.stdout:7/900: fdatasync d9/d43/d95/fd0 0 2024-04-24T08:25:05.394 INFO:tasks.workunit.client.1.smithi149.stdout:0/810: fdatasync d2/d9/df/d2f/d6c/fbb 0 2024-04-24T08:25:05.394 INFO:tasks.workunit.client.0.smithi090.stdout:1/600: dwrite d3/d1d/d34/d73/f99 [0,4194304] 0 2024-04-24T08:25:05.401 INFO:tasks.workunit.client.1.smithi149.stdout:7/901: creat d9/d43/d4f/dc8/f129 x:0 0 0 2024-04-24T08:25:05.401 INFO:tasks.workunit.client.1.smithi149.stdout:1/936: write d0/f25 [876609,55462] 0 2024-04-24T08:25:05.409 INFO:tasks.workunit.client.1.smithi149.stdout:0/811: unlink d2/d9/df/d2f/f34 0 2024-04-24T08:25:05.413 INFO:tasks.workunit.client.1.smithi149.stdout:8/837: dread d1/de/f38 [0,4194304] 0 2024-04-24T08:25:05.415 INFO:tasks.workunit.client.0.smithi090.stdout:9/559: rename d0/d19/f1d to d0/d2a/faa 0 2024-04-24T08:25:05.416 INFO:tasks.workunit.client.1.smithi149.stdout:9/915: dwrite d1/da/dd/f24 [0,4194304] 0 2024-04-24T08:25:05.417 INFO:tasks.workunit.client.0.smithi090.stdout:3/529: symlink d2/d15/da6/lb6 0 2024-04-24T08:25:05.421 INFO:tasks.workunit.client.0.smithi090.stdout:9/560: write d0/dc/d24/d7c/d4d/d32/f3c [1432916,81915] 0 2024-04-24T08:25:05.422 INFO:tasks.workunit.client.0.smithi090.stdout:0/591: mkdir d1a/d4b/d53/d79/d35/dbd 0 2024-04-24T08:25:05.424 INFO:tasks.workunit.client.1.smithi149.stdout:6/773: dwrite d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/f100 [0,4194304] 0 2024-04-24T08:25:05.425 INFO:tasks.workunit.client.1.smithi149.stdout:8/838: chown d1/de/d18/d32/d65/d8b/l9b 0 1 2024-04-24T08:25:05.430 INFO:tasks.workunit.client.0.smithi090.stdout:8/549: truncate d2/da/d13/d34/f72 47087 0 2024-04-24T08:25:05.432 INFO:tasks.workunit.client.0.smithi090.stdout:8/550: stat d2/da/f87 0 2024-04-24T08:25:05.433 INFO:tasks.workunit.client.1.smithi149.stdout:6/774: chown d0/d1a/d28/ffe 3999424 1 2024-04-24T08:25:05.434 INFO:tasks.workunit.client.0.smithi090.stdout:4/502: symlink d3/df/d77/l96 0 2024-04-24T08:25:05.434 INFO:tasks.workunit.client.1.smithi149.stdout:4/849: dwrite d2/d18/d1e/d3c/d48/d69/f70 [0,4194304] 0 2024-04-24T08:25:05.435 INFO:tasks.workunit.client.0.smithi090.stdout:4/503: readlink d3/d25/l48 0 2024-04-24T08:25:05.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/643: readlink db/d54/l63 0 2024-04-24T08:25:05.443 INFO:tasks.workunit.client.1.smithi149.stdout:3/960: dwrite d0/d5/d1a/d70/fba [0,4194304] 0 2024-04-24T08:25:05.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/628: creat d8/dc/d19/d8e/d81/d96/fbd x:0 0 0 2024-04-24T08:25:05.450 INFO:tasks.workunit.client.0.smithi090.stdout:1/601: unlink d3/d5/d3e/c29 0 2024-04-24T08:25:05.459 INFO:tasks.workunit.client.0.smithi090.stdout:9/561: mkdir d0/dc/d24/d7c/d4d/d32/dab 0 2024-04-24T08:25:05.470 INFO:tasks.workunit.client.0.smithi090.stdout:4/504: dwrite d3/df/f36 [4194304,4194304] 0 2024-04-24T08:25:05.476 INFO:tasks.workunit.client.0.smithi090.stdout:6/629: chown d8/d1d/d1f/d3e/c4e 149437 1 2024-04-24T08:25:05.477 INFO:tasks.workunit.client.0.smithi090.stdout:9/562: mknod d0/d69/d7d/d5b/d96/cac 0 2024-04-24T08:25:05.478 INFO:tasks.workunit.client.0.smithi090.stdout:2/654: dwrite d0/d7/f11 [0,4194304] 0 2024-04-24T08:25:05.479 INFO:tasks.workunit.client.0.smithi090.stdout:9/563: stat d0/dc/d24/d7c/d4d/l23 0 2024-04-24T08:25:05.482 INFO:tasks.workunit.client.0.smithi090.stdout:6/630: truncate d8/dc/d19/d8e/d81/d96/fbd 656518 0 2024-04-24T08:25:05.483 INFO:tasks.workunit.client.0.smithi090.stdout:7/644: truncate db/d17/d29/f8e 345458 0 2024-04-24T08:25:05.485 INFO:tasks.workunit.client.0.smithi090.stdout:7/645: chown db/c5b 108587643 1 2024-04-24T08:25:05.485 INFO:tasks.workunit.client.0.smithi090.stdout:1/602: fsync d3/d5/f16 0 2024-04-24T08:25:05.490 INFO:tasks.workunit.client.0.smithi090.stdout:4/505: fsync d3/d18/d1d/f1f 0 2024-04-24T08:25:05.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/631: creat d8/d1d/d1f/d3e/d4f/d7e/fbe x:0 0 0 2024-04-24T08:25:05.496 INFO:tasks.workunit.client.0.smithi090.stdout:8/551: getdents d2/da/d13/d34 0 2024-04-24T08:25:05.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/506: creat d3/d8e/f97 x:0 0 0 2024-04-24T08:25:05.500 INFO:tasks.workunit.client.0.smithi090.stdout:6/632: write d8/dc/f10 [4562311,77245] 0 2024-04-24T08:25:05.501 INFO:tasks.workunit.client.0.smithi090.stdout:2/655: dwrite d0/fb0 [0,4194304] 0 2024-04-24T08:25:05.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/552: dread - d2/da/d13/d64/d50/fb7 zero size 2024-04-24T08:25:05.506 INFO:tasks.workunit.client.0.smithi090.stdout:9/564: dwrite d0/d19/f25 [4194304,4194304] 0 2024-04-24T08:25:05.510 INFO:tasks.workunit.client.0.smithi090.stdout:0/592: dread f8 [0,4194304] 0 2024-04-24T08:25:05.512 INFO:tasks.workunit.client.0.smithi090.stdout:9/565: truncate d0/dc/d24/d7c/f86 199953 0 2024-04-24T08:25:05.514 INFO:tasks.workunit.client.0.smithi090.stdout:9/566: chown d0/dc/d24/d7c/d4d/f52 93308415 1 2024-04-24T08:25:05.516 INFO:tasks.workunit.client.0.smithi090.stdout:8/553: symlink d2/da/d13/d64/d24/db3/lbe 0 2024-04-24T08:25:05.517 INFO:tasks.workunit.client.1.smithi149.stdout:1/937: symlink d0/ddf/da0/db8/dc5/l130 0 2024-04-24T08:25:05.518 INFO:tasks.workunit.client.0.smithi090.stdout:0/593: fsync d1a/f2c 0 2024-04-24T08:25:05.519 INFO:tasks.workunit.client.0.smithi090.stdout:7/646: dread db/d17/f2e [0,4194304] 0 2024-04-24T08:25:05.521 INFO:tasks.workunit.client.1.smithi149.stdout:0/812: mknod d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/c103 0 2024-04-24T08:25:05.523 INFO:tasks.workunit.client.0.smithi090.stdout:6/633: rename d8/cd to d8/dc/d19/cbf 0 2024-04-24T08:25:05.526 INFO:tasks.workunit.client.0.smithi090.stdout:9/567: creat d0/dc/d24/d2d/d49/d62/fad x:0 0 0 2024-04-24T08:25:05.530 INFO:tasks.workunit.client.0.smithi090.stdout:9/568: stat d0/dc/d24/d7c/d4d/d32/d9f/d78 0 2024-04-24T08:25:05.531 INFO:tasks.workunit.client.1.smithi149.stdout:8/839: rename d1/d1f/f50 to d1/d1f/d22/f11a 0 2024-04-24T08:25:05.533 INFO:tasks.workunit.client.0.smithi090.stdout:9/569: chown d0/dc/d24/d7c/l76 139758 1 2024-04-24T08:25:05.535 INFO:tasks.workunit.client.1.smithi149.stdout:4/850: truncate d2/d10/f31 2311426 0 2024-04-24T08:25:05.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/507: dwrite d3/d18/d1d/f46 [4194304,4194304] 0 2024-04-24T08:25:05.537 INFO:tasks.workunit.client.0.smithi090.stdout:7/647: dwrite db/d17/d29/d32/f61 [0,4194304] 0 2024-04-24T08:25:05.537 INFO:tasks.workunit.client.0.smithi090.stdout:4/508: fdatasync d3/d18/d5c/f74 0 2024-04-24T08:25:05.538 INFO:tasks.workunit.client.0.smithi090.stdout:0/594: creat d1a/d4b/d53/d79/d69/d8a/fbe x:0 0 0 2024-04-24T08:25:05.539 INFO:tasks.workunit.client.1.smithi149.stdout:1/938: dread d0/d5/db/d63/f6b [0,4194304] 0 2024-04-24T08:25:05.539 INFO:tasks.workunit.client.0.smithi090.stdout:0/595: fdatasync d1a/f1d 0 2024-04-24T08:25:05.540 INFO:tasks.workunit.client.0.smithi090.stdout:7/648: write db/dbd/fd0 [68615,51044] 0 2024-04-24T08:25:05.541 INFO:tasks.workunit.client.0.smithi090.stdout:3/530: truncate d2/df/d1a/d37/d53/f65 2810256 0 2024-04-24T08:25:05.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/509: stat d3/d38/d6b/c6c 0 2024-04-24T08:25:05.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/554: rename d2/c9b to d2/da/d13/d64/d24/db3/db4/cbf 0 2024-04-24T08:25:05.543 INFO:tasks.workunit.client.0.smithi090.stdout:0/596: chown d1a/d4b/d53/d79/d69/d90 7508 1 2024-04-24T08:25:05.555 INFO:tasks.workunit.client.1.smithi149.stdout:6/775: dwrite d0/d1a/d28/d31/d4c/d8a/fe5 [0,4194304] 0 2024-04-24T08:25:05.560 INFO:tasks.workunit.client.1.smithi149.stdout:4/851: fsync d2/d24/d7f/ffb 0 2024-04-24T08:25:05.560 INFO:tasks.workunit.client.0.smithi090.stdout:6/634: rename d8/d1d/d2f/d85/fb3 to d8/d1d/d1f/d3e/d44/fc0 0 2024-04-24T08:25:05.561 INFO:tasks.workunit.client.1.smithi149.stdout:3/961: getdents d0/d5/da/d59/d12d 0 2024-04-24T08:25:05.562 INFO:tasks.workunit.client.0.smithi090.stdout:3/531: symlink d2/df/d57/lb7 0 2024-04-24T08:25:05.563 INFO:tasks.workunit.client.1.smithi149.stdout:4/852: truncate d2/f8a 282717 0 2024-04-24T08:25:05.572 INFO:tasks.workunit.client.1.smithi149.stdout:8/840: creat d1/de/d18/d32/dde/f11b x:0 0 0 2024-04-24T08:25:05.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/555: dread d2/da/d13/d64/f8e [0,4194304] 0 2024-04-24T08:25:05.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/649: creat db/d17/d29/dc0/fe1 x:0 0 0 2024-04-24T08:25:05.577 INFO:tasks.workunit.client.1.smithi149.stdout:1/939: symlink d0/d1b/l131 0 2024-04-24T08:25:05.579 INFO:tasks.workunit.client.1.smithi149.stdout:1/940: readlink d0/d1b/db5/d108/lba 0 2024-04-24T08:25:05.579 INFO:tasks.workunit.client.0.smithi090.stdout:7/650: truncate db/d17/d29/d32/d5a/d75/f79 837194 0 2024-04-24T08:25:05.581 INFO:tasks.workunit.client.0.smithi090.stdout:3/532: write d2/d15/d36/d6f/d92/fac [894226,60456] 0 2024-04-24T08:25:05.584 INFO:tasks.workunit.client.1.smithi149.stdout:6/776: rename d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/lb0 to d0/d1a/d28/d31/d4c/d4e/d62/d7a/l109 0 2024-04-24T08:25:05.584 INFO:tasks.workunit.client.0.smithi090.stdout:4/510: truncate d3/d38/f4d 486998 0 2024-04-24T08:25:05.586 INFO:tasks.workunit.client.1.smithi149.stdout:6/777: chown d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/laf 92102 1 2024-04-24T08:25:05.587 INFO:tasks.workunit.client.0.smithi090.stdout:9/570: link d0/dc/d24/d7c/d4d/l2b d0/dc/d24/d7c/d4d/d32/d9f/lae 0 2024-04-24T08:25:05.592 INFO:tasks.workunit.client.1.smithi149.stdout:1/941: creat d0/d5/db/d18/d4d/f132 x:0 0 0 2024-04-24T08:25:05.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/651: dwrite db/d17/d39/d86/dae/fc7 [0,4194304] 0 2024-04-24T08:25:05.593 INFO:tasks.workunit.client.0.smithi090.stdout:4/511: unlink d3/d38/d49/f63 0 2024-04-24T08:25:05.596 INFO:tasks.workunit.client.0.smithi090.stdout:8/556: creat d2/da/d13/d64/db2/dbb/fc0 x:0 0 0 2024-04-24T08:25:05.596 INFO:tasks.workunit.client.0.smithi090.stdout:4/512: truncate d3/df/f30 2000358 0 2024-04-24T08:25:05.597 INFO:tasks.workunit.client.1.smithi149.stdout:9/916: dwrite d1/da/d13/fd1 [0,4194304] 0 2024-04-24T08:25:05.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/841: mknod d1/de/c11c 0 2024-04-24T08:25:05.613 INFO:tasks.workunit.client.0.smithi090.stdout:2/656: truncate d0/d7/d67/d7f/d31/f41 8287405 0 2024-04-24T08:25:05.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/571: dread - d0/dc/d24/d2d/f80 zero size 2024-04-24T08:25:05.615 INFO:tasks.workunit.client.1.smithi149.stdout:3/962: rmdir d0/d5/da/d59/d12d 0 2024-04-24T08:25:05.619 INFO:tasks.workunit.client.1.smithi149.stdout:4/853: creat d2/d24/d2e/d3f/d94/dfd/f10a x:0 0 0 2024-04-24T08:25:05.620 INFO:tasks.workunit.client.0.smithi090.stdout:5/643: sync 2024-04-24T08:25:05.621 INFO:tasks.workunit.client.1.smithi149.stdout:1/942: rename d0/d5/db/d18/d2f/ld2 to d0/d5/db/d18/d2f/l133 0 2024-04-24T08:25:05.622 INFO:tasks.workunit.client.1.smithi149.stdout:3/963: read d0/d5/da/d19/d1b/d72/f107 [2370712,61865] 0 2024-04-24T08:25:05.624 INFO:tasks.workunit.client.1.smithi149.stdout:7/902: truncate d9/d43/f7e 4076258 0 2024-04-24T08:25:05.625 INFO:tasks.workunit.client.0.smithi090.stdout:7/652: truncate db/d17/d29/d38/f53 4411281 0 2024-04-24T08:25:05.626 INFO:tasks.workunit.client.1.smithi149.stdout:8/842: mknod d1/de/d18/dd3/c11d 0 2024-04-24T08:25:05.629 INFO:tasks.workunit.client.0.smithi090.stdout:8/557: mknod d2/d62/d71/cc1 0 2024-04-24T08:25:05.633 INFO:tasks.workunit.client.1.smithi149.stdout:1/943: creat d0/d5/db/d18/d4d/d50/f134 x:0 0 0 2024-04-24T08:25:05.635 INFO:tasks.workunit.client.1.smithi149.stdout:0/813: write d2/d9/d1a/d36/d85/fe5 [377146,66615] 0 2024-04-24T08:25:05.636 INFO:tasks.workunit.client.0.smithi090.stdout:5/644: dwrite d8/d25/f3d [0,4194304] 0 2024-04-24T08:25:05.637 INFO:tasks.workunit.client.1.smithi149.stdout:9/917: fsync d1/da/d13/d74/dbb/fc5 0 2024-04-24T08:25:05.642 INFO:tasks.workunit.client.0.smithi090.stdout:7/653: truncate db/d17/d29/d38/d3e/f58 4523187 0 2024-04-24T08:25:05.644 INFO:tasks.workunit.client.0.smithi090.stdout:7/654: chown db/dbd/fe0 105009 1 2024-04-24T08:25:05.644 INFO:tasks.workunit.client.0.smithi090.stdout:8/558: fdatasync d2/da/d13/d64/d24/f5b 0 2024-04-24T08:25:05.646 INFO:tasks.workunit.client.0.smithi090.stdout:8/559: chown d2/d62/d6b/d9c 30 1 2024-04-24T08:25:05.646 INFO:tasks.workunit.client.0.smithi090.stdout:7/655: readlink db/d17/d29/d38/d3e/la1 0 2024-04-24T08:25:05.647 INFO:tasks.workunit.client.0.smithi090.stdout:0/597: truncate d1a/d4b/d53/f6a 638277 0 2024-04-24T08:25:05.649 INFO:tasks.workunit.client.0.smithi090.stdout:2/657: fsync d0/d7/fd 0 2024-04-24T08:25:05.649 INFO:tasks.workunit.client.0.smithi090.stdout:3/533: write d2/df/d1a/f4c [1723278,52469] 0 2024-04-24T08:25:05.652 INFO:tasks.workunit.client.0.smithi090.stdout:5/645: creat d8/d76/dad/fd1 x:0 0 0 2024-04-24T08:25:05.652 INFO:tasks.workunit.client.0.smithi090.stdout:5/646: chown f7 1 1 2024-04-24T08:25:05.655 INFO:tasks.workunit.client.1.smithi149.stdout:0/814: mknod d2/d9/d1a/d36/dcc/c104 0 2024-04-24T08:25:05.658 INFO:tasks.workunit.client.0.smithi090.stdout:4/513: creat d3/df/f98 x:0 0 0 2024-04-24T08:25:05.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/815: truncate d2/d9/df/fec 1964145 0 2024-04-24T08:25:05.659 INFO:tasks.workunit.client.1.smithi149.stdout:9/918: dwrite d1/d2/d33/d76/f7c [0,4194304] 0 2024-04-24T08:25:05.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/656: write db/f11 [1208103,5529] 0 2024-04-24T08:25:05.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/560: chown d2/da/d13/d64/c5a 61686 1 2024-04-24T08:25:05.668 INFO:tasks.workunit.client.1.smithi149.stdout:0/816: dread d2/d9/df/d2f/d6c/f77 [0,4194304] 0 2024-04-24T08:25:05.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/561: write d2/d62/d6b/d9c/fa1 [3079702,109073] 0 2024-04-24T08:25:05.672 INFO:tasks.workunit.client.0.smithi090.stdout:2/658: write d0/d7/d6f/f29 [2134769,4510] 0 2024-04-24T08:25:05.673 INFO:tasks.workunit.client.1.smithi149.stdout:9/919: creat d1/d2/d33/d5d/d83/dbf/f151 x:0 0 0 2024-04-24T08:25:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/534: dread - d2/d25/d33/f81 zero size 2024-04-24T08:25:05.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/647: symlink d8/da/d14/ld2 0 2024-04-24T08:25:05.681 INFO:tasks.workunit.client.0.smithi090.stdout:4/514: dwrite d3/d38/d49/d6f/f7d [0,4194304] 0 2024-04-24T08:25:05.683 INFO:tasks.workunit.client.0.smithi090.stdout:5/648: dread - d8/da/d3a/d2d/d37/f63 zero size 2024-04-24T08:25:05.684 INFO:tasks.workunit.client.0.smithi090.stdout:5/649: chown d8/da/d14 798333461 1 2024-04-24T08:25:05.685 INFO:tasks.workunit.client.0.smithi090.stdout:4/515: chown d3/d18/d5c/f3d 107 1 2024-04-24T08:25:05.689 INFO:tasks.workunit.client.0.smithi090.stdout:8/562: fdatasync d2/da/d13/d64/f8e 0 2024-04-24T08:25:05.690 INFO:tasks.workunit.client.1.smithi149.stdout:4/854: dwrite d2/d18/d1e/f33 [0,4194304] 0 2024-04-24T08:25:05.691 INFO:tasks.workunit.client.1.smithi149.stdout:0/817: dwrite d2/d9/d1a/d36/d85/dcd/fee [0,4194304] 0 2024-04-24T08:25:05.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/903: getdents d9/d43 0 2024-04-24T08:25:05.703 INFO:tasks.workunit.client.1.smithi149.stdout:3/964: truncate d0/d5/ff 1672222 0 2024-04-24T08:25:05.708 INFO:tasks.workunit.client.0.smithi090.stdout:2/659: creat d0/d7/d67/d7f/d31/d46/d60/fdf x:0 0 0 2024-04-24T08:25:05.720 INFO:tasks.workunit.client.1.smithi149.stdout:1/944: dwrite d0/ddf/feb [0,4194304] 0 2024-04-24T08:25:05.723 INFO:tasks.workunit.client.0.smithi090.stdout:0/598: write f18 [4488759,706] 0 2024-04-24T08:25:05.730 INFO:tasks.workunit.client.0.smithi090.stdout:2/660: creat d0/d7/d67/d7f/d31/d46/d60/fe0 x:0 0 0 2024-04-24T08:25:05.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/661: write d0/d14/f1d [4492110,100441] 0 2024-04-24T08:25:05.741 INFO:tasks.workunit.client.0.smithi090.stdout:7/657: dwrite db/d17/d29/d38/f44 [0,4194304] 0 2024-04-24T08:25:05.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/599: symlink d1a/d4b/d53/d57/lbf 0 2024-04-24T08:25:05.746 INFO:tasks.workunit.client.0.smithi090.stdout:1/603: sync 2024-04-24T08:25:05.746 INFO:tasks.workunit.client.1.smithi149.stdout:0/818: dread d2/d9/df/d2f/d6c/dc1/fa9 [0,4194304] 0 2024-04-24T08:25:05.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/604: fdatasync d3/d5/da/fc1 0 2024-04-24T08:25:05.755 INFO:tasks.workunit.client.1.smithi149.stdout:3/965: getdents d0/d5/d53 0 2024-04-24T08:25:05.755 INFO:tasks.workunit.client.1.smithi149.stdout:7/904: write d9/f8b [2338723,40318] 0 2024-04-24T08:25:05.755 INFO:tasks.workunit.client.0.smithi090.stdout:7/658: unlink db/d17/d39/d59/cd3 0 2024-04-24T08:25:05.757 INFO:tasks.workunit.client.0.smithi090.stdout:7/659: stat db/d17/d29 0 2024-04-24T08:25:05.758 INFO:tasks.workunit.client.1.smithi149.stdout:9/920: dwrite d1/da/dd/fac [0,4194304] 0 2024-04-24T08:25:05.758 INFO:tasks.workunit.client.1.smithi149.stdout:4/855: dread d2/d10/f27 [0,4194304] 0 2024-04-24T08:25:05.759 INFO:tasks.workunit.client.0.smithi090.stdout:8/563: link d2/da/d13/d34/f72 d2/da/d13/d64/d24/db3/fc2 0 2024-04-24T08:25:05.759 INFO:tasks.workunit.client.0.smithi090.stdout:3/535: dread d2/df/d57/d4f/f86 [0,4194304] 0 2024-04-24T08:25:05.759 INFO:tasks.workunit.client.0.smithi090.stdout:5/650: dwrite d8/faf [0,4194304] 0 2024-04-24T08:25:05.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/605: mkdir d3/d1d/d34/d73/dd8 0 2024-04-24T08:25:05.764 INFO:tasks.workunit.client.0.smithi090.stdout:5/651: fsync d8/d59/dc7/d7a/d50/f68 0 2024-04-24T08:25:05.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/905: read d9/fe5 [3091247,18412] 0 2024-04-24T08:25:05.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/600: dread d1a/d4b/f4d [0,4194304] 0 2024-04-24T08:25:05.768 INFO:tasks.workunit.client.0.smithi090.stdout:5/652: chown d8/d59/dc7/d7a/d50/lb1 1897 1 2024-04-24T08:25:05.774 INFO:tasks.workunit.client.0.smithi090.stdout:0/601: read d1a/d4b/d53/f7f [3112588,29431] 0 2024-04-24T08:25:05.778 INFO:tasks.workunit.client.1.smithi149.stdout:1/945: dwrite d0/ddf/da0/fe3 [0,4194304] 0 2024-04-24T08:25:05.779 INFO:tasks.workunit.client.0.smithi090.stdout:0/602: chown d1a/d4b/d53/d79/d69/d8a/fa2 872 1 2024-04-24T08:25:05.783 INFO:tasks.workunit.client.0.smithi090.stdout:0/603: write d1a/d4b/d53/d79/d35/d68/f76 [2488095,66450] 0 2024-04-24T08:25:05.795 INFO:tasks.workunit.client.0.smithi090.stdout:5/653: dwrite d8/d9c/fc5 [0,4194304] 0 2024-04-24T08:25:05.801 INFO:tasks.workunit.client.0.smithi090.stdout:5/654: read f7 [3385818,125846] 0 2024-04-24T08:25:05.801 INFO:tasks.workunit.client.0.smithi090.stdout:7/660: dread db/d17/d39/f3c [0,4194304] 0 2024-04-24T08:25:05.803 INFO:tasks.workunit.client.1.smithi149.stdout:3/966: dread d0/d5/da/d19/d1b/d72/fcd [0,4194304] 0 2024-04-24T08:25:05.807 INFO:tasks.workunit.client.1.smithi149.stdout:3/967: write d0/f11 [3217270,110996] 0 2024-04-24T08:25:05.808 INFO:tasks.workunit.client.0.smithi090.stdout:6/635: sync 2024-04-24T08:25:05.808 INFO:tasks.workunit.client.0.smithi090.stdout:9/572: sync 2024-04-24T08:25:05.808 INFO:tasks.workunit.client.0.smithi090.stdout:6/636: readlink d8/l9d 0 2024-04-24T08:25:05.814 INFO:tasks.workunit.client.1.smithi149.stdout:3/968: write d0/d5/da/d19/d1b/d133/d38/f50 [7183447,109451] 0 2024-04-24T08:25:05.817 INFO:tasks.workunit.client.0.smithi090.stdout:4/516: truncate d3/d38/f4d 708727 0 2024-04-24T08:25:05.819 INFO:tasks.workunit.client.0.smithi090.stdout:4/517: readlink d3/df/d69/l88 0 2024-04-24T08:25:05.823 INFO:tasks.workunit.client.0.smithi090.stdout:7/661: dread db/d17/d29/d38/f44 [0,4194304] 0 2024-04-24T08:25:05.823 INFO:tasks.workunit.client.0.smithi090.stdout:9/573: dread d0/f18 [0,4194304] 0 2024-04-24T08:25:05.842 INFO:tasks.workunit.client.0.smithi090.stdout:7/662: dwrite db/d17/d29/d32/d5a/d75/f81 [0,4194304] 0 2024-04-24T08:25:05.895 INFO:tasks.workunit.client.1.smithi149.stdout:2/773: sync 2024-04-24T08:25:05.897 INFO:tasks.workunit.client.0.smithi090.stdout:8/564: creat d2/da/d13/d34/fc3 x:0 0 0 2024-04-24T08:25:05.899 INFO:tasks.workunit.client.1.smithi149.stdout:4/856: mknod d2/d18/da4/c10b 0 2024-04-24T08:25:05.911 INFO:tasks.workunit.client.1.smithi149.stdout:1/946: fdatasync d0/d1b/d2a/f115 0 2024-04-24T08:25:05.913 INFO:tasks.workunit.client.1.smithi149.stdout:1/947: readlink d0/d1b/db5/d108/dd3/lf6 0 2024-04-24T08:25:05.914 INFO:tasks.workunit.client.1.smithi149.stdout:9/921: dwrite d1/de/d9d/dfd/f101 [0,4194304] 0 2024-04-24T08:25:05.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/774: creat d0/d6/feb x:0 0 0 2024-04-24T08:25:05.943 INFO:tasks.workunit.client.0.smithi090.stdout:6/637: unlink d8/d1d/d29/f3b 0 2024-04-24T08:25:05.948 INFO:tasks.workunit.client.1.smithi149.stdout:7/906: creat d9/dc/d11/d83/da6/d67/d69/d11c/f12a x:0 0 0 2024-04-24T08:25:05.957 INFO:tasks.workunit.client.1.smithi149.stdout:9/922: truncate d1/d2/d33/f80 5062750 0 2024-04-24T08:25:05.958 INFO:tasks.workunit.client.0.smithi090.stdout:8/565: dread - d2/da/d13/d64/f6d zero size 2024-04-24T08:25:05.959 INFO:tasks.workunit.client.1.smithi149.stdout:0/819: link d2/l3 d2/d9/l105 0 2024-04-24T08:25:05.961 INFO:tasks.workunit.client.1.smithi149.stdout:0/820: chown d2/d9/df/d40/lc8 105 1 2024-04-24T08:25:05.966 INFO:tasks.workunit.client.1.smithi149.stdout:7/907: chown d9/dc/f6b 1950128172 1 2024-04-24T08:25:05.971 INFO:tasks.workunit.client.1.smithi149.stdout:0/821: readlink d2/d9/d1a/d36/d85/db5/lf5 0 2024-04-24T08:25:05.972 INFO:tasks.workunit.client.1.smithi149.stdout:0/822: chown d2/d17 18 1 2024-04-24T08:25:05.973 INFO:tasks.workunit.client.1.smithi149.stdout:4/857: link d2/d18/db9/dd8/l53 d2/da8/l10c 0 2024-04-24T08:25:05.974 INFO:tasks.workunit.client.0.smithi090.stdout:7/663: creat db/d54/db1/fe2 x:0 0 0 2024-04-24T08:25:05.975 INFO:tasks.workunit.client.1.smithi149.stdout:2/775: creat d0/d6/db/d28/d7f/d8f/db9/ddf/de3/fec x:0 0 0 2024-04-24T08:25:05.977 INFO:tasks.workunit.client.1.smithi149.stdout:7/908: creat d9/dc/d7b/dfc/f12b x:0 0 0 2024-04-24T08:25:05.979 INFO:tasks.workunit.client.1.smithi149.stdout:9/923: mknod d1/de/c152 0 2024-04-24T08:25:05.981 INFO:tasks.workunit.client.1.smithi149.stdout:0/823: mkdir d2/d9/d1a/d36/d85/db5/d106 0 2024-04-24T08:25:05.983 INFO:tasks.workunit.client.1.smithi149.stdout:4/858: mknod d2/d10/d1a/c10d 0 2024-04-24T08:25:05.983 INFO:tasks.workunit.client.1.smithi149.stdout:0/824: read - d2/d9/df/d48/f52 zero size 2024-04-24T08:25:05.987 INFO:tasks.workunit.client.1.smithi149.stdout:2/776: creat d0/d6/db/d39/d3d/d44/d57/fed x:0 0 0 2024-04-24T08:25:05.990 INFO:tasks.workunit.client.1.smithi149.stdout:2/777: write d0/d6/db/d39/d78/fe9 [4920,103487] 0 2024-04-24T08:25:05.990 INFO:tasks.workunit.client.0.smithi090.stdout:4/518: rmdir d3/df/d69/d8b/d91 0 2024-04-24T08:25:05.992 INFO:tasks.workunit.client.0.smithi090.stdout:4/519: readlink d3/df/l31 0 2024-04-24T08:25:05.993 INFO:tasks.workunit.client.1.smithi149.stdout:7/909: creat d9/dc/d11/d83/da6/d67/d69/d123/f12c x:0 0 0 2024-04-24T08:25:05.995 INFO:tasks.workunit.client.0.smithi090.stdout:7/664: creat db/d17/d29/d38/d3e/d46/fe3 x:0 0 0 2024-04-24T08:25:05.998 INFO:tasks.workunit.client.1.smithi149.stdout:6/778: sync 2024-04-24T08:25:06.002 INFO:tasks.workunit.client.1.smithi149.stdout:6/779: chown d0/d1a/d28/d31/d4c/d80/d6f/fcb 871136097 1 2024-04-24T08:25:06.009 INFO:tasks.workunit.client.0.smithi090.stdout:7/665: creat db/d17/d29/d32/fe4 x:0 0 0 2024-04-24T08:25:06.010 INFO:tasks.workunit.client.1.smithi149.stdout:7/910: dwrite d9/dc/d7b/fc1 [0,4194304] 0 2024-04-24T08:25:06.011 INFO:tasks.workunit.client.0.smithi090.stdout:7/666: chown db/d17/d39/fa2 2954129 1 2024-04-24T08:25:06.012 INFO:tasks.workunit.client.0.smithi090.stdout:4/520: dread d3/d38/f4d [0,4194304] 0 2024-04-24T08:25:06.014 INFO:tasks.workunit.client.0.smithi090.stdout:7/667: write db/dbd/fe0 [966057,115867] 0 2024-04-24T08:25:06.015 INFO:tasks.workunit.client.0.smithi090.stdout:7/668: chown db/d17 83 1 2024-04-24T08:25:06.016 INFO:tasks.workunit.client.0.smithi090.stdout:7/669: readlink db/d17/l2b 0 2024-04-24T08:25:06.019 INFO:tasks.workunit.client.0.smithi090.stdout:4/521: symlink d3/d38/d49/d6f/l99 0 2024-04-24T08:25:06.023 INFO:tasks.workunit.client.1.smithi149.stdout:6/780: truncate d0/d1a/d28/d31/d33/fa8 751920 0 2024-04-24T08:25:06.024 INFO:tasks.workunit.client.1.smithi149.stdout:0/825: dread d2/d3c/f61 [0,4194304] 0 2024-04-24T08:25:06.026 INFO:tasks.workunit.client.1.smithi149.stdout:6/781: stat d0/d1a/d28/l98 0 2024-04-24T08:25:06.030 INFO:tasks.workunit.client.1.smithi149.stdout:1/948: symlink d0/d1b/db5/l135 0 2024-04-24T08:25:06.032 INFO:tasks.workunit.client.1.smithi149.stdout:6/782: dread d0/d1a/d28/d31/d4c/d4e/d62/fbc [0,4194304] 0 2024-04-24T08:25:06.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/778: getdents d0/d6/db/d28/d7f 0 2024-04-24T08:25:06.037 INFO:tasks.workunit.client.0.smithi090.stdout:4/522: creat d3/df/f9a x:0 0 0 2024-04-24T08:25:06.037 INFO:tasks.workunit.client.1.smithi149.stdout:2/779: readlink d0/d6/db/l32 0 2024-04-24T08:25:06.043 INFO:tasks.workunit.client.0.smithi090.stdout:4/523: creat d3/d24/d92/f9b x:0 0 0 2024-04-24T08:25:06.046 INFO:tasks.workunit.client.1.smithi149.stdout:1/949: getdents d0/d5/d4f/d60 0 2024-04-24T08:25:06.047 INFO:tasks.workunit.client.1.smithi149.stdout:6/783: rmdir d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/d107 0 2024-04-24T08:25:06.049 INFO:tasks.workunit.client.1.smithi149.stdout:6/784: chown d0/d1a/d28/d31/d33/cd6 0 1 2024-04-24T08:25:06.065 INFO:tasks.workunit.client.0.smithi090.stdout:3/536: write d2/df/d1a/f28 [8048651,115827] 0 2024-04-24T08:25:06.069 INFO:tasks.workunit.client.0.smithi090.stdout:3/537: fdatasync d2/df/d1a/d37/f47 0 2024-04-24T08:25:06.072 INFO:tasks.workunit.client.1.smithi149.stdout:4/859: link d2/d18/d1e/fcd d2/d10/f10e 0 2024-04-24T08:25:06.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/662: rename d0/d2c to d0/d7/de1 0 2024-04-24T08:25:06.079 INFO:tasks.workunit.client.0.smithi090.stdout:2/663: chown d0/fc8 19662 1 2024-04-24T08:25:06.082 INFO:tasks.workunit.client.0.smithi090.stdout:0/604: rename d1a/d4b/d53/d79/d3c/c4a to d1a/d4b/d53/d79/d69/d90/cc0 0 2024-04-24T08:25:06.087 INFO:tasks.workunit.client.0.smithi090.stdout:3/538: rename d2/df/d57/l40 to d2/d25/d33/lb8 0 2024-04-24T08:25:06.088 INFO:tasks.workunit.client.0.smithi090.stdout:0/605: mknod d1a/d4b/d53/d79/d69/d8a/cc1 0 2024-04-24T08:25:06.090 INFO:tasks.workunit.client.1.smithi149.stdout:8/843: sync 2024-04-24T08:25:06.091 INFO:tasks.workunit.client.0.smithi090.stdout:3/539: symlink d2/d25/lb9 0 2024-04-24T08:25:06.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/844: read d1/de/d18/fb7 [301902,47781] 0 2024-04-24T08:25:06.093 INFO:tasks.workunit.client.0.smithi090.stdout:3/540: chown d2/df/d1a/l69 14527061 1 2024-04-24T08:25:06.096 INFO:tasks.workunit.client.1.smithi149.stdout:3/969: write d0/d5/ff [1642844,83958] 0 2024-04-24T08:25:06.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/845: truncate d1/d1f/d22/f5f 1252654 0 2024-04-24T08:25:06.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/574: dwrite d0/f5 [4194304,4194304] 0 2024-04-24T08:25:06.103 INFO:tasks.workunit.client.0.smithi090.stdout:3/541: symlink d2/d15/d36/d6f/d9d/lba 0 2024-04-24T08:25:06.104 INFO:tasks.workunit.client.0.smithi090.stdout:3/542: stat d2/df/d1a/d37 0 2024-04-24T08:25:06.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/575: mknod d0/d69/d7d/d5b/d92/caf 0 2024-04-24T08:25:06.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/655: write d8/d59/dc7/d7a/d50/f57 [4988228,98175] 0 2024-04-24T08:25:06.112 INFO:tasks.workunit.client.0.smithi090.stdout:1/606: dwrite d3/d5/f32 [0,4194304] 0 2024-04-24T08:25:06.114 INFO:tasks.workunit.client.0.smithi090.stdout:1/607: stat d3/d5/da/dc/d80/fbc 0 2024-04-24T08:25:06.117 INFO:tasks.workunit.client.0.smithi090.stdout:9/576: creat d0/dc/d24/d7c/fb0 x:0 0 0 2024-04-24T08:25:06.118 INFO:tasks.workunit.client.0.smithi090.stdout:3/543: rename d2/d15/d36/d6f/d7a/c7e to d2/d15/d36/d6f/cbb 0 2024-04-24T08:25:06.122 INFO:tasks.workunit.client.0.smithi090.stdout:1/608: creat d3/d5/d58/fd9 x:0 0 0 2024-04-24T08:25:06.125 INFO:tasks.workunit.client.0.smithi090.stdout:6/638: unlink d8/d1d/d1f/d3e/l75 0 2024-04-24T08:25:06.129 INFO:tasks.workunit.client.0.smithi090.stdout:1/609: creat d3/d68/d7f/fda x:0 0 0 2024-04-24T08:25:06.133 INFO:tasks.workunit.client.1.smithi149.stdout:9/924: dwrite d1/d4c/f54 [0,4194304] 0 2024-04-24T08:25:06.134 INFO:tasks.workunit.client.0.smithi090.stdout:1/610: creat d3/d25/d30/dc5/fdb x:0 0 0 2024-04-24T08:25:06.134 INFO:tasks.workunit.client.0.smithi090.stdout:7/670: write db/d17/d29/d38/f53 [5064040,8820] 0 2024-04-24T08:25:06.139 INFO:tasks.workunit.client.0.smithi090.stdout:8/566: dwrite d2/da/d13/d64/d24/f21 [0,4194304] 0 2024-04-24T08:25:06.139 INFO:tasks.workunit.client.0.smithi090.stdout:9/577: rename d0/dc/d24/d2d/c55 to d0/d69/d7d/cb1 0 2024-04-24T08:25:06.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/611: chown d3/d1d/d34/d73/f99 40542521 1 2024-04-24T08:25:06.147 INFO:tasks.workunit.client.0.smithi090.stdout:9/578: chown d0/d19/c9e 904026 1 2024-04-24T08:25:06.148 INFO:tasks.workunit.client.0.smithi090.stdout:8/567: mknod d2/d62/d83/cc4 0 2024-04-24T08:25:06.151 INFO:tasks.workunit.client.0.smithi090.stdout:1/612: read - d3/d5/d3e/f76 zero size 2024-04-24T08:25:06.153 INFO:tasks.workunit.client.0.smithi090.stdout:6/639: dwrite d8/dc/fad [0,4194304] 0 2024-04-24T08:25:06.154 INFO:tasks.workunit.client.0.smithi090.stdout:7/671: link db/f11 db/d17/d29/fe5 0 2024-04-24T08:25:06.158 INFO:tasks.workunit.client.0.smithi090.stdout:1/613: mknod d3/d1d/d34/d5a/cdc 0 2024-04-24T08:25:06.159 INFO:tasks.workunit.client.0.smithi090.stdout:5/656: dread d8/d25/fc8 [0,4194304] 0 2024-04-24T08:25:06.165 INFO:tasks.workunit.client.0.smithi090.stdout:1/614: creat d3/d5/da/d97/d9c/fdd x:0 0 0 2024-04-24T08:25:06.167 INFO:tasks.workunit.client.0.smithi090.stdout:5/657: fsync d8/d59/f61 0 2024-04-24T08:25:06.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/664: truncate d0/d7/d67/d7f/f78 481342 0 2024-04-24T08:25:06.172 INFO:tasks.workunit.client.0.smithi090.stdout:1/615: creat d3/d25/d30/d9d/da2/fde x:0 0 0 2024-04-24T08:25:06.173 INFO:tasks.workunit.client.0.smithi090.stdout:2/665: mkdir d0/d5e/de2 0 2024-04-24T08:25:06.175 INFO:tasks.workunit.client.0.smithi090.stdout:5/658: truncate d8/da/d3a/fbd 190610 0 2024-04-24T08:25:06.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/616: write d3/d68/f8f [1268302,54016] 0 2024-04-24T08:25:06.179 INFO:tasks.workunit.client.0.smithi090.stdout:9/579: dwrite d0/d69/d7d/f85 [0,4194304] 0 2024-04-24T08:25:06.180 INFO:tasks.workunit.client.0.smithi090.stdout:7/672: dwrite db/d54/d83/f8f [0,4194304] 0 2024-04-24T08:25:06.181 INFO:tasks.workunit.client.0.smithi090.stdout:2/666: read d0/d7/d28/fbf [10343,90425] 0 2024-04-24T08:25:06.188 INFO:tasks.workunit.client.0.smithi090.stdout:8/568: write d2/d62/d83/faf [3977201,2762] 0 2024-04-24T08:25:06.189 INFO:tasks.workunit.client.0.smithi090.stdout:7/673: rmdir db 39 2024-04-24T08:25:06.191 INFO:tasks.workunit.client.0.smithi090.stdout:6/640: write d8/dc/d19/f64 [3124144,53403] 0 2024-04-24T08:25:06.192 INFO:tasks.workunit.client.0.smithi090.stdout:2/667: rename d0/d14/la6 to d0/d5e/le3 0 2024-04-24T08:25:06.196 INFO:tasks.workunit.client.0.smithi090.stdout:2/668: fsync d0/d14/f38 0 2024-04-24T08:25:06.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/606: dwrite d1a/d4b/d53/d79/d35/d62/f6e [0,4194304] 0 2024-04-24T08:25:06.200 INFO:tasks.workunit.client.0.smithi090.stdout:8/569: dread - d2/da/d13/d34/f66 zero size 2024-04-24T08:25:06.204 INFO:tasks.workunit.client.0.smithi090.stdout:3/544: dwrite d2/f2f [0,4194304] 0 2024-04-24T08:25:06.206 INFO:tasks.workunit.client.0.smithi090.stdout:5/659: write d8/da/d14/f17 [4841272,14590] 0 2024-04-24T08:25:06.206 INFO:tasks.workunit.client.0.smithi090.stdout:6/641: mknod d8/cc1 0 2024-04-24T08:25:06.207 INFO:tasks.workunit.client.0.smithi090.stdout:4/524: sync 2024-04-24T08:25:06.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/669: write d0/d7/d28/fbf [553586,135] 0 2024-04-24T08:25:06.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/607: symlink d1a/d4b/d53/d79/d58/db4/lc2 0 2024-04-24T08:25:06.214 INFO:tasks.workunit.client.0.smithi090.stdout:8/570: chown d2/da/d13/d64/db2/dbb/fc0 21734 1 2024-04-24T08:25:06.215 INFO:tasks.workunit.client.0.smithi090.stdout:6/642: fsync d8/dc/d19/d8e/fa6 0 2024-04-24T08:25:06.215 INFO:tasks.workunit.client.0.smithi090.stdout:7/674: creat db/d17/d29/d32/d5a/fe6 x:0 0 0 2024-04-24T08:25:06.219 INFO:tasks.workunit.client.0.smithi090.stdout:1/617: dread d3/f2e [0,4194304] 0 2024-04-24T08:25:06.220 INFO:tasks.workunit.client.0.smithi090.stdout:4/525: mknod d3/d38/d49/d6f/c9c 0 2024-04-24T08:25:06.220 INFO:tasks.workunit.client.0.smithi090.stdout:9/580: dwrite d0/d69/f9d [0,4194304] 0 2024-04-24T08:25:06.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/526: dread - d3/df/f9a zero size 2024-04-24T08:25:06.224 INFO:tasks.workunit.client.0.smithi090.stdout:1/618: write d3/d5/da/d97/fd7 [726394,28644] 0 2024-04-24T08:25:06.226 INFO:tasks.workunit.client.0.smithi090.stdout:7/675: creat db/d17/d29/d38/d3e/fe7 x:0 0 0 2024-04-24T08:25:06.229 INFO:tasks.workunit.client.0.smithi090.stdout:0/608: creat d1a/d4b/d53/d79/d35/d62/fc3 x:0 0 0 2024-04-24T08:25:06.230 INFO:tasks.workunit.client.0.smithi090.stdout:7/676: chown db/c5b 2673 1 2024-04-24T08:25:06.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/527: chown d3/l2e 44254 1 2024-04-24T08:25:06.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/581: rename d0/d69/d7d/d5b/fa6 to d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/fb2 0 2024-04-24T08:25:06.236 INFO:tasks.workunit.client.0.smithi090.stdout:1/619: mknod d3/d5/da/d97/d9c/dc4/cdf 0 2024-04-24T08:25:06.237 INFO:tasks.workunit.client.0.smithi090.stdout:8/571: creat d2/da/d13/fc5 x:0 0 0 2024-04-24T08:25:06.239 INFO:tasks.workunit.client.0.smithi090.stdout:4/528: symlink d3/d18/d5c/l9d 0 2024-04-24T08:25:06.240 INFO:tasks.workunit.client.0.smithi090.stdout:7/677: rmdir db/d17/d29/d32/d5a/d75 39 2024-04-24T08:25:06.242 INFO:tasks.workunit.client.0.smithi090.stdout:4/529: fsync d3/d18/f5b 0 2024-04-24T08:25:06.243 INFO:tasks.workunit.client.0.smithi090.stdout:1/620: rmdir d3/d68 39 2024-04-24T08:25:06.244 INFO:tasks.workunit.client.0.smithi090.stdout:9/582: write d0/dc/f1f [3513718,117386] 0 2024-04-24T08:25:06.244 INFO:tasks.workunit.client.0.smithi090.stdout:4/530: readlink d3/d18/d5c/l9d 0 2024-04-24T08:25:06.250 INFO:tasks.workunit.client.0.smithi090.stdout:1/621: creat d3/d25/d30/d9d/fe0 x:0 0 0 2024-04-24T08:25:06.253 INFO:tasks.workunit.client.0.smithi090.stdout:0/609: rename d1a/d4b/d53/d79/d69/d78/f99 to d1a/d4b/fc4 0 2024-04-24T08:25:06.254 INFO:tasks.workunit.client.0.smithi090.stdout:4/531: fsync d3/d38/d49/f4f 0 2024-04-24T08:25:06.255 INFO:tasks.workunit.client.0.smithi090.stdout:2/670: dwrite d0/d7/d67/d7f/d31/d46/d60/d65/fc0 [0,4194304] 0 2024-04-24T08:25:06.257 INFO:tasks.workunit.client.0.smithi090.stdout:7/678: creat db/d17/fe8 x:0 0 0 2024-04-24T08:25:06.261 INFO:tasks.workunit.client.0.smithi090.stdout:4/532: stat d3/c5e 0 2024-04-24T08:25:06.261 INFO:tasks.workunit.client.0.smithi090.stdout:6/643: truncate d8/dc/d19/d8e/d81/d96/d97/fae 3823850 0 2024-04-24T08:25:06.262 INFO:tasks.workunit.client.0.smithi090.stdout:9/583: creat d0/dc/d24/d2d/d49/fb3 x:0 0 0 2024-04-24T08:25:06.267 INFO:tasks.workunit.client.0.smithi090.stdout:4/533: fdatasync d3/d38/f85 0 2024-04-24T08:25:06.267 INFO:tasks.workunit.client.0.smithi090.stdout:3/545: dread d2/df/f3c [0,4194304] 0 2024-04-24T08:25:06.267 INFO:tasks.workunit.client.0.smithi090.stdout:2/671: creat d0/d14/d4d/db4/fe4 x:0 0 0 2024-04-24T08:25:06.272 INFO:tasks.workunit.client.0.smithi090.stdout:4/534: fdatasync d3/d38/d6b/f93 0 2024-04-24T08:25:06.274 INFO:tasks.workunit.client.0.smithi090.stdout:0/610: unlink d1a/d4b/d53/d79/d35/c36 0 2024-04-24T08:25:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:05 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:06.277 INFO:tasks.workunit.client.0.smithi090.stdout:6/644: mkdir d8/d1d/d1f/dc2 0 2024-04-24T08:25:06.277 INFO:tasks.workunit.client.0.smithi090.stdout:7/679: dread db/d17/d29/f8e [0,4194304] 0 2024-04-24T08:25:06.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/672: creat d0/d7/d28/daa/db1/fe5 x:0 0 0 2024-04-24T08:25:06.278 INFO:tasks.workunit.client.0.smithi090.stdout:0/611: readlink d1a/d4b/d53/d79/l87 0 2024-04-24T08:25:06.279 INFO:tasks.workunit.client.0.smithi090.stdout:3/546: readlink d2/d25/d33/l89 0 2024-04-24T08:25:06.286 INFO:tasks.workunit.client.0.smithi090.stdout:8/572: dwrite d2/f12 [0,4194304] 0 2024-04-24T08:25:06.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:05 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:06.295 INFO:tasks.workunit.client.0.smithi090.stdout:1/622: dwrite d3/d5/d58/f9f [0,4194304] 0 2024-04-24T08:25:06.296 INFO:tasks.workunit.client.0.smithi090.stdout:6/645: truncate d8/d1d/d1f/d3e/f6a 4743616 0 2024-04-24T08:25:06.297 INFO:tasks.workunit.client.0.smithi090.stdout:0/612: creat d1a/d4b/d53/d57/fc5 x:0 0 0 2024-04-24T08:25:06.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/547: mknod d2/df/d1a/d37/d53/d60/cbc 0 2024-04-24T08:25:06.305 INFO:tasks.workunit.client.0.smithi090.stdout:0/613: fsync d1a/d4b/fa0 0 2024-04-24T08:25:06.305 INFO:tasks.workunit.client.0.smithi090.stdout:3/548: chown d2/d84/f8d 61191379 1 2024-04-24T08:25:06.305 INFO:tasks.workunit.client.0.smithi090.stdout:6/646: mknod d8/d1d/d2f/cc3 0 2024-04-24T08:25:06.306 INFO:tasks.workunit.client.0.smithi090.stdout:4/535: getdents d3 0 2024-04-24T08:25:06.307 INFO:tasks.workunit.client.0.smithi090.stdout:9/584: dwrite d0/dc/d24/d7c/d4d/d32/d9f/f9b [0,4194304] 0 2024-04-24T08:25:06.309 INFO:tasks.workunit.client.0.smithi090.stdout:6/647: truncate d8/dc/fad 4454230 0 2024-04-24T08:25:06.310 INFO:tasks.workunit.client.0.smithi090.stdout:9/585: chown d0/d19/la7 0 1 2024-04-24T08:25:06.311 INFO:tasks.workunit.client.0.smithi090.stdout:4/536: chown d3/d8e/f97 1242121 1 2024-04-24T08:25:06.322 INFO:tasks.workunit.client.0.smithi090.stdout:0/614: read d1a/d4b/d53/d79/d35/f3e [71792,82124] 0 2024-04-24T08:25:06.325 INFO:tasks.workunit.client.0.smithi090.stdout:7/680: write db/d17/d29/d38/d3e/f58 [2925761,108389] 0 2024-04-24T08:25:06.332 INFO:tasks.workunit.client.0.smithi090.stdout:3/549: truncate d2/df/d21/f30 780674 0 2024-04-24T08:25:06.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/537: symlink d3/d25/l9e 0 2024-04-24T08:25:06.332 INFO:tasks.workunit.client.0.smithi090.stdout:9/586: mknod d0/d19/cb4 0 2024-04-24T08:25:06.332 INFO:tasks.workunit.client.0.smithi090.stdout:6/648: rmdir d8/d1d/d1f/d3e/d44 39 2024-04-24T08:25:06.335 INFO:tasks.workunit.client.0.smithi090.stdout:0/615: symlink d1a/d4b/d53/d79/d35/dbd/lc6 0 2024-04-24T08:25:06.337 INFO:tasks.workunit.client.0.smithi090.stdout:8/573: link d2/da/d13/c41 d2/da/d13/d64/db2/db6/cc6 0 2024-04-24T08:25:06.339 INFO:tasks.workunit.client.0.smithi090.stdout:0/616: chown d1a/d4b/d53/d57/fa4 0 1 2024-04-24T08:25:06.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/538: rename d3/d38/d6b/f64 to d3/d87/f9f 0 2024-04-24T08:25:06.340 INFO:tasks.workunit.client.0.smithi090.stdout:6/649: mknod d8/d1d/d1f/d3e/d4f/cc4 0 2024-04-24T08:25:06.344 INFO:tasks.workunit.client.0.smithi090.stdout:9/587: rename d0/dc/c22 to d0/dc/d24/d7c/d4d/d32/cb5 0 2024-04-24T08:25:06.345 INFO:tasks.workunit.client.0.smithi090.stdout:6/650: write d8/d1d/d1f/d3e/d4f/f98 [670588,75338] 0 2024-04-24T08:25:06.348 INFO:tasks.workunit.client.0.smithi090.stdout:4/539: fdatasync d3/d38/f4d 0 2024-04-24T08:25:06.352 INFO:tasks.workunit.client.0.smithi090.stdout:0/617: creat d1a/d4b/d53/d79/d35/fc7 x:0 0 0 2024-04-24T08:25:06.355 INFO:tasks.workunit.client.0.smithi090.stdout:1/623: dread d3/d5/da/dc/f87 [0,4194304] 0 2024-04-24T08:25:06.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/574: getdents d2/d97 0 2024-04-24T08:25:06.363 INFO:tasks.workunit.client.0.smithi090.stdout:0/618: mkdir d1a/d4b/d53/d57/dc8 0 2024-04-24T08:25:06.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/651: link d8/d1d/d2f/f3c d8/d1d/d2f/d85/fc5 0 2024-04-24T08:25:06.370 INFO:tasks.workunit.client.0.smithi090.stdout:4/540: dwrite d3/d18/d1d/f4e [0,4194304] 0 2024-04-24T08:25:06.374 INFO:tasks.workunit.client.1.smithi149.stdout:6/785: symlink d0/d1a/d28/d31/d4c/d80/d6f/l10a 0 2024-04-24T08:25:06.378 INFO:tasks.workunit.client.1.smithi149.stdout:4/860: write d2/d24/d2e/f66 [3633518,44060] 0 2024-04-24T08:25:06.379 INFO:tasks.workunit.client.1.smithi149.stdout:2/780: write d0/d6/db/d27/fab [615849,14226] 0 2024-04-24T08:25:06.379 INFO:tasks.workunit.client.1.smithi149.stdout:1/950: write d0/d5/db/d18/f6f [2054208,4117] 0 2024-04-24T08:25:06.382 INFO:tasks.workunit.client.0.smithi090.stdout:1/624: mknod d3/d5/da/ce1 0 2024-04-24T08:25:06.387 INFO:tasks.workunit.client.0.smithi090.stdout:6/652: mknod d8/d1d/d29/cc6 0 2024-04-24T08:25:06.387 INFO:tasks.workunit.client.0.smithi090.stdout:8/575: symlink d2/da/lc7 0 2024-04-24T08:25:06.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/846: dread d1/de/d18/d43/f88 [0,4194304] 0 2024-04-24T08:25:06.389 INFO:tasks.workunit.client.0.smithi090.stdout:9/588: dread d0/f1b [0,4194304] 0 2024-04-24T08:25:06.389 INFO:tasks.workunit.client.1.smithi149.stdout:7/911: dwrite d9/dc/d11/d83/da6/f5c [0,4194304] 0 2024-04-24T08:25:06.390 INFO:tasks.workunit.client.0.smithi090.stdout:3/550: write d2/d25/d33/f44 [3874875,76279] 0 2024-04-24T08:25:06.391 INFO:tasks.workunit.client.0.smithi090.stdout:9/589: chown d0/d19/c9e 267745 1 2024-04-24T08:25:06.394 INFO:tasks.workunit.client.1.smithi149.stdout:2/781: chown d0/d6/db/d39/d78/d8a/f99 63912 1 2024-04-24T08:25:06.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/625: creat d3/d5/da/d97/d9c/fe2 x:0 0 0 2024-04-24T08:25:06.397 INFO:tasks.workunit.client.1.smithi149.stdout:1/951: truncate d0/d5/db/d18/d4d/f132 803685 0 2024-04-24T08:25:06.397 INFO:tasks.workunit.client.0.smithi090.stdout:1/626: write d3/d5/d58/f9f [3025381,60501] 0 2024-04-24T08:25:06.401 INFO:tasks.workunit.client.0.smithi090.stdout:8/576: chown d2/da/d13/d64/d24/db3/fc2 147424 1 2024-04-24T08:25:06.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/619: symlink d1a/d4b/d53/d79/d3c/lc9 0 2024-04-24T08:25:06.408 INFO:tasks.workunit.client.1.smithi149.stdout:0/826: dwrite d2/d9/fe [0,4194304] 0 2024-04-24T08:25:06.409 INFO:tasks.workunit.client.0.smithi090.stdout:1/627: stat d3/d5/f33 0 2024-04-24T08:25:06.412 INFO:tasks.workunit.client.0.smithi090.stdout:4/541: link d3/d18/d5c/c5a d3/d25/ca0 0 2024-04-24T08:25:06.414 INFO:tasks.workunit.client.1.smithi149.stdout:9/925: dwrite d1/d2/d33/dea/d126/f105 [0,4194304] 0 2024-04-24T08:25:06.414 INFO:tasks.workunit.client.1.smithi149.stdout:4/861: symlink d2/d18/d1e/d3c/d59/l10f 0 2024-04-24T08:25:06.424 INFO:tasks.workunit.client.1.smithi149.stdout:7/912: creat d9/dc/d7b/f12d x:0 0 0 2024-04-24T08:25:06.429 INFO:tasks.workunit.client.0.smithi090.stdout:5/660: dread d8/da/d3a/d96/dcb/f1f [0,4194304] 0 2024-04-24T08:25:06.441 INFO:tasks.workunit.client.1.smithi149.stdout:7/913: truncate d9/d43/d4f/dc8/f129 93848 0 2024-04-24T08:25:06.441 INFO:tasks.workunit.client.1.smithi149.stdout:0/827: dwrite d2/d9/f13 [0,4194304] 0 2024-04-24T08:25:06.443 INFO:tasks.workunit.client.0.smithi090.stdout:6/653: write d8/dc/d19/d8e/f4b [1194101,7394] 0 2024-04-24T08:25:06.445 INFO:tasks.workunit.client.1.smithi149.stdout:7/914: chown d9/dc/d7b/laa 12176524 1 2024-04-24T08:25:06.454 INFO:tasks.workunit.client.0.smithi090.stdout:8/577: mknod d2/da/d13/d64/cc8 0 2024-04-24T08:25:06.457 INFO:tasks.workunit.client.0.smithi090.stdout:3/551: link d2/df/d1a/l69 d2/d25/d33/lbd 0 2024-04-24T08:25:06.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/620: mknod d1a/d4b/d53/d79/d35/db8/cca 0 2024-04-24T08:25:06.460 INFO:tasks.workunit.client.0.smithi090.stdout:5/661: write d8/d7c/da9/fc6 [6323250,72576] 0 2024-04-24T08:25:06.463 INFO:tasks.workunit.client.0.smithi090.stdout:8/578: write d2/da/d13/d64/d24/f5b [194247,65017] 0 2024-04-24T08:25:06.464 INFO:tasks.workunit.client.0.smithi090.stdout:9/590: dread d0/f2 [0,4194304] 0 2024-04-24T08:25:06.469 INFO:tasks.workunit.client.0.smithi090.stdout:0/621: rmdir d1a/d4b/d53/d57 39 2024-04-24T08:25:06.471 INFO:tasks.workunit.client.0.smithi090.stdout:7/681: sync 2024-04-24T08:25:06.473 INFO:tasks.workunit.client.0.smithi090.stdout:4/542: dread d3/d38/f85 [0,4194304] 0 2024-04-24T08:25:06.475 INFO:tasks.workunit.client.1.smithi149.stdout:6/786: dread d0/f12 [0,4194304] 0 2024-04-24T08:25:06.476 INFO:tasks.workunit.client.0.smithi090.stdout:6/654: rename d8/dc/f53 to d8/d1d/d1f/d3e/d44/d90/fc7 0 2024-04-24T08:25:06.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/787: readlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/laf 0 2024-04-24T08:25:06.482 INFO:tasks.workunit.client.0.smithi090.stdout:2/673: truncate d0/d7/d67/d7f/f78 439046 0 2024-04-24T08:25:06.483 INFO:tasks.workunit.client.0.smithi090.stdout:1/628: dwrite d3/f2e [0,4194304] 0 2024-04-24T08:25:06.486 INFO:tasks.workunit.client.0.smithi090.stdout:1/629: stat d3/d5/da/d97/d9c/dc4/ld1 0 2024-04-24T08:25:06.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/655: symlink d8/d1d/d29/lc8 0 2024-04-24T08:25:06.495 INFO:tasks.workunit.client.0.smithi090.stdout:6/656: chown d8/f15 46 1 2024-04-24T08:25:06.495 INFO:tasks.workunit.client.0.smithi090.stdout:3/552: dread d2/d25/d33/f43 [0,4194304] 0 2024-04-24T08:25:06.497 INFO:tasks.workunit.client.0.smithi090.stdout:2/674: unlink d0/d14/d17/f1b 0 2024-04-24T08:25:06.498 INFO:tasks.workunit.client.0.smithi090.stdout:6/657: chown d8/d1d/d2f/d85/fbc 10455759 1 2024-04-24T08:25:06.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/682: mkdir db/de9 0 2024-04-24T08:25:06.504 INFO:tasks.workunit.client.0.smithi090.stdout:0/622: write d1a/d4b/d53/d79/d82/f8d [44512,70031] 0 2024-04-24T08:25:06.508 INFO:tasks.workunit.client.0.smithi090.stdout:0/623: chown d1a/d4b/d53/d79/d69/d78 26189 1 2024-04-24T08:25:06.509 INFO:tasks.workunit.client.0.smithi090.stdout:9/591: creat d0/dc/d24/d2d/d49/fb6 x:0 0 0 2024-04-24T08:25:06.510 INFO:tasks.workunit.client.0.smithi090.stdout:5/662: dwrite d8/d76/fb2 [4194304,4194304] 0 2024-04-24T08:25:06.511 INFO:tasks.workunit.client.0.smithi090.stdout:0/624: write f18 [5622991,127360] 0 2024-04-24T08:25:06.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/579: dwrite d2/da/d13/f7e [0,4194304] 0 2024-04-24T08:25:06.513 INFO:tasks.workunit.client.0.smithi090.stdout:3/553: unlink d2/d84/f8d 0 2024-04-24T08:25:06.513 INFO:tasks.workunit.client.0.smithi090.stdout:5/663: readlink d8/d99/lbb 0 2024-04-24T08:25:06.516 INFO:tasks.workunit.client.0.smithi090.stdout:8/580: chown d2/da/d13/d64/f16 28 1 2024-04-24T08:25:06.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/658: rmdir d8/d1d/d1f/d3e 39 2024-04-24T08:25:06.517 INFO:tasks.workunit.client.0.smithi090.stdout:2/675: creat d0/d7/d67/d7f/fe6 x:0 0 0 2024-04-24T08:25:06.522 INFO:tasks.workunit.client.0.smithi090.stdout:8/581: chown d2/da/d13/d64/d24/l3b 5511 1 2024-04-24T08:25:06.525 INFO:tasks.workunit.client.1.smithi149.stdout:4/862: creat d2/d18/d1e/d3c/d59/f110 x:0 0 0 2024-04-24T08:25:06.526 INFO:tasks.workunit.client.0.smithi090.stdout:4/543: creat d3/d18/d1d/fa1 x:0 0 0 2024-04-24T08:25:06.527 INFO:tasks.workunit.client.0.smithi090.stdout:4/544: readlink d3/df/d69/l88 0 2024-04-24T08:25:06.528 INFO:tasks.workunit.client.1.smithi149.stdout:2/782: unlink d0/d6/db/d39/d3d/l52 0 2024-04-24T08:25:06.530 INFO:tasks.workunit.client.1.smithi149.stdout:7/915: mknod d9/dc/d7b/c12e 0 2024-04-24T08:25:06.535 INFO:tasks.workunit.client.0.smithi090.stdout:3/554: chown d2/d15/d36/d6f/cbb 21098 1 2024-04-24T08:25:06.535 INFO:tasks.workunit.client.0.smithi090.stdout:9/592: dwrite d0/dc/d24/d2d/d49/fb3 [0,4194304] 0 2024-04-24T08:25:06.535 INFO:tasks.workunit.client.0.smithi090.stdout:5/664: dread - d8/d59/dc7/d7a/d50/d9f/fba zero size 2024-04-24T08:25:06.544 INFO:tasks.workunit.client.1.smithi149.stdout:7/916: rmdir d9/d43/d64 39 2024-04-24T08:25:06.544 INFO:tasks.workunit.client.0.smithi090.stdout:7/683: dwrite db/d17/d29/d38/d3e/fb7 [0,4194304] 0 2024-04-24T08:25:06.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/684: read - db/d17/d29/d38/d3e/fe7 zero size 2024-04-24T08:25:06.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/545: dwrite d3/d18/d1d/f27 [0,4194304] 0 2024-04-24T08:25:06.552 INFO:tasks.workunit.client.0.smithi090.stdout:1/630: write d3/d5/da/dc/f3a [2057614,115929] 0 2024-04-24T08:25:06.553 INFO:tasks.workunit.client.0.smithi090.stdout:4/546: write d3/d38/d6b/f93 [361828,43144] 0 2024-04-24T08:25:06.553 INFO:tasks.workunit.client.1.smithi149.stdout:4/863: rename d2/c5 to d2/d10/d1a/da2/c111 0 2024-04-24T08:25:06.557 INFO:tasks.workunit.client.1.smithi149.stdout:4/864: truncate d2/d24/d2e/f66 5174018 0 2024-04-24T08:25:06.558 INFO:tasks.workunit.client.0.smithi090.stdout:5/665: creat d8/da/d14/fd3 x:0 0 0 2024-04-24T08:25:06.564 INFO:tasks.workunit.client.1.smithi149.stdout:9/926: write d1/d2/d33/dea/d133/dad/db7/f129 [895908,46531] 0 2024-04-24T08:25:06.566 INFO:tasks.workunit.client.0.smithi090.stdout:9/593: creat d0/d69/d7d/d5b/fb7 x:0 0 0 2024-04-24T08:25:06.567 INFO:tasks.workunit.client.1.smithi149.stdout:6/788: truncate d0/f3f 547185 0 2024-04-24T08:25:06.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/594: stat d0/d19/la7 0 2024-04-24T08:25:06.569 INFO:tasks.workunit.client.1.smithi149.stdout:2/783: rename d0/d6/db/f3e to d0/d6/db/d39/d78/d8a/fee 0 2024-04-24T08:25:06.570 INFO:tasks.workunit.client.1.smithi149.stdout:8/847: write d1/f9 [2979101,23481] 0 2024-04-24T08:25:06.572 INFO:tasks.workunit.client.0.smithi090.stdout:5/666: dwrite d8/d9c/fc5 [0,4194304] 0 2024-04-24T08:25:06.577 INFO:tasks.workunit.client.1.smithi149.stdout:0/828: link d2/d9/d1a/d36/d85/dcd/dda/fe0 d2/d17/f107 0 2024-04-24T08:25:06.578 INFO:tasks.workunit.client.0.smithi090.stdout:2/676: write d0/d7/d67/fba [216808,45811] 0 2024-04-24T08:25:06.579 INFO:tasks.workunit.client.0.smithi090.stdout:4/547: creat d3/d8e/fa2 x:0 0 0 2024-04-24T08:25:06.579 INFO:tasks.workunit.client.0.smithi090.stdout:7/685: chown db/d54/d78/l7d 1843 1 2024-04-24T08:25:06.580 INFO:tasks.workunit.client.1.smithi149.stdout:9/927: creat d1/de/d12/f153 x:0 0 0 2024-04-24T08:25:06.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/631: creat d3/d5/da/dc/fe3 x:0 0 0 2024-04-24T08:25:06.591 INFO:tasks.workunit.client.1.smithi149.stdout:8/848: stat d1/de/d18/d32/d65/de5 0 2024-04-24T08:25:06.592 INFO:tasks.workunit.client.1.smithi149.stdout:3/970: sync 2024-04-24T08:25:06.593 INFO:tasks.workunit.client.0.smithi090.stdout:6/659: write d8/dc/d19/d8e/d81/d96/d97/fae [848691,84917] 0 2024-04-24T08:25:06.597 INFO:tasks.workunit.client.1.smithi149.stdout:7/917: rmdir d9/d43/d4f/da5 39 2024-04-24T08:25:06.599 INFO:tasks.workunit.client.0.smithi090.stdout:5/667: symlink d8/da/d14/ld4 0 2024-04-24T08:25:06.599 INFO:tasks.workunit.client.1.smithi149.stdout:0/829: rmdir d2/d3c 39 2024-04-24T08:25:06.600 INFO:tasks.workunit.client.0.smithi090.stdout:5/668: chown d8/d25/fc8 2 1 2024-04-24T08:25:06.600 INFO:tasks.workunit.client.0.smithi090.stdout:0/625: link d1a/d4b/d53/d79/c3a d1a/d4b/d53/d79/d69/d8a/ccb 0 2024-04-24T08:25:06.602 INFO:tasks.workunit.client.1.smithi149.stdout:0/830: write d2/d9/d1a/d36/d85/fe5 [61363,53023] 0 2024-04-24T08:25:06.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/686: creat db/dbd/fea x:0 0 0 2024-04-24T08:25:06.605 INFO:tasks.workunit.client.0.smithi090.stdout:9/595: write d0/d19/f1e [2210033,97420] 0 2024-04-24T08:25:06.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/555: write d2/df/d1a/d37/d53/f65 [1102764,130895] 0 2024-04-24T08:25:06.606 INFO:tasks.workunit.client.1.smithi149.stdout:8/849: dread - d1/d1f/d31/fbb zero size 2024-04-24T08:25:06.607 INFO:tasks.workunit.client.0.smithi090.stdout:1/632: truncate d3/d5/da/dc/d80/f7d 98050 0 2024-04-24T08:25:06.611 INFO:tasks.workunit.client.0.smithi090.stdout:5/669: fdatasync d8/da/d3a/d96/dcb/d21/f48 0 2024-04-24T08:25:06.611 INFO:tasks.workunit.client.1.smithi149.stdout:7/918: write d9/d43/d64/fc5 [7535833,16228] 0 2024-04-24T08:25:06.613 INFO:tasks.workunit.client.1.smithi149.stdout:4/865: dwrite d2/f7 [0,4194304] 0 2024-04-24T08:25:06.613 INFO:tasks.workunit.client.1.smithi149.stdout:9/928: creat d1/d2/d33/d5d/d83/df4/dff/f154 x:0 0 0 2024-04-24T08:25:06.613 INFO:tasks.workunit.client.0.smithi090.stdout:5/670: chown d8/da/d14/d2f 90233 1 2024-04-24T08:25:06.617 INFO:tasks.workunit.client.1.smithi149.stdout:6/789: dread d0/fc [0,4194304] 0 2024-04-24T08:25:06.618 INFO:tasks.workunit.client.1.smithi149.stdout:7/919: chown d9/dc/d11/d83/da6/d67/d69/dad/f110 3173397 1 2024-04-24T08:25:06.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/596: rmdir d0/d2a 39 2024-04-24T08:25:06.619 INFO:tasks.workunit.client.0.smithi090.stdout:0/626: dwrite d1a/d4b/d53/d79/d69/d8a/fa2 [0,4194304] 0 2024-04-24T08:25:06.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/850: stat d1/d1f/d22/f5f 0 2024-04-24T08:25:06.622 INFO:tasks.workunit.client.1.smithi149.stdout:3/971: mknod d0/d5/d53/d4c/c140 0 2024-04-24T08:25:06.624 INFO:tasks.workunit.client.1.smithi149.stdout:2/784: link d0/d6/db/fcd d0/d6/fef 0 2024-04-24T08:25:06.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/633: creat d3/d5/da/dc/d80/fe4 x:0 0 0 2024-04-24T08:25:06.627 INFO:tasks.workunit.client.0.smithi090.stdout:7/687: dread db/d17/d29/d32/d5a/d75/f79 [0,4194304] 0 2024-04-24T08:25:06.630 INFO:tasks.workunit.client.0.smithi090.stdout:5/671: symlink d8/da/d3a/d2d/ld5 0 2024-04-24T08:25:06.633 INFO:tasks.workunit.client.0.smithi090.stdout:4/548: link d3/d38/d49/c73 d3/d24/d92/ca3 0 2024-04-24T08:25:06.634 INFO:tasks.workunit.client.0.smithi090.stdout:9/597: dread d0/d69/f6f [0,4194304] 0 2024-04-24T08:25:06.635 INFO:tasks.workunit.client.1.smithi149.stdout:7/920: rename d9/dc/de/c15 to d9/dc/d2e/d11e/dac/c12f 0 2024-04-24T08:25:06.637 INFO:tasks.workunit.client.0.smithi090.stdout:0/627: readlink d1a/l37 0 2024-04-24T08:25:06.639 INFO:tasks.workunit.client.0.smithi090.stdout:3/556: symlink d2/lbe 0 2024-04-24T08:25:06.640 INFO:tasks.workunit.client.1.smithi149.stdout:8/851: fsync d1/de/d18/f6a 0 2024-04-24T08:25:06.641 INFO:tasks.workunit.client.1.smithi149.stdout:7/921: chown d9/dc/d11/d83/da6/cf0 810976 1 2024-04-24T08:25:06.643 INFO:tasks.workunit.client.1.smithi149.stdout:7/922: fdatasync d9/dc/d2e/d3c/db8/f121 0 2024-04-24T08:25:06.646 INFO:tasks.workunit.client.1.smithi149.stdout:9/929: unlink d1/d2/d15/f3c 0 2024-04-24T08:25:06.648 INFO:tasks.workunit.client.1.smithi149.stdout:6/790: symlink d0/d1a/d28/d31/d4c/d4e/d62/d101/l10b 0 2024-04-24T08:25:06.650 INFO:tasks.workunit.client.1.smithi149.stdout:3/972: dread d0/d5/da/d19/d76/f9b [0,4194304] 0 2024-04-24T08:25:06.650 INFO:tasks.workunit.client.1.smithi149.stdout:7/923: readlink d9/dc/d11/d83/da6/d67/d69/d10f/l111 0 2024-04-24T08:25:06.652 INFO:tasks.workunit.client.0.smithi090.stdout:6/660: rename d8/d1d/d1f/d3e/la7 to d8/dc/d19/d8e/d81/lc9 0 2024-04-24T08:25:06.654 INFO:tasks.workunit.client.0.smithi090.stdout:1/634: rmdir d3/d25/d30/d9d/da2 39 2024-04-24T08:25:06.656 INFO:tasks.workunit.client.1.smithi149.stdout:1/952: sync 2024-04-24T08:25:06.656 INFO:tasks.workunit.client.1.smithi149.stdout:1/953: readlink d0/d1b/db5/d108/l7d 0 2024-04-24T08:25:06.658 INFO:tasks.workunit.client.1.smithi149.stdout:8/852: mknod d1/de/d18/daf/c11e 0 2024-04-24T08:25:06.663 INFO:tasks.workunit.client.0.smithi090.stdout:4/549: creat d3/df/d7c/fa4 x:0 0 0 2024-04-24T08:25:06.667 INFO:tasks.workunit.client.0.smithi090.stdout:0/628: mkdir d1a/d4b/d53/d79/d69/d90/dcc 0 2024-04-24T08:25:06.669 INFO:tasks.workunit.client.1.smithi149.stdout:4/866: dread d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/f103 [0,4194304] 0 2024-04-24T08:25:06.669 INFO:tasks.workunit.client.1.smithi149.stdout:0/831: dread d2/d9/df/d2f/d6c/d72/f99 [0,4194304] 0 2024-04-24T08:25:06.672 INFO:tasks.workunit.client.1.smithi149.stdout:8/853: symlink d1/de/d18/d32/d63/d76/l11f 0 2024-04-24T08:25:06.673 INFO:tasks.workunit.client.1.smithi149.stdout:2/785: dwrite d0/d6/db/d39/d78/d7c/fe6 [0,4194304] 0 2024-04-24T08:25:06.673 INFO:tasks.workunit.client.1.smithi149.stdout:7/924: dwrite d9/dc/d11/d83/da6/d67/f9e [0,4194304] 0 2024-04-24T08:25:06.676 INFO:tasks.workunit.client.0.smithi090.stdout:4/550: dwrite d3/d18/d1d/f40 [0,4194304] 0 2024-04-24T08:25:06.678 INFO:tasks.workunit.client.1.smithi149.stdout:8/854: read - d1/de/d18/d32/dde/f11b zero size 2024-04-24T08:25:06.678 INFO:tasks.workunit.client.1.smithi149.stdout:1/954: rename d0/ddf/da0/fe3 to d0/d1b/db5/d108/dd3/d123/f136 0 2024-04-24T08:25:06.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/973: chown d0/d5/da/d19/d1b/d133/dd4/fde 330089418 1 2024-04-24T08:25:06.683 INFO:tasks.workunit.client.1.smithi149.stdout:0/832: mkdir d2/d17/d41/d68/d108 0 2024-04-24T08:25:06.685 INFO:tasks.workunit.client.1.smithi149.stdout:3/974: readlink d0/d5/da/d19/d1b/d133/d38/l12b 0 2024-04-24T08:25:06.686 INFO:tasks.workunit.client.0.smithi090.stdout:3/557: read d2/d15/d36/f83 [3556548,67185] 0 2024-04-24T08:25:06.693 INFO:tasks.workunit.client.0.smithi090.stdout:7/688: dwrite db/d17/d29/d32/f4c [0,4194304] 0 2024-04-24T08:25:06.697 INFO:tasks.workunit.client.0.smithi090.stdout:9/598: link d0/d69/f9d d0/d2a/fb8 0 2024-04-24T08:25:06.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/672: dwrite d8/da/d3a/d96/dcb/fa0 [0,4194304] 0 2024-04-24T08:25:06.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/558: read d2/df/d1a/f4c [1472744,106435] 0 2024-04-24T08:25:06.708 INFO:tasks.workunit.client.1.smithi149.stdout:2/786: dwrite d0/d6/db/d27/d64/db3/fdd [0,4194304] 0 2024-04-24T08:25:06.710 INFO:tasks.workunit.client.0.smithi090.stdout:8/582: sync 2024-04-24T08:25:06.712 INFO:tasks.workunit.client.0.smithi090.stdout:6/661: write f6 [4314433,106264] 0 2024-04-24T08:25:06.714 INFO:tasks.workunit.client.1.smithi149.stdout:0/833: rmdir d2/d9/df/d2f/d6c/dc1/d4e/d6f 39 2024-04-24T08:25:06.715 INFO:tasks.workunit.client.1.smithi149.stdout:2/787: dread - d0/d6/fef zero size 2024-04-24T08:25:06.719 INFO:tasks.workunit.client.0.smithi090.stdout:9/599: dwrite d0/d19/f1a [0,4194304] 0 2024-04-24T08:25:06.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/689: creat db/de9/feb x:0 0 0 2024-04-24T08:25:06.724 INFO:tasks.workunit.client.1.smithi149.stdout:9/930: link d1/d2/d15/f23 d1/de/f155 0 2024-04-24T08:25:06.727 INFO:tasks.workunit.client.1.smithi149.stdout:0/834: fdatasync d2/d9/df/d2f/d6c/dc1/fa9 0 2024-04-24T08:25:06.728 INFO:tasks.workunit.client.1.smithi149.stdout:8/855: creat d1/de/d18/f120 x:0 0 0 2024-04-24T08:25:06.728 INFO:tasks.workunit.client.0.smithi090.stdout:5/673: link d8/d76/fb2 d8/d59/dc7/d7a/dbc/fd6 0 2024-04-24T08:25:06.729 INFO:tasks.workunit.client.0.smithi090.stdout:8/583: fdatasync d2/da/d13/d34/f40 0 2024-04-24T08:25:06.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/551: getdents d3/d38/d49/d6f 0 2024-04-24T08:25:06.732 INFO:tasks.workunit.client.0.smithi090.stdout:6/662: dread d8/d1d/d1f/d3e/d4d/f99 [0,4194304] 0 2024-04-24T08:25:06.732 INFO:tasks.workunit.client.1.smithi149.stdout:9/931: symlink d1/da/d13/d67/l156 0 2024-04-24T08:25:06.733 INFO:tasks.workunit.client.1.smithi149.stdout:1/955: dread d0/ddf/fc2 [0,4194304] 0 2024-04-24T08:25:06.741 INFO:tasks.workunit.client.0.smithi090.stdout:0/629: write d1a/d4b/d53/d79/d69/d78/f92 [376890,115848] 0 2024-04-24T08:25:06.745 INFO:tasks.workunit.client.1.smithi149.stdout:4/867: write d2/d10/d1a/fbe [5245820,113009] 0 2024-04-24T08:25:06.748 INFO:tasks.workunit.client.0.smithi090.stdout:8/584: creat d2/d62/d71/fc9 x:0 0 0 2024-04-24T08:25:06.750 INFO:tasks.workunit.client.1.smithi149.stdout:9/932: creat d1/d2/d33/d5d/d83/dbf/f157 x:0 0 0 2024-04-24T08:25:06.754 INFO:tasks.workunit.client.0.smithi090.stdout:5/674: dwrite d8/d59/dc7/d7a/f4e [0,4194304] 0 2024-04-24T08:25:06.755 INFO:tasks.workunit.client.1.smithi149.stdout:2/788: dwrite d0/d6/db/d39/d3d/d44/d57/f71 [0,4194304] 0 2024-04-24T08:25:06.755 INFO:tasks.workunit.client.1.smithi149.stdout:0/835: link d2/d9/df/d2f/d6c/f9a d2/d9/df/d48/dba/f109 0 2024-04-24T08:25:06.756 INFO:tasks.workunit.client.0.smithi090.stdout:5/675: fsync d8/da/d3a/d96/dcb/f22 0 2024-04-24T08:25:06.757 INFO:tasks.workunit.client.1.smithi149.stdout:3/975: write d0/fda [2022,52205] 0 2024-04-24T08:25:06.759 INFO:tasks.workunit.client.1.smithi149.stdout:7/925: dwrite d9/dc/de/f16 [4194304,4194304] 0 2024-04-24T08:25:06.769 INFO:tasks.workunit.client.1.smithi149.stdout:4/868: fsync d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 0 2024-04-24T08:25:06.769 INFO:tasks.workunit.client.0.smithi090.stdout:3/559: dwrite d2/d15/f1d [0,4194304] 0 2024-04-24T08:25:06.771 INFO:tasks.workunit.client.0.smithi090.stdout:3/560: chown d2/df/d1a/d37/d53/d60 47781 1 2024-04-24T08:25:06.776 INFO:tasks.workunit.client.0.smithi090.stdout:4/552: symlink d3/d38/la5 0 2024-04-24T08:25:06.778 INFO:tasks.workunit.client.0.smithi090.stdout:9/600: write d0/dc/d24/d2d/d49/d62/f51 [1714787,56743] 0 2024-04-24T08:25:06.780 INFO:tasks.workunit.client.0.smithi090.stdout:8/585: creat d2/da/d13/d60/d76/fca x:0 0 0 2024-04-24T08:25:06.780 INFO:tasks.workunit.client.0.smithi090.stdout:2/677: sync 2024-04-24T08:25:06.781 INFO:tasks.workunit.client.1.smithi149.stdout:8/856: link d1/de/d18/fb7 d1/de/d18/d32/d63/d9e/dec/f121 0 2024-04-24T08:25:06.789 INFO:tasks.workunit.client.1.smithi149.stdout:8/857: fsync d1/de/d18/d43/fb6 0 2024-04-24T08:25:06.793 INFO:tasks.workunit.client.1.smithi149.stdout:3/976: creat d0/d5/da/d19/d1b/d133/d38/f141 x:0 0 0 2024-04-24T08:25:06.794 INFO:tasks.workunit.client.0.smithi090.stdout:5/676: read - d8/da/d14/f69 zero size 2024-04-24T08:25:06.795 INFO:tasks.workunit.client.1.smithi149.stdout:1/956: dwrite d0/ddd/f4e [0,4194304] 0 2024-04-24T08:25:06.798 INFO:tasks.workunit.client.0.smithi090.stdout:7/690: link db/d17/l93 db/d17/d29/d32/d5a/d75/lec 0 2024-04-24T08:25:06.800 INFO:tasks.workunit.client.1.smithi149.stdout:1/957: chown d0/d1b/db5/f8b 39882408 1 2024-04-24T08:25:06.803 INFO:tasks.workunit.client.0.smithi090.stdout:0/630: getdents d1a/d4b/d53/d79/d69/d90/dbc 0 2024-04-24T08:25:06.809 INFO:tasks.workunit.client.0.smithi090.stdout:4/553: unlink d3/df/d77/f89 0 2024-04-24T08:25:06.809 INFO:tasks.workunit.client.1.smithi149.stdout:0/836: creat d2/d3c/dc7/f10a x:0 0 0 2024-04-24T08:25:06.812 INFO:tasks.workunit.client.1.smithi149.stdout:2/789: unlink d0/d6/db/d27/d81/d9f/dba/cbb 0 2024-04-24T08:25:06.812 INFO:tasks.workunit.client.0.smithi090.stdout:9/601: symlink d0/dc/d24/d2d/d49/d81/lb9 0 2024-04-24T08:25:06.812 INFO:tasks.workunit.client.0.smithi090.stdout:8/586: mknod d2/da/db0/ccb 0 2024-04-24T08:25:06.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/858: dwrite d1/d1f/d22/f71 [0,4194304] 0 2024-04-24T08:25:06.813 INFO:tasks.workunit.client.0.smithi090.stdout:9/602: fdatasync d0/d19/f25 0 2024-04-24T08:25:06.814 INFO:tasks.workunit.client.1.smithi149.stdout:9/933: mknod d1/d2/d34/c158 0 2024-04-24T08:25:06.816 INFO:tasks.workunit.client.1.smithi149.stdout:8/859: write d1/de/d18/d43/fb6 [7181380,97480] 0 2024-04-24T08:25:06.817 INFO:tasks.workunit.client.0.smithi090.stdout:6/663: rename d8/dc/d19/d8e/f52 to d8/d1d/d1f/d3e/fca 0 2024-04-24T08:25:06.819 INFO:tasks.workunit.client.0.smithi090.stdout:6/664: fsync d8/dc/d19/f64 0 2024-04-24T08:25:06.819 INFO:tasks.workunit.client.0.smithi090.stdout:2/678: symlink d0/d7/d57/d79/le7 0 2024-04-24T08:25:06.820 INFO:tasks.workunit.client.1.smithi149.stdout:3/977: symlink d0/d5/d73/l142 0 2024-04-24T08:25:06.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/677: creat d8/d59/dc7/d7b/fd7 x:0 0 0 2024-04-24T08:25:06.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/665: write d8/dc/d19/d8e/d81/d96/fbb [549588,12518] 0 2024-04-24T08:25:06.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/679: chown d0/d7/d67/d7f/d31/d46/c8a 2 1 2024-04-24T08:25:06.823 INFO:tasks.workunit.client.0.smithi090.stdout:7/691: fsync db/d17/d29/fe5 0 2024-04-24T08:25:06.825 INFO:tasks.workunit.client.0.smithi090.stdout:2/680: chown d0/d14/d4d/ca0 15680 1 2024-04-24T08:25:06.826 INFO:tasks.workunit.client.0.smithi090.stdout:0/631: truncate d1a/d4b/d53/d79/d35/d68/f7b 381250 0 2024-04-24T08:25:06.827 INFO:tasks.workunit.client.0.smithi090.stdout:4/554: rmdir d3/d18 39 2024-04-24T08:25:06.829 INFO:tasks.workunit.client.0.smithi090.stdout:2/681: fsync d0/d7/d28/daa/db1/fe5 0 2024-04-24T08:25:06.832 INFO:tasks.workunit.client.0.smithi090.stdout:7/692: truncate db/d17/d29/dc0/fe1 822387 0 2024-04-24T08:25:06.834 INFO:tasks.workunit.client.0.smithi090.stdout:4/555: write d3/d24/d92/f9b [900689,88391] 0 2024-04-24T08:25:06.835 INFO:tasks.workunit.client.1.smithi149.stdout:2/790: fdatasync d0/d6/db/d39/d3d/d44/d75/f79 0 2024-04-24T08:25:06.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/635: sync 2024-04-24T08:25:06.840 INFO:tasks.workunit.client.1.smithi149.stdout:8/860: creat d1/de/d18/d43/f122 x:0 0 0 2024-04-24T08:25:06.847 INFO:tasks.workunit.client.0.smithi090.stdout:6/666: fdatasync d8/f46 0 2024-04-24T08:25:06.848 INFO:tasks.workunit.client.1.smithi149.stdout:7/926: rmdir d9/dc/d7b/d126 0 2024-04-24T08:25:06.849 INFO:tasks.workunit.client.1.smithi149.stdout:4/869: dwrite d2/d24/d2e/d3f/d94/fe8 [0,4194304] 0 2024-04-24T08:25:06.851 INFO:tasks.workunit.client.1.smithi149.stdout:9/934: mknod d1/d2/d15/c159 0 2024-04-24T08:25:06.854 INFO:tasks.workunit.client.1.smithi149.stdout:0/837: dread - d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/fd7 zero size 2024-04-24T08:25:06.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/603: symlink d0/dc/lba 0 2024-04-24T08:25:06.858 INFO:tasks.workunit.client.1.smithi149.stdout:2/791: unlink d0/d6/db/d28/d7f/d9b/fe2 0 2024-04-24T08:25:06.859 INFO:tasks.workunit.client.0.smithi090.stdout:8/587: dread d2/da/f87 [0,4194304] 0 2024-04-24T08:25:06.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/588: chown d2/d62/d6b/f7d 289285 1 2024-04-24T08:25:06.863 INFO:tasks.workunit.client.0.smithi090.stdout:4/556: readlink d3/d18/d5c/l52 0 2024-04-24T08:25:06.864 INFO:tasks.workunit.client.1.smithi149.stdout:3/978: dread d0/d26/d3c/db1/fd0 [8388608,4194304] 0 2024-04-24T08:25:06.865 INFO:tasks.workunit.client.0.smithi090.stdout:4/557: chown d3/d38/d49/d6f/f7d 152492 1 2024-04-24T08:25:06.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/558: stat d3/d24/f44 0 2024-04-24T08:25:06.867 INFO:tasks.workunit.client.1.smithi149.stdout:8/861: fsync d1/d1f/fa6 0 2024-04-24T08:25:06.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/636: creat d3/d5/d58/fe5 x:0 0 0 2024-04-24T08:25:06.869 INFO:tasks.workunit.client.0.smithi090.stdout:4/559: write d3/d18/f5b [574576,81538] 0 2024-04-24T08:25:06.871 INFO:tasks.workunit.client.1.smithi149.stdout:4/870: creat d2/d10/d91/f112 x:0 0 0 2024-04-24T08:25:06.871 INFO:tasks.workunit.client.1.smithi149.stdout:0/838: dread - d2/d17/d41/fb3 zero size 2024-04-24T08:25:06.873 INFO:tasks.workunit.client.1.smithi149.stdout:4/871: stat d2/d10/d1a/f3e 0 2024-04-24T08:25:06.874 INFO:tasks.workunit.client.0.smithi090.stdout:5/678: dread d8/da/d14/d2f/d44/f49 [0,4194304] 0 2024-04-24T08:25:06.876 INFO:tasks.workunit.client.1.smithi149.stdout:0/839: write d2/d9/f2d [2925822,101794] 0 2024-04-24T08:25:06.878 INFO:tasks.workunit.client.1.smithi149.stdout:1/958: dread d0/d1b/db5/d108/f10a [0,4194304] 0 2024-04-24T08:25:06.879 INFO:tasks.workunit.client.1.smithi149.stdout:7/927: mknod d9/d43/d4f/c130 0 2024-04-24T08:25:06.880 INFO:tasks.workunit.client.0.smithi090.stdout:3/561: rename d2/df/d1a/d37/d59/f5e to d2/d15/fbf 0 2024-04-24T08:25:06.883 INFO:tasks.workunit.client.1.smithi149.stdout:8/862: mknod d1/de/d18/d32/d63/c123 0 2024-04-24T08:25:06.885 INFO:tasks.workunit.client.0.smithi090.stdout:7/693: link db/d17/f3a db/d17/d29/d38/d3e/d46/fed 0 2024-04-24T08:25:06.888 INFO:tasks.workunit.client.1.smithi149.stdout:0/840: fdatasync d2/fa 0 2024-04-24T08:25:06.891 INFO:tasks.workunit.client.0.smithi090.stdout:0/632: rename d1a/f56 to d1a/fcd 0 2024-04-24T08:25:06.892 INFO:tasks.workunit.client.1.smithi149.stdout:7/928: chown d9/dc/d2e/d11e/dac/c112 11 1 2024-04-24T08:25:06.893 INFO:tasks.workunit.client.1.smithi149.stdout:9/935: write d1/de/f18 [1258637,69842] 0 2024-04-24T08:25:06.893 INFO:tasks.workunit.client.0.smithi090.stdout:4/560: mkdir d3/d87/da6 0 2024-04-24T08:25:06.895 INFO:tasks.workunit.client.1.smithi149.stdout:1/959: creat d0/ddf/da0/db8/dec/f137 x:0 0 0 2024-04-24T08:25:06.895 INFO:tasks.workunit.client.0.smithi090.stdout:2/682: link d0/d14/d4d/ca0 d0/d7/d28/daa/ce8 0 2024-04-24T08:25:06.896 INFO:tasks.workunit.client.0.smithi090.stdout:2/683: chown d0/d7/l23 225 1 2024-04-24T08:25:06.897 INFO:tasks.workunit.client.1.smithi149.stdout:2/792: write d0/d6c/fa8 [340198,47906] 0 2024-04-24T08:25:06.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/872: truncate d2/d18/d1e/d3c/d48/d69/f77 613917 0 2024-04-24T08:25:06.900 INFO:tasks.workunit.client.1.smithi149.stdout:2/793: dread - d0/d6/db/d39/d3d/fd6 zero size 2024-04-24T08:25:06.900 INFO:tasks.workunit.client.1.smithi149.stdout:8/863: symlink d1/d1f/d25/d111/l124 0 2024-04-24T08:25:06.902 INFO:tasks.workunit.client.0.smithi090.stdout:8/589: mkdir d2/da/d13/d64/dcc 0 2024-04-24T08:25:06.904 INFO:tasks.workunit.client.0.smithi090.stdout:8/590: dread - d2/da/d13/f94 zero size 2024-04-24T08:25:06.905 INFO:tasks.workunit.client.0.smithi090.stdout:8/591: fdatasync d2/d62/fbc 0 2024-04-24T08:25:06.908 INFO:tasks.workunit.client.1.smithi149.stdout:3/979: dread d0/d5/f6f [0,4194304] 0 2024-04-24T08:25:06.908 INFO:tasks.workunit.client.0.smithi090.stdout:8/592: write d2/da/f38 [2135850,115103] 0 2024-04-24T08:25:06.909 INFO:tasks.workunit.client.1.smithi149.stdout:3/980: stat d0/d5/da/d59 0 2024-04-24T08:25:06.909 INFO:tasks.workunit.client.0.smithi090.stdout:9/604: rename d0/dc/d24/d7c/l76 to d0/d19/lbb 0 2024-04-24T08:25:06.911 INFO:tasks.workunit.client.0.smithi090.stdout:8/593: chown d2/d62/d83/faf 428 1 2024-04-24T08:25:06.911 INFO:tasks.workunit.client.0.smithi090.stdout:9/605: chown d0/dc/d24/d7c/d4d/d32 0 1 2024-04-24T08:25:06.912 INFO:tasks.workunit.client.0.smithi090.stdout:1/637: mknod d3/d5/ce6 0 2024-04-24T08:25:06.914 INFO:tasks.workunit.client.1.smithi149.stdout:1/960: symlink d0/db2/l138 0 2024-04-24T08:25:06.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/606: read d0/d2a/f38 [3288963,68531] 0 2024-04-24T08:25:06.916 INFO:tasks.workunit.client.1.smithi149.stdout:4/873: symlink d2/d18/d1e/d3c/d48/d69/l113 0 2024-04-24T08:25:06.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/607: read - d0/d19/fa0 zero size 2024-04-24T08:25:06.918 INFO:tasks.workunit.client.0.smithi090.stdout:0/633: readlink d1a/l8f 0 2024-04-24T08:25:06.920 INFO:tasks.workunit.client.0.smithi090.stdout:5/679: mknod d8/da/d3a/d96/dcb/d21/d4c/cd8 0 2024-04-24T08:25:06.920 INFO:tasks.workunit.client.0.smithi090.stdout:5/680: fdatasync d8/f9e 0 2024-04-24T08:25:06.923 INFO:tasks.workunit.client.0.smithi090.stdout:3/562: truncate d2/f24 13381 0 2024-04-24T08:25:06.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/841: link d2/d17/f24 d2/d17/d41/d68/d108/f10b 0 2024-04-24T08:25:06.927 INFO:tasks.workunit.client.1.smithi149.stdout:7/929: truncate d9/d43/d4f/d66/f6e 404194 0 2024-04-24T08:25:06.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/694: rename db/c80 to db/d17/d39/d86/dae/cee 0 2024-04-24T08:25:06.932 INFO:tasks.workunit.client.1.smithi149.stdout:1/961: mknod d0/ddf/da0/db8/c139 0 2024-04-24T08:25:06.938 INFO:tasks.workunit.client.1.smithi149.stdout:8/864: mkdir d1/de/d18/d32/d65/d125 0 2024-04-24T08:25:06.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/594: mkdir d2/da/d13/d64/d24/db3/db4/dcd 0 2024-04-24T08:25:06.940 INFO:tasks.workunit.client.0.smithi090.stdout:8/595: chown d2/da/lc7 5 1 2024-04-24T08:25:06.940 INFO:tasks.workunit.client.0.smithi090.stdout:9/608: mkdir d0/d69/d7d/dbc 0 2024-04-24T08:25:06.943 INFO:tasks.workunit.client.1.smithi149.stdout:9/936: dwrite d1/de/d12/f3d [0,4194304] 0 2024-04-24T08:25:06.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/842: fdatasync d2/fb 0 2024-04-24T08:25:06.945 INFO:tasks.workunit.client.0.smithi090.stdout:9/609: truncate d0/dc/d24/d2d/d49/fb6 784370 0 2024-04-24T08:25:06.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/634: truncate d1a/d4b/d53/f5d 556744 0 2024-04-24T08:25:06.951 INFO:tasks.workunit.client.0.smithi090.stdout:5/681: creat d8/d59/dc7/d7a/d50/d9f/fd9 x:0 0 0 2024-04-24T08:25:06.956 INFO:tasks.workunit.client.1.smithi149.stdout:2/794: dwrite d0/d6/db/d27/d64/d66/db1/dc9/fd7 [0,4194304] 0 2024-04-24T08:25:06.961 INFO:tasks.workunit.client.0.smithi090.stdout:7/695: creat db/d17/d29/d38/d99/fef x:0 0 0 2024-04-24T08:25:06.962 INFO:tasks.workunit.client.0.smithi090.stdout:9/610: dwrite d0/d69/d7d/d5b/d96/fa1 [0,4194304] 0 2024-04-24T08:25:06.964 INFO:tasks.workunit.client.0.smithi090.stdout:7/696: stat db/d17/d29/d32/d5a/d75/f79 0 2024-04-24T08:25:06.970 INFO:tasks.workunit.client.0.smithi090.stdout:8/596: mknod d2/da/d13/d64/db2/cce 0 2024-04-24T08:25:06.974 INFO:tasks.workunit.client.1.smithi149.stdout:0/843: creat d2/d9/d1a/d36/dcc/f10c x:0 0 0 2024-04-24T08:25:06.979 INFO:tasks.workunit.client.1.smithi149.stdout:4/874: dwrite d2/d24/d2e/f98 [4194304,4194304] 0 2024-04-24T08:25:06.979 INFO:tasks.workunit.client.1.smithi149.stdout:9/937: truncate d1/d2/fba 994539 0 2024-04-24T08:25:06.980 INFO:tasks.workunit.client.0.smithi090.stdout:0/635: symlink d1a/d4b/d53/d79/d69/d8a/lce 0 2024-04-24T08:25:06.981 INFO:tasks.workunit.client.0.smithi090.stdout:5/682: mknod d8/d25/cda 0 2024-04-24T08:25:06.983 INFO:tasks.workunit.client.0.smithi090.stdout:6/667: sync 2024-04-24T08:25:06.985 INFO:tasks.workunit.client.0.smithi090.stdout:6/668: dread - d8/d1d/d1f/d3e/d8f/f9e zero size 2024-04-24T08:25:06.985 INFO:tasks.workunit.client.1.smithi149.stdout:9/938: write d1/d2/d33/d60/d10b/f130 [871804,75141] 0 2024-04-24T08:25:06.987 INFO:tasks.workunit.client.0.smithi090.stdout:6/669: read - d8/d1d/d1f/f9c zero size 2024-04-24T08:25:06.988 INFO:tasks.workunit.client.0.smithi090.stdout:9/611: rmdir d0/dc 39 2024-04-24T08:25:06.989 INFO:tasks.workunit.client.1.smithi149.stdout:7/930: write d9/dc/d2e/d3c/ffa [812,4299] 0 2024-04-24T08:25:06.989 INFO:tasks.workunit.client.0.smithi090.stdout:6/670: fsync d8/dc/f10 0 2024-04-24T08:25:06.991 INFO:tasks.workunit.client.0.smithi090.stdout:4/561: link d3/d38/f4d d3/df/d77/fa7 0 2024-04-24T08:25:06.992 INFO:tasks.workunit.client.0.smithi090.stdout:4/562: readlink d3/df/d69/l88 0 2024-04-24T08:25:07.000 INFO:tasks.workunit.client.1.smithi149.stdout:8/865: creat d1/de/d18/d32/d65/dad/f126 x:0 0 0 2024-04-24T08:25:07.004 INFO:tasks.workunit.client.1.smithi149.stdout:1/962: dwrite d0/d5/db/d18/d2f/fa4 [0,4194304] 0 2024-04-24T08:25:07.006 INFO:tasks.workunit.client.0.smithi090.stdout:6/671: dwrite f6 [0,4194304] 0 2024-04-24T08:25:07.007 INFO:tasks.workunit.client.0.smithi090.stdout:5/683: creat d8/d76/dad/fdb x:0 0 0 2024-04-24T08:25:07.008 INFO:tasks.workunit.client.1.smithi149.stdout:1/963: chown d0/ddf/d53/d61/fc7 57829519 1 2024-04-24T08:25:07.009 INFO:tasks.workunit.client.1.smithi149.stdout:1/964: chown d0/d1b/d2a 135 1 2024-04-24T08:25:07.009 INFO:tasks.workunit.client.1.smithi149.stdout:0/844: dread d2/d9/df/d40/f67 [0,4194304] 0 2024-04-24T08:25:07.010 INFO:tasks.workunit.client.1.smithi149.stdout:7/931: mkdir d9/dc/d11/d83/da6/d67/d69/dad/d131 0 2024-04-24T08:25:07.013 INFO:tasks.workunit.client.1.smithi149.stdout:4/875: mknod d2/d24/c114 0 2024-04-24T08:25:07.014 INFO:tasks.workunit.client.0.smithi090.stdout:1/638: link d3/d1d/d34/l89 d3/d1d/d34/d73/dd8/le7 0 2024-04-24T08:25:07.016 INFO:tasks.workunit.client.1.smithi149.stdout:2/795: getdents d0/d6/db/d27/d64/d66 0 2024-04-24T08:25:07.016 INFO:tasks.workunit.client.1.smithi149.stdout:8/866: creat d1/de/d18/d91/d9f/f127 x:0 0 0 2024-04-24T08:25:07.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/845: dread - d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e/fde zero size 2024-04-24T08:25:07.021 INFO:tasks.workunit.client.0.smithi090.stdout:4/563: creat d3/df/d69/fa8 x:0 0 0 2024-04-24T08:25:07.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/697: dwrite db/d17/d29/d32/d5a/d75/fbe [0,4194304] 0 2024-04-24T08:25:07.022 INFO:tasks.workunit.client.1.smithi149.stdout:9/939: read d1/d2/d34/ff2 [1662696,69455] 0 2024-04-24T08:25:07.023 INFO:tasks.workunit.client.0.smithi090.stdout:4/564: read - d3/df/f84 zero size 2024-04-24T08:25:07.026 INFO:tasks.workunit.client.0.smithi090.stdout:5/684: dread d8/faf [0,4194304] 0 2024-04-24T08:25:07.029 INFO:tasks.workunit.client.0.smithi090.stdout:8/597: link d2/da/d13/d64/d50/c51 d2/d62/d6b/d9c/ccf 0 2024-04-24T08:25:07.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/876: getdents d2/d10/d1a/dac/ddd 0 2024-04-24T08:25:07.030 INFO:tasks.workunit.client.0.smithi090.stdout:6/672: symlink d8/dc/lcb 0 2024-04-24T08:25:07.034 INFO:tasks.workunit.client.1.smithi149.stdout:0/846: mkdir d2/d9/d1a/d36/dcc/d10d 0 2024-04-24T08:25:07.037 INFO:tasks.workunit.client.0.smithi090.stdout:4/565: creat d3/df/d7c/fa9 x:0 0 0 2024-04-24T08:25:07.038 INFO:tasks.workunit.client.1.smithi149.stdout:9/940: rename d1/d2/d33/d5d/c11c to d1/da/d13/c15a 0 2024-04-24T08:25:07.038 INFO:tasks.workunit.client.1.smithi149.stdout:7/932: dwrite d9/dc/d72/f76 [0,4194304] 0 2024-04-24T08:25:07.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/796: unlink d0/d6/db/d39/d78/d7c/fe8 0 2024-04-24T08:25:07.047 INFO:tasks.workunit.client.0.smithi090.stdout:7/698: dwrite db/d17/d29/d38/d3e/f58 [0,4194304] 0 2024-04-24T08:25:07.051 INFO:tasks.workunit.client.0.smithi090.stdout:6/673: rename d8/f15 to d8/d1d/d1f/d3e/d8f/fcc 0 2024-04-24T08:25:07.051 INFO:tasks.workunit.client.0.smithi090.stdout:4/566: mknod d3/df/d77/caa 0 2024-04-24T08:25:07.053 INFO:tasks.workunit.client.1.smithi149.stdout:7/933: mkdir d9/dc/d2e/d51/d132 0 2024-04-24T08:25:07.054 INFO:tasks.workunit.client.0.smithi090.stdout:6/674: chown d8/dc/d19/f34 6948 1 2024-04-24T08:25:07.055 INFO:tasks.workunit.client.1.smithi149.stdout:2/797: fdatasync d0/d6/db/d28/d7f/d8f/fbe 0 2024-04-24T08:25:07.057 INFO:tasks.workunit.client.0.smithi090.stdout:9/612: link d0/dc/c11 d0/dc/d24/d2d/cbd 0 2024-04-24T08:25:07.058 INFO:tasks.workunit.client.1.smithi149.stdout:8/867: write d1/d1f/d31/fc3 [590638,106735] 0 2024-04-24T08:25:07.062 INFO:tasks.workunit.client.0.smithi090.stdout:6/675: read d8/d1d/d29/f6c [373300,128010] 0 2024-04-24T08:25:07.064 INFO:tasks.workunit.client.0.smithi090.stdout:8/598: mkdir d2/da/d13/d64/db2/db6/dd0 0 2024-04-24T08:25:07.067 INFO:tasks.workunit.client.0.smithi090.stdout:4/567: unlink d3/d24/d92/f9b 0 2024-04-24T08:25:07.069 INFO:tasks.workunit.client.1.smithi149.stdout:1/965: dwrite d0/d5/db/d18/f29 [4194304,4194304] 0 2024-04-24T08:25:07.071 INFO:tasks.workunit.client.1.smithi149.stdout:8/868: creat d1/d1f/d31/f128 x:0 0 0 2024-04-24T08:25:07.071 INFO:tasks.workunit.client.0.smithi090.stdout:7/699: dwrite db/d54/db1/fe2 [0,4194304] 0 2024-04-24T08:25:07.074 INFO:tasks.workunit.client.1.smithi149.stdout:8/869: chown d1/d1f/d31/fbb 8038888 1 2024-04-24T08:25:07.076 INFO:tasks.workunit.client.0.smithi090.stdout:0/636: dwrite d1a/f20 [4194304,4194304] 0 2024-04-24T08:25:07.080 INFO:tasks.workunit.client.1.smithi149.stdout:1/966: truncate d0/d1b/d5d/f9a 2128253 0 2024-04-24T08:25:07.081 INFO:tasks.workunit.client.1.smithi149.stdout:6/791: sync 2024-04-24T08:25:07.081 INFO:tasks.workunit.client.0.smithi090.stdout:3/563: sync 2024-04-24T08:25:07.081 INFO:tasks.workunit.client.0.smithi090.stdout:2/684: sync 2024-04-24T08:25:07.084 INFO:tasks.workunit.client.0.smithi090.stdout:7/700: chown db/d17/d29/d32/f4c 38857498 1 2024-04-24T08:25:07.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/599: symlink d2/da/d13/d64/dcc/ld1 0 2024-04-24T08:25:07.086 INFO:tasks.workunit.client.0.smithi090.stdout:7/701: chown db/d54 26 1 2024-04-24T08:25:07.087 INFO:tasks.workunit.client.0.smithi090.stdout:3/564: chown d2/d25/d33/f80 7368 1 2024-04-24T08:25:07.088 INFO:tasks.workunit.client.1.smithi149.stdout:0/847: truncate d2/d9/df/d40/f88 713205 0 2024-04-24T08:25:07.088 INFO:tasks.workunit.client.0.smithi090.stdout:0/637: chown d1a/d4b/d53/d79/d69/d90/dbc 920 1 2024-04-24T08:25:07.091 INFO:tasks.workunit.client.1.smithi149.stdout:1/967: dread - d0/d1b/d10d/d121/f12a zero size 2024-04-24T08:25:07.092 INFO:tasks.workunit.client.1.smithi149.stdout:9/941: dread d1/d2/d15/d1c/f2a [0,4194304] 0 2024-04-24T08:25:07.092 INFO:tasks.workunit.client.1.smithi149.stdout:4/877: dwrite d2/d24/d2e/d38/fec [0,4194304] 0 2024-04-24T08:25:07.097 INFO:tasks.workunit.client.1.smithi149.stdout:0/848: creat d2/d9/df/d2f/f10e x:0 0 0 2024-04-24T08:25:07.097 INFO:tasks.workunit.client.1.smithi149.stdout:4/878: chown d2/d10/d91/df7 792058572 1 2024-04-24T08:25:07.099 INFO:tasks.workunit.client.0.smithi090.stdout:7/702: readlink db/d17/d39/d59/ldd 0 2024-04-24T08:25:07.102 INFO:tasks.workunit.client.1.smithi149.stdout:9/942: dread - d1/da/d13/f134 zero size 2024-04-24T08:25:07.106 INFO:tasks.workunit.client.0.smithi090.stdout:9/613: dwrite d0/dc/d24/d2d/f5d [0,4194304] 0 2024-04-24T08:25:07.106 INFO:tasks.workunit.client.1.smithi149.stdout:0/849: chown d2/d9/df/d56/f5f 0 1 2024-04-24T08:25:07.113 INFO:tasks.workunit.client.1.smithi149.stdout:2/798: dread d0/d6/db/d28/f3a [0,4194304] 0 2024-04-24T08:25:07.113 INFO:tasks.workunit.client.0.smithi090.stdout:3/565: truncate d2/df/d1a/f4c 4909785 0 2024-04-24T08:25:07.114 INFO:tasks.workunit.client.0.smithi090.stdout:2/685: mkdir d0/d5e/de2/de9 0 2024-04-24T08:25:07.115 INFO:tasks.workunit.client.0.smithi090.stdout:6/676: dwrite d8/d1d/d1f/d3e/d44/f82 [0,4194304] 0 2024-04-24T08:25:07.115 INFO:tasks.workunit.client.1.smithi149.stdout:7/934: dwrite d9/d43/f7e [0,4194304] 0 2024-04-24T08:25:07.116 INFO:tasks.workunit.client.0.smithi090.stdout:5/685: dwrite d8/da/d3a/d2d/d37/f6a [0,4194304] 0 2024-04-24T08:25:07.118 INFO:tasks.workunit.client.0.smithi090.stdout:3/566: write d2/f2f [3231515,7721] 0 2024-04-24T08:25:07.119 INFO:tasks.workunit.client.0.smithi090.stdout:1/639: sync 2024-04-24T08:25:07.120 INFO:tasks.workunit.client.0.smithi090.stdout:6/677: write d8/d1d/d2f/f84 [418207,82351] 0 2024-04-24T08:25:07.122 INFO:tasks.workunit.client.1.smithi149.stdout:7/935: chown d9/dc/d2e/d3c/c125 27811 1 2024-04-24T08:25:07.131 INFO:tasks.workunit.client.1.smithi149.stdout:9/943: mknod d1/d2/dc8/dc9/c15b 0 2024-04-24T08:25:07.134 INFO:tasks.workunit.client.1.smithi149.stdout:8/870: dread d1/de/d18/f3e [0,4194304] 0 2024-04-24T08:25:07.136 INFO:tasks.workunit.client.0.smithi090.stdout:4/568: dwrite d3/df/d77/fa7 [0,4194304] 0 2024-04-24T08:25:07.136 INFO:tasks.workunit.client.0.smithi090.stdout:0/638: rmdir d1a/d4b/d53/d57 39 2024-04-24T08:25:07.137 INFO:tasks.workunit.client.1.smithi149.stdout:4/879: fsync d2/d18/d1e/d3c/d48/d69/f77 0 2024-04-24T08:25:07.138 INFO:tasks.workunit.client.0.smithi090.stdout:4/569: readlink d3/d87/l8a 0 2024-04-24T08:25:07.141 INFO:tasks.workunit.client.0.smithi090.stdout:2/686: dwrite d0/d14/d4d/db4/fe4 [0,4194304] 0 2024-04-24T08:25:07.142 INFO:tasks.workunit.client.1.smithi149.stdout:0/850: symlink d2/d9/df/d2f/d6c/dc1/d4e/d6f/l10f 0 2024-04-24T08:25:07.145 INFO:tasks.workunit.client.1.smithi149.stdout:7/936: mkdir d9/dc/d2e/d11e/d133 0 2024-04-24T08:25:07.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/686: mknod d8/da/d3a/d96/dcb/cdc 0 2024-04-24T08:25:07.151 INFO:tasks.workunit.client.1.smithi149.stdout:9/944: truncate d1/d99/fd7 775663 0 2024-04-24T08:25:07.152 INFO:tasks.workunit.client.1.smithi149.stdout:4/880: truncate d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf 901408 0 2024-04-24T08:25:07.155 INFO:tasks.workunit.client.0.smithi090.stdout:6/678: creat d8/d1d/d1f/d3e/d44/d90/da2/fcd x:0 0 0 2024-04-24T08:25:07.158 INFO:tasks.workunit.client.0.smithi090.stdout:6/679: fdatasync d8/dc/f10 0 2024-04-24T08:25:07.158 INFO:tasks.workunit.client.1.smithi149.stdout:8/871: mkdir d1/de/d18/daf/d129 0 2024-04-24T08:25:07.158 INFO:tasks.workunit.client.1.smithi149.stdout:7/937: rmdir d9/d43/d4f/d66/d96 39 2024-04-24T08:25:07.159 INFO:tasks.workunit.client.0.smithi090.stdout:6/680: fdatasync d8/dc/f72 0 2024-04-24T08:25:07.160 INFO:tasks.workunit.client.0.smithi090.stdout:7/703: write db/d17/d39/d59/f6d [7345006,32980] 0 2024-04-24T08:25:07.160 INFO:tasks.workunit.client.1.smithi149.stdout:6/792: truncate d0/d1a/d28/d31/d4c/d80/d6f/dd7/fed 1752226 0 2024-04-24T08:25:07.161 INFO:tasks.workunit.client.0.smithi090.stdout:8/600: dwrite d2/da/d13/d64/f3f [0,4194304] 0 2024-04-24T08:25:07.167 INFO:tasks.workunit.client.1.smithi149.stdout:0/851: creat d2/d9/df/d48/dba/f110 x:0 0 0 2024-04-24T08:25:07.170 INFO:tasks.workunit.client.1.smithi149.stdout:0/852: fsync d2/d9/d1a/f6b 0 2024-04-24T08:25:07.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/945: mknod d1/d2/d33/d60/d10b/c15c 0 2024-04-24T08:25:07.183 INFO:tasks.workunit.client.1.smithi149.stdout:4/881: rename d2/d18/db9/dd8/lbf to d2/d24/da7/l115 0 2024-04-24T08:25:07.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/968: dwrite d0/d1b/d5d/de5/f111 [0,4194304] 0 2024-04-24T08:25:07.190 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:06 smithi149 ceph-mon[31282]: pgmap v26: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 205 MiB/s wr, 408 op/s 2024-04-24T08:25:07.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/687: rename d0/d7/d67/d7f/d31/d46/d60/d65/d86 to d0/d7/d57/dea 0 2024-04-24T08:25:07.203 INFO:tasks.workunit.client.0.smithi090.stdout:2/688: write d0/d7/d67/fba [346896,5057] 0 2024-04-24T08:25:07.206 INFO:tasks.workunit.client.0.smithi090.stdout:2/689: write d0/d7/d6f/f29 [550069,32797] 0 2024-04-24T08:25:07.211 INFO:tasks.workunit.client.1.smithi149.stdout:6/793: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d66/d10c 0 2024-04-24T08:25:07.214 INFO:tasks.workunit.client.0.smithi090.stdout:1/640: link d3/d5/d58/f9f d3/d25/d30/dc5/fe8 0 2024-04-24T08:25:07.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/946: creat d1/de/d9d/f15d x:0 0 0 2024-04-24T08:25:07.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/681: fdatasync d8/dc/f11 0 2024-04-24T08:25:07.222 INFO:tasks.workunit.client.0.smithi090.stdout:2/690: dwrite d0/d14/d4d/d50/d96/f98 [0,4194304] 0 2024-04-24T08:25:07.223 INFO:tasks.workunit.client.1.smithi149.stdout:9/947: dread - d1/da/dd/d79/fa6 zero size 2024-04-24T08:25:07.223 INFO:tasks.workunit.client.1.smithi149.stdout:2/799: getdents d0/d6/db/d39/d3d/d58 0 2024-04-24T08:25:07.224 INFO:tasks.workunit.client.0.smithi090.stdout:8/601: truncate d2/da/d13/f5c 655174 0 2024-04-24T08:25:07.224 INFO:tasks.workunit.client.1.smithi149.stdout:4/882: rmdir d2/d18/d1e/d3c/d48 39 2024-04-24T08:25:07.226 INFO:tasks.workunit.client.1.smithi149.stdout:3/981: sync 2024-04-24T08:25:07.226 INFO:tasks.workunit.client.0.smithi090.stdout:8/602: chown d2/da 1 1 2024-04-24T08:25:07.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/691: chown d0/d14/d4d/db4/fe4 5 1 2024-04-24T08:25:07.227 INFO:tasks.workunit.client.0.smithi090.stdout:9/614: link d0/l27 d0/d19/lbe 0 2024-04-24T08:25:07.229 INFO:tasks.workunit.client.0.smithi090.stdout:3/567: dread d2/df/d21/f27 [0,4194304] 0 2024-04-24T08:25:07.230 INFO:tasks.workunit.client.1.smithi149.stdout:1/969: unlink d0/d5/db/d18/d4d/d50/c74 0 2024-04-24T08:25:07.231 INFO:tasks.workunit.client.0.smithi090.stdout:2/692: chown d0/d7/d67/d7f/d31/cb5 298 1 2024-04-24T08:25:07.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/948: chown d1/d2/d33/dea/d133/dad/db7/d12d/f13a 7522 1 2024-04-24T08:25:07.233 INFO:tasks.workunit.client.1.smithi149.stdout:0/853: symlink d2/d9/df/d2f/d6c/dce/l111 0 2024-04-24T08:25:07.234 INFO:tasks.workunit.client.0.smithi090.stdout:2/693: write d0/d7/d67/d7f/d31/d90/f9d [1022869,24480] 0 2024-04-24T08:25:07.237 INFO:tasks.workunit.client.0.smithi090.stdout:2/694: write d0/d14/f1d [1649537,27376] 0 2024-04-24T08:25:07.239 INFO:tasks.workunit.client.0.smithi090.stdout:6/682: dread d8/dc/fad [0,4194304] 0 2024-04-24T08:25:07.241 INFO:tasks.workunit.client.0.smithi090.stdout:1/641: dread - d3/d5/d3e/f9b zero size 2024-04-24T08:25:07.242 INFO:tasks.workunit.client.1.smithi149.stdout:2/800: truncate d0/d6/db/d28/f46 478765 0 2024-04-24T08:25:07.243 INFO:tasks.workunit.client.0.smithi090.stdout:4/570: write d3/d18/d5c/f66 [53834,23758] 0 2024-04-24T08:25:07.254 INFO:tasks.workunit.client.1.smithi149.stdout:7/938: dread d9/dc/d72/f8c [0,4194304] 0 2024-04-24T08:25:07.255 INFO:tasks.workunit.client.1.smithi149.stdout:8/872: creat d1/de/f12a x:0 0 0 2024-04-24T08:25:07.257 INFO:tasks.workunit.client.0.smithi090.stdout:0/639: creat d1a/d4b/d53/d79/fcf x:0 0 0 2024-04-24T08:25:07.261 INFO:tasks.workunit.client.0.smithi090.stdout:8/603: creat d2/d62/d83/da3/fd2 x:0 0 0 2024-04-24T08:25:07.262 INFO:tasks.workunit.client.1.smithi149.stdout:1/970: mkdir d0/d5/d4f/d13a 0 2024-04-24T08:25:07.262 INFO:tasks.workunit.client.0.smithi090.stdout:8/604: chown d2/d62/d71/l9e 525591269 1 2024-04-24T08:25:07.264 INFO:tasks.workunit.client.0.smithi090.stdout:8/605: write d2/d62/d71/fc9 [779454,57823] 0 2024-04-24T08:25:07.267 INFO:tasks.workunit.client.1.smithi149.stdout:0/854: unlink d2/d3c/d57/cf2 0 2024-04-24T08:25:07.270 INFO:tasks.workunit.client.1.smithi149.stdout:1/971: dread d0/d5/db/fcd [0,4194304] 0 2024-04-24T08:25:07.275 INFO:tasks.workunit.client.1.smithi149.stdout:2/801: truncate d0/d6/db/d39/d3d/f6b 2433701 0 2024-04-24T08:25:07.277 INFO:tasks.workunit.client.1.smithi149.stdout:4/883: write d2/d24/d2e/f4a [1493347,95294] 0 2024-04-24T08:25:07.277 INFO:tasks.workunit.client.0.smithi090.stdout:6/683: mknod d8/d1d/d2f/cce 0 2024-04-24T08:25:07.278 INFO:tasks.workunit.client.0.smithi090.stdout:6/684: chown d8/d1d/d1f/dc2 450790662 1 2024-04-24T08:25:07.281 INFO:tasks.workunit.client.0.smithi090.stdout:7/704: write db/d17/f85 [586202,66762] 0 2024-04-24T08:25:07.282 INFO:tasks.workunit.client.1.smithi149.stdout:7/939: truncate d9/dc/d72/fa1 1254413 0 2024-04-24T08:25:07.284 INFO:tasks.workunit.client.0.smithi090.stdout:4/571: unlink d3/d18/d5c/f95 0 2024-04-24T08:25:07.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/687: dwrite d8/d25/f26 [0,4194304] 0 2024-04-24T08:25:07.285 INFO:tasks.workunit.client.0.smithi090.stdout:7/705: fsync db/d17/d29/d32/f4c 0 2024-04-24T08:25:07.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/568: write d2/d25/d33/f81 [716954,77993] 0 2024-04-24T08:25:07.288 INFO:tasks.workunit.client.1.smithi149.stdout:0/855: mknod d2/d9/df/d56/dca/c112 0 2024-04-24T08:25:07.292 INFO:tasks.workunit.client.0.smithi090.stdout:9/615: symlink d0/d69/d7d/dbc/lbf 0 2024-04-24T08:25:07.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:06 smithi090 ceph-mon[25016]: pgmap v26: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 205 MiB/s wr, 408 op/s 2024-04-24T08:25:07.293 INFO:tasks.workunit.client.1.smithi149.stdout:4/884: mkdir d2/d10/d91/df7/d116 0 2024-04-24T08:25:07.296 INFO:tasks.workunit.client.1.smithi149.stdout:6/794: getdents d0/d1a/d28/d31/d4c/d4e/d62 0 2024-04-24T08:25:07.298 INFO:tasks.workunit.client.1.smithi149.stdout:8/873: mknod d1/de/d18/d32/d63/d9e/dec/c12b 0 2024-04-24T08:25:07.299 INFO:tasks.workunit.client.0.smithi090.stdout:6/685: dwrite d8/dc/d19/d8e/fa6 [0,4194304] 0 2024-04-24T08:25:07.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/686: chown d8/d1d/d2f/cce 48323992 1 2024-04-24T08:25:07.303 INFO:tasks.workunit.client.1.smithi149.stdout:7/940: fdatasync d9/dc/de/f12 0 2024-04-24T08:25:07.306 INFO:tasks.workunit.client.1.smithi149.stdout:1/972: write d0/f5f [2491243,84931] 0 2024-04-24T08:25:07.308 INFO:tasks.workunit.client.1.smithi149.stdout:0/856: creat d2/d3c/d57/f113 x:0 0 0 2024-04-24T08:25:07.312 INFO:tasks.workunit.client.1.smithi149.stdout:9/949: rename d1/d2/d15/d1c/l122 to d1/d2/l15e 0 2024-04-24T08:25:07.317 INFO:tasks.workunit.client.1.smithi149.stdout:4/885: read - d2/d10/f10e zero size 2024-04-24T08:25:07.319 INFO:tasks.workunit.client.0.smithi090.stdout:6/687: dwrite d8/dc/d19/d8e/fa6 [0,4194304] 0 2024-04-24T08:25:07.319 INFO:tasks.workunit.client.1.smithi149.stdout:6/795: unlink d0/fa 0 2024-04-24T08:25:07.322 INFO:tasks.workunit.client.1.smithi149.stdout:8/874: creat d1/de/d18/d43/d10e/f12c x:0 0 0 2024-04-24T08:25:07.326 INFO:tasks.workunit.client.0.smithi090.stdout:2/695: sync 2024-04-24T08:25:07.326 INFO:tasks.workunit.client.0.smithi090.stdout:5/688: creat d8/da/d3a/d96/dcb/d21/d4c/fdd x:0 0 0 2024-04-24T08:25:07.326 INFO:tasks.workunit.client.0.smithi090.stdout:9/616: mknod d0/d69/d7d/d5b/d92/cc0 0 2024-04-24T08:25:07.327 INFO:tasks.workunit.client.1.smithi149.stdout:0/857: dread - d2/d9/df/d56/f9c zero size 2024-04-24T08:25:07.328 INFO:tasks.workunit.client.0.smithi090.stdout:1/642: write d3/d5/da/f82 [375300,31803] 0 2024-04-24T08:25:07.329 INFO:tasks.workunit.client.1.smithi149.stdout:0/858: stat d2/d9/d69/cab 0 2024-04-24T08:25:07.329 INFO:tasks.workunit.client.0.smithi090.stdout:9/617: write d0/d19/f1e [7727373,122708] 0 2024-04-24T08:25:07.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/569: truncate d2/df/d1a/d37/d59/f61 1234537 0 2024-04-24T08:25:07.331 INFO:tasks.workunit.client.0.smithi090.stdout:9/618: stat d0/dc/f9c 0 2024-04-24T08:25:07.333 INFO:tasks.workunit.client.0.smithi090.stdout:1/643: truncate d3/d5/d58/fd9 299812 0 2024-04-24T08:25:07.333 INFO:tasks.workunit.client.0.smithi090.stdout:2/696: read d0/d7/d28/fbf [460172,106003] 0 2024-04-24T08:25:07.334 INFO:tasks.workunit.client.1.smithi149.stdout:6/796: dread d0/f4 [0,4194304] 0 2024-04-24T08:25:07.336 INFO:tasks.workunit.client.1.smithi149.stdout:3/982: rename d0/d5/da/d5e/l58 to d0/d5/d1a/d70/da3/l143 0 2024-04-24T08:25:07.339 INFO:tasks.workunit.client.0.smithi090.stdout:0/640: truncate d1a/d4b/f61 2863235 0 2024-04-24T08:25:07.344 INFO:tasks.workunit.client.0.smithi090.stdout:3/570: dread d2/f2f [0,4194304] 0 2024-04-24T08:25:07.346 INFO:tasks.workunit.client.0.smithi090.stdout:7/706: dwrite db/d17/d29/f8e [0,4194304] 0 2024-04-24T08:25:07.350 INFO:tasks.workunit.client.1.smithi149.stdout:0/859: symlink d2/d9/df/d2f/d6c/d72/l114 0 2024-04-24T08:25:07.360 INFO:tasks.workunit.client.1.smithi149.stdout:7/941: rename d9/dc/f6b to d9/dc/d72/f134 0 2024-04-24T08:25:07.365 INFO:tasks.workunit.client.1.smithi149.stdout:7/942: fdatasync d9/f8b 0 2024-04-24T08:25:07.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/886: dwrite d2/d24/d2e/d3f/d94/fdc [0,4194304] 0 2024-04-24T08:25:07.373 INFO:tasks.workunit.client.1.smithi149.stdout:6/797: write d0/f26 [127291,75337] 0 2024-04-24T08:25:07.374 INFO:tasks.workunit.client.0.smithi090.stdout:1/644: dread d3/d1d/d54/fbb [0,4194304] 0 2024-04-24T08:25:07.375 INFO:tasks.workunit.client.0.smithi090.stdout:1/645: chown d3/d5/f33 138 1 2024-04-24T08:25:07.377 INFO:tasks.workunit.client.1.smithi149.stdout:0/860: dwrite d2/f8 [4194304,4194304] 0 2024-04-24T08:25:07.379 INFO:tasks.workunit.client.1.smithi149.stdout:9/950: link d1/d2/f19 d1/d2/d15/d1c/dca/d118/f15f 0 2024-04-24T08:25:07.387 INFO:tasks.workunit.client.1.smithi149.stdout:8/875: dread d1/d1f/d5e/d9c/fb5 [4194304,4194304] 0 2024-04-24T08:25:07.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/973: getdents d0/ddd 0 2024-04-24T08:25:07.393 INFO:tasks.workunit.client.1.smithi149.stdout:6/798: creat d0/d1a/d28/d31/d38/d54/f10d x:0 0 0 2024-04-24T08:25:07.393 INFO:tasks.workunit.client.1.smithi149.stdout:8/876: readlink d1/de/d18/d32/d63/d76/la7 0 2024-04-24T08:25:07.394 INFO:tasks.workunit.client.1.smithi149.stdout:1/974: write d0/f25 [798341,106903] 0 2024-04-24T08:25:07.399 INFO:tasks.workunit.client.1.smithi149.stdout:9/951: fsync d1/da/d13/f1e 0 2024-04-24T08:25:07.408 INFO:tasks.workunit.client.1.smithi149.stdout:0/861: dwrite d2/d3c/dc7/f10a [0,4194304] 0 2024-04-24T08:25:07.408 INFO:tasks.workunit.client.1.smithi149.stdout:8/877: unlink d1/de/d18/d32/l106 0 2024-04-24T08:25:07.411 INFO:tasks.workunit.client.0.smithi090.stdout:5/689: chown d8/da/d14/d2f/d44/c83 42585 1 2024-04-24T08:25:07.412 INFO:tasks.workunit.client.0.smithi090.stdout:5/690: readlink d8/l1a 0 2024-04-24T08:25:07.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/691: chown d8/d25/cda 12562402 1 2024-04-24T08:25:07.417 INFO:tasks.workunit.client.1.smithi149.stdout:8/878: chown d1/de/d18/d43/d10e/d119 2967 1 2024-04-24T08:25:07.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/975: rename d0/ddf/da0/fe0 to d0/d1b/d5d/dd6/f13b 0 2024-04-24T08:25:07.417 INFO:tasks.workunit.client.1.smithi149.stdout:7/943: creat d9/d43/f135 x:0 0 0 2024-04-24T08:25:07.418 INFO:tasks.workunit.client.1.smithi149.stdout:9/952: mkdir d1/d2/d33/d60/d160 0 2024-04-24T08:25:07.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/619: rmdir d0/dc/d24/d7c/d4d/d32/d9f/d72/d77 39 2024-04-24T08:25:07.421 INFO:tasks.workunit.client.1.smithi149.stdout:3/983: write d0/d5/da/d19/d1b/d133/dd4/ff8 [157299,102717] 0 2024-04-24T08:25:07.425 INFO:tasks.workunit.client.0.smithi090.stdout:4/572: creat d3/d18/d1d/fab x:0 0 0 2024-04-24T08:25:07.427 INFO:tasks.workunit.client.1.smithi149.stdout:4/887: dread d2/d10/d1a/da2/fa3 [0,4194304] 0 2024-04-24T08:25:07.429 INFO:tasks.workunit.client.1.smithi149.stdout:4/888: dread - d2/d10/f10e zero size 2024-04-24T08:25:07.430 INFO:tasks.workunit.client.1.smithi149.stdout:3/984: read d0/d5/da/d19/d1b/d133/dd4/fde [236134,52746] 0 2024-04-24T08:25:07.433 INFO:tasks.workunit.client.1.smithi149.stdout:3/985: read d0/d5/d53/f47 [555484,73699] 0 2024-04-24T08:25:07.445 INFO:tasks.workunit.client.0.smithi090.stdout:1/646: mknod d3/d5/da/d94/ce9 0 2024-04-24T08:25:07.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/606: getdents d2/da/d13/d60/d76 0 2024-04-24T08:25:07.447 INFO:tasks.workunit.client.0.smithi090.stdout:1/647: readlink d3/d1d/d34/d73/dd8/le7 0 2024-04-24T08:25:07.449 INFO:tasks.workunit.client.0.smithi090.stdout:1/648: dread - d3/d5/d58/fe5 zero size 2024-04-24T08:25:07.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/607: write d2/da/d13/d64/d24/f5b [3803369,69029] 0 2024-04-24T08:25:07.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/692: creat d8/d59/dc7/d7a/da1/fde x:0 0 0 2024-04-24T08:25:07.452 INFO:tasks.workunit.client.1.smithi149.stdout:0/862: dread d2/d9/f89 [0,4194304] 0 2024-04-24T08:25:07.460 INFO:tasks.workunit.client.0.smithi090.stdout:6/688: creat d8/d1d/fcf x:0 0 0 2024-04-24T08:25:07.461 INFO:tasks.workunit.client.0.smithi090.stdout:3/571: mkdir d2/df/d1a/d37/dc0 0 2024-04-24T08:25:07.468 INFO:tasks.workunit.client.0.smithi090.stdout:2/697: creat d0/feb x:0 0 0 2024-04-24T08:25:07.471 INFO:tasks.workunit.client.0.smithi090.stdout:4/573: link d3/df/f9a d3/df/d69/d8b/fac 0 2024-04-24T08:25:07.474 INFO:tasks.workunit.client.0.smithi090.stdout:6/689: rename d8/d1d/d29/f6c to d8/d1d/d1f/d3e/d8f/fd0 0 2024-04-24T08:25:07.477 INFO:tasks.workunit.client.0.smithi090.stdout:1/649: mknod d3/d1d/d34/d73/dd8/cea 0 2024-04-24T08:25:07.481 INFO:tasks.workunit.client.0.smithi090.stdout:2/698: fsync d0/d7/d57/dea/fbe 0 2024-04-24T08:25:07.482 INFO:tasks.workunit.client.0.smithi090.stdout:7/707: getdents db/d17/d39/d86 0 2024-04-24T08:25:07.485 INFO:tasks.workunit.client.0.smithi090.stdout:4/574: creat d3/d38/d6b/fad x:0 0 0 2024-04-24T08:25:07.488 INFO:tasks.workunit.client.0.smithi090.stdout:6/690: symlink d8/d1d/d29/ld1 0 2024-04-24T08:25:07.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/699: fsync d0/d7/f95 0 2024-04-24T08:25:07.493 INFO:tasks.workunit.client.0.smithi090.stdout:5/693: creat d8/d59/fdf x:0 0 0 2024-04-24T08:25:07.494 INFO:tasks.workunit.client.0.smithi090.stdout:9/620: dwrite d0/dc/d24/d2d/f3f [0,4194304] 0 2024-04-24T08:25:07.495 INFO:tasks.workunit.client.0.smithi090.stdout:8/608: dwrite d2/da/f82 [0,4194304] 0 2024-04-24T08:25:07.495 INFO:tasks.workunit.client.0.smithi090.stdout:1/650: fsync d3/d25/d30/f7e 0 2024-04-24T08:25:07.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/572: truncate d2/df/d1a/d37/f55 731432 0 2024-04-24T08:25:07.499 INFO:tasks.workunit.client.0.smithi090.stdout:0/641: dwrite d1a/d4b/d53/d79/d58/d96/fba [0,4194304] 0 2024-04-24T08:25:07.511 INFO:tasks.workunit.client.0.smithi090.stdout:7/708: creat db/d17/d39/d59/ff0 x:0 0 0 2024-04-24T08:25:07.514 INFO:tasks.workunit.client.1.smithi149.stdout:1/976: truncate d0/ddf/d53/d61/f70 1500377 0 2024-04-24T08:25:07.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/575: creat d3/d38/d49/d6f/fae x:0 0 0 2024-04-24T08:25:07.516 INFO:tasks.workunit.client.1.smithi149.stdout:7/944: dread - d9/dc/d11/d83/da6/d67/fd8 zero size 2024-04-24T08:25:07.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/621: creat d0/d69/d7d/d5b/d92/fc1 x:0 0 0 2024-04-24T08:25:07.521 INFO:tasks.workunit.client.0.smithi090.stdout:1/651: dwrite d3/d5/da/d97/d9c/fdd [0,4194304] 0 2024-04-24T08:25:07.522 INFO:tasks.workunit.client.0.smithi090.stdout:9/622: write d0/d2a/faa [4405268,39622] 0 2024-04-24T08:25:07.523 INFO:tasks.workunit.client.0.smithi090.stdout:8/609: mknod d2/da/d13/d60/cd3 0 2024-04-24T08:25:07.527 INFO:tasks.workunit.client.1.smithi149.stdout:3/986: creat d0/d5/da/d5a/d63/d6a/d10e/f144 x:0 0 0 2024-04-24T08:25:07.528 INFO:tasks.workunit.client.0.smithi090.stdout:7/709: creat db/d17/d29/d38/d3e/ff1 x:0 0 0 2024-04-24T08:25:07.529 INFO:tasks.workunit.client.0.smithi090.stdout:7/710: chown db/c89 7639348 1 2024-04-24T08:25:07.535 INFO:tasks.workunit.client.1.smithi149.stdout:9/953: symlink d1/d4c/d91/l161 0 2024-04-24T08:25:07.538 INFO:tasks.workunit.client.1.smithi149.stdout:8/879: unlink d1/d1f/d22/l103 0 2024-04-24T08:25:07.539 INFO:tasks.workunit.client.1.smithi149.stdout:9/954: write d1/da/d13/d67/de4/fed [4945210,53371] 0 2024-04-24T08:25:07.540 INFO:tasks.workunit.client.0.smithi090.stdout:3/573: fdatasync d2/f24 0 2024-04-24T08:25:07.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/977: unlink d0/d1b/d5d/c11e 0 2024-04-24T08:25:07.545 INFO:tasks.workunit.client.0.smithi090.stdout:8/610: dwrite d2/d62/d83/da3/fd2 [0,4194304] 0 2024-04-24T08:25:07.547 INFO:tasks.workunit.client.1.smithi149.stdout:4/889: mknod d2/d18/c117 0 2024-04-24T08:25:07.550 INFO:tasks.workunit.client.1.smithi149.stdout:3/987: truncate d0/d5/d53/d4c/fdb 708803 0 2024-04-24T08:25:07.555 INFO:tasks.workunit.client.1.smithi149.stdout:3/988: dread d0/d5/da/d19/d76/f9b [0,4194304] 0 2024-04-24T08:25:07.556 INFO:tasks.workunit.client.0.smithi090.stdout:7/711: dread db/d17/d29/d32/d5a/d75/f7f [0,4194304] 0 2024-04-24T08:25:07.561 INFO:tasks.workunit.client.1.smithi149.stdout:3/989: truncate d0/d5/d53/d4c/f13b 551737 0 2024-04-24T08:25:07.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/623: dread d0/f5 [0,4194304] 0 2024-04-24T08:25:07.563 INFO:tasks.workunit.client.1.smithi149.stdout:7/945: mknod d9/d43/d4f/d66/c136 0 2024-04-24T08:25:07.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/694: link d8/da/d3a/d96/dcb/l2a d8/d7c/le0 0 2024-04-24T08:25:07.568 INFO:tasks.workunit.client.1.smithi149.stdout:0/863: dwrite d2/d9/df/d2f/d6c/f9a [0,4194304] 0 2024-04-24T08:25:07.578 INFO:tasks.workunit.client.1.smithi149.stdout:9/955: creat d1/d2/d33/dea/d133/d147/f162 x:0 0 0 2024-04-24T08:25:07.578 INFO:tasks.workunit.client.0.smithi090.stdout:0/642: creat d1a/fd0 x:0 0 0 2024-04-24T08:25:07.580 INFO:tasks.workunit.client.1.smithi149.stdout:9/956: readlink d1/d2/d33/dea/d133/l90 0 2024-04-24T08:25:07.585 INFO:tasks.workunit.client.1.smithi149.stdout:8/880: dread d1/d1f/d25/f80 [0,4194304] 0 2024-04-24T08:25:07.586 INFO:tasks.workunit.client.1.smithi149.stdout:7/946: creat d9/dc/d11/d83/da6/d67/d69/d123/f137 x:0 0 0 2024-04-24T08:25:07.586 INFO:tasks.workunit.client.0.smithi090.stdout:3/574: mknod d2/d15/cc1 0 2024-04-24T08:25:07.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/611: rename d2/da/d13/d60/d91/lab to d2/d62/d83/da3/ld4 0 2024-04-24T08:25:07.593 INFO:tasks.workunit.client.0.smithi090.stdout:4/576: getdents d3 0 2024-04-24T08:25:07.594 INFO:tasks.workunit.client.1.smithi149.stdout:0/864: link d2/d3c/f61 d2/d17/d41/d68/d108/f115 0 2024-04-24T08:25:07.596 INFO:tasks.workunit.client.1.smithi149.stdout:1/978: getdents d0/d1b/d5d/dd6/ddb 0 2024-04-24T08:25:07.598 INFO:tasks.workunit.client.0.smithi090.stdout:5/695: symlink d8/da/d3a/d96/dcb/d21/le1 0 2024-04-24T08:25:07.599 INFO:tasks.workunit.client.1.smithi149.stdout:8/881: creat d1/d1f/d25/d7a/f12d x:0 0 0 2024-04-24T08:25:07.601 INFO:tasks.workunit.client.1.smithi149.stdout:9/957: dwrite d1/d2/d15/f58 [0,4194304] 0 2024-04-24T08:25:07.606 INFO:tasks.workunit.client.0.smithi090.stdout:9/624: rename d0/dc/d24/d2d/d49/la8 to d0/d69/d7d/d5b/d92/lc2 0 2024-04-24T08:25:07.608 INFO:tasks.workunit.client.1.smithi149.stdout:9/958: readlink d1/d2/d33/dea/d133/dad/db7/d12d/l145 0 2024-04-24T08:25:07.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/612: rmdir d2/da/d13/d64/db2/dbb 39 2024-04-24T08:25:07.612 INFO:tasks.workunit.client.0.smithi090.stdout:4/577: read - d3/d24/f5d zero size 2024-04-24T08:25:07.613 INFO:tasks.workunit.client.1.smithi149.stdout:3/990: dwrite d0/d5/da/d19/d1b/d72/fbb [4194304,4194304] 0 2024-04-24T08:25:07.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/712: dwrite db/d17/d39/f41 [0,4194304] 0 2024-04-24T08:25:07.620 INFO:tasks.workunit.client.1.smithi149.stdout:8/882: unlink d1/de/d18/d32/l53 0 2024-04-24T08:25:07.625 INFO:tasks.workunit.client.0.smithi090.stdout:3/575: write d2/d15/d36/d6f/d7a/f8f [3629032,31394] 0 2024-04-24T08:25:07.627 INFO:tasks.workunit.client.0.smithi090.stdout:3/576: dread - d2/df/fa5 zero size 2024-04-24T08:25:07.633 INFO:tasks.workunit.client.1.smithi149.stdout:3/991: creat d0/d5/d1a/f145 x:0 0 0 2024-04-24T08:25:07.634 INFO:tasks.workunit.client.0.smithi090.stdout:8/613: rename d2/da/d13/d34/l55 to d2/da/db0/ld5 0 2024-04-24T08:25:07.638 INFO:tasks.workunit.client.1.smithi149.stdout:1/979: rename d0/d1b/db5/d108 to d0/d5/db/d63/d13c 0 2024-04-24T08:25:07.639 INFO:tasks.workunit.client.0.smithi090.stdout:0/643: link d1a/d4b/d53/l65 d1a/d4b/d53/d79/d69/d90/ld1 0 2024-04-24T08:25:07.642 INFO:tasks.workunit.client.0.smithi090.stdout:7/713: creat db/d17/d29/d38/d3e/ff2 x:0 0 0 2024-04-24T08:25:07.643 INFO:tasks.workunit.client.1.smithi149.stdout:1/980: write d0/d1b/d2a/ff9 [990594,28950] 0 2024-04-24T08:25:07.644 INFO:tasks.workunit.client.0.smithi090.stdout:9/625: dread d0/d2a/faa [0,4194304] 0 2024-04-24T08:25:07.645 INFO:tasks.workunit.client.0.smithi090.stdout:5/696: symlink d8/da/d3a/d96/dcb/d21/le2 0 2024-04-24T08:25:07.646 INFO:tasks.workunit.client.1.smithi149.stdout:3/992: read d0/d5/da/d5a/d63/d6a/f7c [1373118,102003] 0 2024-04-24T08:25:07.649 INFO:tasks.workunit.client.1.smithi149.stdout:0/865: dread d2/d9/d1a/fbc [0,4194304] 0 2024-04-24T08:25:07.650 INFO:tasks.workunit.client.1.smithi149.stdout:7/947: rename d9/dc/d11/c34 to d9/d43/d4f/d66/d96/c138 0 2024-04-24T08:25:07.654 INFO:tasks.workunit.client.1.smithi149.stdout:9/959: dread d1/da/f2c [0,4194304] 0 2024-04-24T08:25:07.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/614: mknod d2/da/d13/d60/d76/cd6 0 2024-04-24T08:25:07.655 INFO:tasks.workunit.client.1.smithi149.stdout:3/993: fsync d0/d5/d1a/f8f 0 2024-04-24T08:25:07.658 INFO:tasks.workunit.client.1.smithi149.stdout:3/994: fdatasync d0/d5/da/d59/d79/f9e 0 2024-04-24T08:25:07.661 INFO:tasks.workunit.client.1.smithi149.stdout:1/981: creat d0/d1b/d5d/dd6/ddb/d127/d12d/f13d x:0 0 0 2024-04-24T08:25:07.664 INFO:tasks.workunit.client.0.smithi090.stdout:9/626: mknod d0/dc/d24/d7c/d4d/d32/d9f/d78/cc3 0 2024-04-24T08:25:07.665 INFO:tasks.workunit.client.0.smithi090.stdout:4/578: dwrite d3/d38/f60 [4194304,4194304] 0 2024-04-24T08:25:07.665 INFO:tasks.workunit.client.1.smithi149.stdout:7/948: creat d9/dc/d11/d26/f139 x:0 0 0 2024-04-24T08:25:07.666 INFO:tasks.workunit.client.0.smithi090.stdout:9/627: chown d0/d69/d7d/f5c 45030 1 2024-04-24T08:25:07.669 INFO:tasks.workunit.client.0.smithi090.stdout:5/697: mkdir d8/da/d3a/d96/dcb/de3 0 2024-04-24T08:25:07.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/615: dread - d2/da/d13/d64/f8c zero size 2024-04-24T08:25:07.673 INFO:tasks.workunit.client.1.smithi149.stdout:9/960: dwrite d1/d2/d33/dea/d133/dad/db7/d12d/f13a [0,4194304] 0 2024-04-24T08:25:07.674 INFO:tasks.workunit.client.0.smithi090.stdout:8/616: chown d2/da/d13/d64/d24/db3/db4/dcd 134 1 2024-04-24T08:25:07.675 INFO:tasks.workunit.client.0.smithi090.stdout:6/691: sync 2024-04-24T08:25:07.676 INFO:tasks.workunit.client.0.smithi090.stdout:0/644: symlink d1a/d4b/d53/d79/d69/d90/dcc/ld2 0 2024-04-24T08:25:07.679 INFO:tasks.workunit.client.0.smithi090.stdout:7/714: creat db/d90/ff3 x:0 0 0 2024-04-24T08:25:07.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/883: rename d1/c99 to d1/de/d18/d32/d63/d76/c12e 0 2024-04-24T08:25:07.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/866: dwrite d2/d9/df/d40/f67 [0,4194304] 0 2024-04-24T08:25:07.686 INFO:tasks.workunit.client.1.smithi149.stdout:3/995: symlink d0/d5/da/d59/d79/l146 0 2024-04-24T08:25:07.690 INFO:tasks.workunit.client.0.smithi090.stdout:4/579: symlink d3/d24/d92/laf 0 2024-04-24T08:25:07.691 INFO:tasks.workunit.client.0.smithi090.stdout:4/580: chown d3/d18 1224 1 2024-04-24T08:25:07.692 INFO:tasks.workunit.client.0.smithi090.stdout:9/628: unlink d0/d19/fa3 0 2024-04-24T08:25:07.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/698: mknod d8/da/d3a/d2d/d37/ce4 0 2024-04-24T08:25:07.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/629: stat d0/dc/d24/d7c/d4d/d32/d9f/c6a 0 2024-04-24T08:25:07.698 INFO:tasks.workunit.client.0.smithi090.stdout:7/715: dwrite db/d17/d39/d86/dae/fb0 [0,4194304] 0 2024-04-24T08:25:07.699 INFO:tasks.workunit.client.1.smithi149.stdout:7/949: dread d9/d43/d64/f73 [0,4194304] 0 2024-04-24T08:25:07.701 INFO:tasks.workunit.client.0.smithi090.stdout:5/699: truncate d8/da/d3a/fbd 886766 0 2024-04-24T08:25:07.705 INFO:tasks.workunit.client.1.smithi149.stdout:3/996: fsync d0/d5/da/d19/d1b/d133/d38/f7e 0 2024-04-24T08:25:07.705 INFO:tasks.workunit.client.0.smithi090.stdout:0/645: creat d1a/d4b/d53/d79/d58/fd3 x:0 0 0 2024-04-24T08:25:07.708 INFO:tasks.workunit.client.1.smithi149.stdout:0/867: rename d2/d9/df/d2f/d6c/dc1/f79 to d2/d9/df/d2f/d6c/d72/f116 0 2024-04-24T08:25:07.708 INFO:tasks.workunit.client.1.smithi149.stdout:3/997: fsync d0/d5/da/d5e/f136 0 2024-04-24T08:25:07.708 INFO:tasks.workunit.client.0.smithi090.stdout:7/716: dread db/d17/d29/d32/d5a/d75/fa4 [0,4194304] 0 2024-04-24T08:25:07.710 INFO:tasks.workunit.client.1.smithi149.stdout:0/868: chown d2/d17/fb9 2690 1 2024-04-24T08:25:07.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/581: dwrite d3/d18/d1d/fab [0,4194304] 0 2024-04-24T08:25:07.716 INFO:tasks.workunit.client.1.smithi149.stdout:7/950: fsync d9/dc/de/d44/fab 0 2024-04-24T08:25:07.717 INFO:tasks.workunit.client.0.smithi090.stdout:3/577: getdents d2/d90 0 2024-04-24T08:25:07.718 INFO:tasks.workunit.client.1.smithi149.stdout:0/869: read d2/d9/d69/fa7 [2815385,9052] 0 2024-04-24T08:25:07.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/578: stat d2/df/d1a/d37/d53/d60/cbc 0 2024-04-24T08:25:07.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/579: chown d2/df 511909595 1 2024-04-24T08:25:07.720 INFO:tasks.workunit.client.1.smithi149.stdout:8/884: dread d1/de/d18/fc7 [0,4194304] 0 2024-04-24T08:25:07.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/630: creat d0/d69/d7d/fc4 x:0 0 0 2024-04-24T08:25:07.725 INFO:tasks.workunit.client.1.smithi149.stdout:8/885: truncate d1/de/d18/d32/f10a 174444 0 2024-04-24T08:25:07.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/692: mkdir d8/dc/d19/dd2 0 2024-04-24T08:25:07.734 INFO:tasks.workunit.client.1.smithi149.stdout:7/951: fsync d9/dc/de/d44/f74 0 2024-04-24T08:25:07.742 INFO:tasks.workunit.client.0.smithi090.stdout:4/582: creat d3/d24/d92/fb0 x:0 0 0 2024-04-24T08:25:07.742 INFO:tasks.workunit.client.1.smithi149.stdout:0/870: getdents d2/d9/df/d56/dca/d100 0 2024-04-24T08:25:07.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/646: write d1a/d4b/f85 [1059180,102991] 0 2024-04-24T08:25:07.746 INFO:tasks.workunit.client.1.smithi149.stdout:3/998: write d0/d5/da/fa7 [2730075,7214] 0 2024-04-24T08:25:07.747 INFO:tasks.workunit.client.0.smithi090.stdout:4/583: dread d3/d18/d5c/f66 [0,4194304] 0 2024-04-24T08:25:07.749 INFO:tasks.workunit.client.0.smithi090.stdout:1/652: sync 2024-04-24T08:25:07.749 INFO:tasks.workunit.client.0.smithi090.stdout:2/700: sync 2024-04-24T08:25:07.750 INFO:tasks.workunit.client.0.smithi090.stdout:6/693: chown d8/d1d/d2f/l50 5062 1 2024-04-24T08:25:07.750 INFO:tasks.workunit.client.0.smithi090.stdout:7/717: write db/d17/d29/d32/d5a/d75/f7f [579878,2698] 0 2024-04-24T08:25:07.751 INFO:tasks.workunit.client.1.smithi149.stdout:8/886: dread d1/de/d18/d32/d63/d9e/dec/f121 [0,4194304] 0 2024-04-24T08:25:07.751 INFO:tasks.workunit.client.0.smithi090.stdout:2/701: fdatasync d0/d7/d67/fba 0 2024-04-24T08:25:07.752 INFO:tasks.workunit.client.1.smithi149.stdout:7/952: write d9/dc/fb7 [1159623,51909] 0 2024-04-24T08:25:07.754 INFO:tasks.workunit.client.1.smithi149.stdout:7/953: stat d9/d43/d64/f73 0 2024-04-24T08:25:07.755 INFO:tasks.workunit.client.0.smithi090.stdout:8/617: getdents d2/da/db0 0 2024-04-24T08:25:07.756 INFO:tasks.workunit.client.0.smithi090.stdout:1/653: truncate d3/d25/d30/d9d/fd3 4628151 0 2024-04-24T08:25:07.757 INFO:tasks.workunit.client.0.smithi090.stdout:3/580: write d2/f24 [441689,66218] 0 2024-04-24T08:25:07.757 INFO:tasks.workunit.client.0.smithi090.stdout:9/631: mknod d0/dc/d24/cc5 0 2024-04-24T08:25:07.757 INFO:tasks.workunit.client.0.smithi090.stdout:2/702: rename d0/d14 to d0/d14/d17/dc2/dc9/dec 22 2024-04-24T08:25:07.758 INFO:tasks.workunit.client.0.smithi090.stdout:5/700: dwrite d8/d59/dc7/d7a/d50/f79 [0,4194304] 0 2024-04-24T08:25:07.762 INFO:tasks.workunit.client.0.smithi090.stdout:1/654: chown d3/d5/da/d94/la1 11306 1 2024-04-24T08:25:07.762 INFO:tasks.workunit.client.0.smithi090.stdout:8/618: readlink d2/l2e 0 2024-04-24T08:25:07.763 INFO:tasks.workunit.client.0.smithi090.stdout:7/718: creat db/d17/d29/d32/dd4/ff4 x:0 0 0 2024-04-24T08:25:07.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/954: mknod d9/dc/d11/d84/c13a 0 2024-04-24T08:25:07.764 INFO:tasks.workunit.client.1.smithi149.stdout:0/871: dwrite d2/d9/d1a/d36/d85/ffa [0,4194304] 0 2024-04-24T08:25:07.765 INFO:tasks.workunit.client.0.smithi090.stdout:3/581: creat d2/d90/fc2 x:0 0 0 2024-04-24T08:25:07.767 INFO:tasks.workunit.client.0.smithi090.stdout:8/619: truncate d2/d62/d71/fa7 909344 0 2024-04-24T08:25:07.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/703: creat d0/d14/fed x:0 0 0 2024-04-24T08:25:07.769 INFO:tasks.workunit.client.0.smithi090.stdout:4/584: symlink d3/df/lb1 0 2024-04-24T08:25:07.770 INFO:tasks.workunit.client.0.smithi090.stdout:8/620: read d2/d62/d71/fa7 [52332,78661] 0 2024-04-24T08:25:07.772 INFO:tasks.workunit.client.1.smithi149.stdout:3/999: dwrite d0/d5/da/d59/fcf [0,4194304] 0 2024-04-24T08:25:07.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/632: mknod d0/dc/d24/d7c/d4d/cc6 0 2024-04-24T08:25:07.772 INFO:tasks.workunit.client.0.smithi090.stdout:8/621: chown d2/da/d13/d34/l58 5 1 2024-04-24T08:25:07.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/701: dread - d8/da/d3a/d2d/d37/f88 zero size 2024-04-24T08:25:07.773 INFO:tasks.workunit.client.0.smithi090.stdout:4/585: dread - d3/df/f98 zero size 2024-04-24T08:25:07.774 INFO:tasks.workunit.client.1.smithi149.stdout:0/872: dread - d2/d9/df/d2f/d6c/dc1/d4e/d6f/f8a zero size 2024-04-24T08:25:07.775 INFO:tasks.workunit.client.0.smithi090.stdout:1/655: mkdir d3/d5/da/dc/d80/deb 0 2024-04-24T08:25:07.775 INFO:tasks.workunit.client.0.smithi090.stdout:7/719: read db/f21 [1311483,24800] 0 2024-04-24T08:25:07.776 INFO:tasks.workunit.client.0.smithi090.stdout:3/582: mknod d2/d84/cc3 0 2024-04-24T08:25:07.778 INFO:tasks.workunit.client.0.smithi090.stdout:7/720: dread - db/dab/fd8 zero size 2024-04-24T08:25:07.790 INFO:tasks.workunit.client.0.smithi090.stdout:0/647: link d1a/f24 d1a/d4b/d53/d79/d35/d68/fd4 0 2024-04-24T08:25:07.793 INFO:tasks.workunit.client.0.smithi090.stdout:3/583: mkdir d2/d15/dc4 0 2024-04-24T08:25:07.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/704: dwrite d0/fc8 [0,4194304] 0 2024-04-24T08:25:07.802 INFO:tasks.workunit.client.0.smithi090.stdout:4/586: dread d3/d18/d1d/f27 [0,4194304] 0 2024-04-24T08:25:07.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/633: dwrite d0/dc/d24/d2d/d49/d62/fad [0,4194304] 0 2024-04-24T08:25:07.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/587: rename d3/d38 to d3/df/d69/d8b/db2 0 2024-04-24T08:25:07.813 INFO:tasks.workunit.client.0.smithi090.stdout:9/634: fsync d0/d2a/faa 0 2024-04-24T08:25:07.814 INFO:tasks.workunit.client.1.smithi149.stdout:7/955: write d9/dc/d2e/d3c/db8/fdb [351999,54922] 0 2024-04-24T08:25:07.816 INFO:tasks.workunit.client.0.smithi090.stdout:3/584: dwrite d2/d15/d36/d6f/d92/fac [0,4194304] 0 2024-04-24T08:25:07.819 INFO:tasks.workunit.client.0.smithi090.stdout:2/705: rename d0/d7/d67/d7f/d31/d46/d60/d65 to d0/d14/d36/dce/dee 0 2024-04-24T08:25:07.822 INFO:tasks.workunit.client.0.smithi090.stdout:9/635: mknod d0/dc/cc7 0 2024-04-24T08:25:07.823 INFO:tasks.workunit.client.1.smithi149.stdout:7/956: mkdir d9/dc/d11/d83/da6/d67/d69/dad/d131/d13b 0 2024-04-24T08:25:07.825 INFO:tasks.workunit.client.0.smithi090.stdout:3/585: symlink d2/df/d1a/lc5 0 2024-04-24T08:25:07.828 INFO:tasks.workunit.client.0.smithi090.stdout:2/706: unlink d0/d14/d4d/d50/c5a 0 2024-04-24T08:25:07.832 INFO:tasks.workunit.client.0.smithi090.stdout:8/622: dwrite d2/da/d13/d64/f8c [0,4194304] 0 2024-04-24T08:25:07.833 INFO:tasks.workunit.client.0.smithi090.stdout:9/636: unlink d0/dc/d24/d2d/f5d 0 2024-04-24T08:25:07.835 INFO:tasks.workunit.client.0.smithi090.stdout:3/586: creat d2/df/d21/fc6 x:0 0 0 2024-04-24T08:25:07.835 INFO:tasks.workunit.client.0.smithi090.stdout:1/656: write d3/d5/da/d97/d9c/fa9 [1024745,58954] 0 2024-04-24T08:25:07.838 INFO:tasks.workunit.client.0.smithi090.stdout:7/721: write db/d17/d29/d38/f4e [904372,16767] 0 2024-04-24T08:25:07.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/722: readlink db/d17/d39/d59/l71 0 2024-04-24T08:25:07.841 INFO:tasks.workunit.client.0.smithi090.stdout:1/657: chown d3/d5/da/dc/fe3 118754 1 2024-04-24T08:25:07.841 INFO:tasks.workunit.client.0.smithi090.stdout:0/648: write d1a/d4b/d53/d79/d82/f98 [906960,39226] 0 2024-04-24T08:25:07.842 INFO:tasks.workunit.client.0.smithi090.stdout:5/702: dwrite d8/da/f20 [0,4194304] 0 2024-04-24T08:25:07.842 INFO:tasks.workunit.client.0.smithi090.stdout:3/587: write d2/d15/d36/d6f/d92/fa9 [2079120,32191] 0 2024-04-24T08:25:07.848 INFO:tasks.workunit.client.0.smithi090.stdout:3/588: chown d2/d15/d36/d6f/d7a 88878 1 2024-04-24T08:25:07.851 INFO:tasks.workunit.client.0.smithi090.stdout:6/694: sync 2024-04-24T08:25:07.852 INFO:tasks.workunit.client.0.smithi090.stdout:0/649: creat d1a/d4b/d53/d79/d69/d8a/fd5 x:0 0 0 2024-04-24T08:25:07.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/589: chown d2/d15/d36/d6f/d92/fac 32341 1 2024-04-24T08:25:07.853 INFO:tasks.workunit.client.0.smithi090.stdout:1/658: creat d3/d1d/d34/d5a/fec x:0 0 0 2024-04-24T08:25:07.854 INFO:tasks.workunit.client.0.smithi090.stdout:8/623: creat d2/da/d13/d64/db2/fd7 x:0 0 0 2024-04-24T08:25:07.858 INFO:tasks.workunit.client.0.smithi090.stdout:2/707: getdents d0/d7/d28/daa/db1 0 2024-04-24T08:25:07.859 INFO:tasks.workunit.client.1.smithi149.stdout:7/957: dwrite d9/dc/d11/d26/f139 [0,4194304] 0 2024-04-24T08:25:07.859 INFO:tasks.workunit.client.0.smithi090.stdout:4/588: fsync d3/df/d69/d8b/db2/f60 0 2024-04-24T08:25:07.861 INFO:tasks.workunit.client.1.smithi149.stdout:7/958: readlink d9/dc/d11/d26/l36 0 2024-04-24T08:25:07.862 INFO:tasks.workunit.client.0.smithi090.stdout:9/637: dwrite d0/dc/f29 [4194304,4194304] 0 2024-04-24T08:25:07.863 INFO:tasks.workunit.client.0.smithi090.stdout:8/624: chown d2/da/l4f 668123 1 2024-04-24T08:25:07.864 INFO:tasks.workunit.client.0.smithi090.stdout:2/708: readlink d0/d14/d36/d77/ld0 0 2024-04-24T08:25:07.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/659: dread - d3/d5/da/dc/d80/fe4 zero size 2024-04-24T08:25:07.869 INFO:tasks.workunit.client.0.smithi090.stdout:0/650: creat d1a/d4b/d53/d79/d69/d8a/fd6 x:0 0 0 2024-04-24T08:25:07.869 INFO:tasks.workunit.client.0.smithi090.stdout:9/638: stat d0/d69/d7d/d5b/d92 0 2024-04-24T08:25:07.870 INFO:tasks.workunit.client.0.smithi090.stdout:7/723: getdents db 0 2024-04-24T08:25:07.873 INFO:tasks.workunit.client.1.smithi149.stdout:7/959: truncate d9/dc/de/d44/fd5 344246 0 2024-04-24T08:25:07.874 INFO:tasks.workunit.client.0.smithi090.stdout:6/695: creat d8/dc/d19/d8e/fd3 x:0 0 0 2024-04-24T08:25:07.874 INFO:tasks.workunit.client.0.smithi090.stdout:8/625: mkdir d2/d62/d6b/d9c/dd8 0 2024-04-24T08:25:07.876 INFO:tasks.workunit.client.1.smithi149.stdout:7/960: write d9/dc/de/d44/f74 [4440412,98312] 0 2024-04-24T08:25:07.881 INFO:tasks.workunit.client.0.smithi090.stdout:7/724: dread - db/d17/d29/d38/d3e/fcb zero size 2024-04-24T08:25:07.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/709: dread d0/d7/de1/f43 [0,4194304] 0 2024-04-24T08:25:07.884 INFO:tasks.workunit.client.0.smithi090.stdout:9/639: creat d0/dc/d24/d7c/d4d/d32/fc8 x:0 0 0 2024-04-24T08:25:07.884 INFO:tasks.workunit.client.0.smithi090.stdout:8/626: chown d2/da/d13/d64/d50/lac 293841 1 2024-04-24T08:25:07.884 INFO:tasks.workunit.client.1.smithi149.stdout:7/961: read d9/dc/d11/d26/fae [3615193,104687] 0 2024-04-24T08:25:07.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/703: dwrite d8/d59/dc7/d7a/da1/dae/fc9 [0,4194304] 0 2024-04-24T08:25:07.887 INFO:tasks.workunit.client.0.smithi090.stdout:7/725: creat db/de9/ff5 x:0 0 0 2024-04-24T08:25:07.887 INFO:tasks.workunit.client.0.smithi090.stdout:9/640: chown d0/l8 0 1 2024-04-24T08:25:07.900 INFO:tasks.workunit.client.0.smithi090.stdout:7/726: dwrite db/dab/fd8 [0,4194304] 0 2024-04-24T08:25:07.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/727: readlink db/d17/d29/d38/lcc 0 2024-04-24T08:25:07.907 INFO:tasks.workunit.client.1.smithi149.stdout:7/962: creat d9/dc/d2e/d11e/dac/f13c x:0 0 0 2024-04-24T08:25:07.908 INFO:tasks.workunit.client.0.smithi090.stdout:8/627: fdatasync d2/da/d13/d34/f3d 0 2024-04-24T08:25:07.911 INFO:tasks.workunit.client.0.smithi090.stdout:2/710: rename d0/d14/d4d to d0/d7/d28/daa/def 0 2024-04-24T08:25:07.916 INFO:tasks.workunit.client.1.smithi149.stdout:7/963: mkdir d9/dc/d2e/d11e/dac/d103/d13d 0 2024-04-24T08:25:07.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/590: write d2/df/d1a/d37/f55 [840261,45720] 0 2024-04-24T08:25:07.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/628: rmdir d2/da/d13/d60 39 2024-04-24T08:25:07.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/964: chown d9/d43/d64/f73 3 1 2024-04-24T08:25:07.921 INFO:tasks.workunit.client.0.smithi090.stdout:5/704: rename d8/da/d3a/d96/dcb/d21/d4c/f9a to d8/d59/dc7/d7a/fe5 0 2024-04-24T08:25:07.923 INFO:tasks.workunit.client.0.smithi090.stdout:2/711: link d0/d14/d17/f18 d0/d7/d28/daa/db1/ff0 0 2024-04-24T08:25:07.924 INFO:tasks.workunit.client.0.smithi090.stdout:0/651: write d1a/f1e [4369440,112703] 0 2024-04-24T08:25:07.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/965: rename d9/dc/d2e/d11e/dac/f13c to d9/dc/d2e/d51/f13e 0 2024-04-24T08:25:07.927 INFO:tasks.workunit.client.0.smithi090.stdout:7/728: link db/d17/d29/d38/d3e/fe7 db/d54/ddb/ff6 0 2024-04-24T08:25:07.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/696: truncate d8/dc/d19/f3d 4038269 0 2024-04-24T08:25:07.930 INFO:tasks.workunit.client.0.smithi090.stdout:9/641: dread d0/dc/d24/d7c/f36 [0,4194304] 0 2024-04-24T08:25:07.932 INFO:tasks.workunit.client.0.smithi090.stdout:5/705: creat d8/d59/dc7/d7a/da1/dae/fe6 x:0 0 0 2024-04-24T08:25:07.932 INFO:tasks.workunit.client.0.smithi090.stdout:4/589: dwrite d3/df/d69/d8b/db2/f42 [4194304,4194304] 0 2024-04-24T08:25:07.933 INFO:tasks.workunit.client.1.smithi149.stdout:7/966: mknod d9/dc/d11/c13f 0 2024-04-24T08:25:07.935 INFO:tasks.workunit.client.1.smithi149.stdout:7/967: chown d9/dc/d2e/d3c/l63 254840894 1 2024-04-24T08:25:07.938 INFO:tasks.workunit.client.0.smithi090.stdout:6/697: chown d8/d1d/d1f/d3e/d44/d90/fc7 128416328 1 2024-04-24T08:25:07.939 INFO:tasks.workunit.client.0.smithi090.stdout:8/629: link d2/da/d13/d64/f3f d2/da/d13/d64/db2/fd9 0 2024-04-24T08:25:07.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/968: fsync d9/dc/d11/d26/f9b 0 2024-04-24T08:25:07.941 INFO:tasks.workunit.client.0.smithi090.stdout:2/712: truncate d0/d7/d67/d7f/f78 1139224 0 2024-04-24T08:25:07.944 INFO:tasks.workunit.client.0.smithi090.stdout:3/591: dwrite d2/d25/f7c [0,4194304] 0 2024-04-24T08:25:07.950 INFO:tasks.workunit.client.0.smithi090.stdout:9/642: truncate d0/d19/f56 2586425 0 2024-04-24T08:25:07.955 INFO:tasks.workunit.client.0.smithi090.stdout:9/643: write d0/dc/d24/d7c/d4d/d32/d9f/f9b [4037326,127507] 0 2024-04-24T08:25:07.956 INFO:tasks.workunit.client.1.smithi149.stdout:7/969: link d9/dc/d2e/d3c/db8/fdb d9/dc/d11/d83/da6/f140 0 2024-04-24T08:25:07.958 INFO:tasks.workunit.client.0.smithi090.stdout:7/729: rename db/d17/d29/d32/d5a/d75/ccd to db/d17/d29/d38/d3e/cf7 0 2024-04-24T08:25:07.959 INFO:tasks.workunit.client.0.smithi090.stdout:3/592: symlink d2/df/d21/lc7 0 2024-04-24T08:25:07.961 INFO:tasks.workunit.client.0.smithi090.stdout:8/630: getdents d2/d62/d6b/d9c/dd8 0 2024-04-24T08:25:07.961 INFO:tasks.workunit.client.1.smithi149.stdout:7/970: creat d9/dc/d2e/d11e/d133/f141 x:0 0 0 2024-04-24T08:25:07.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/730: readlink db/d17/d39/d86/la7 0 2024-04-24T08:25:07.965 INFO:tasks.workunit.client.0.smithi090.stdout:6/698: creat d8/d1d/d1f/d8c/fd4 x:0 0 0 2024-04-24T08:25:07.965 INFO:tasks.workunit.client.0.smithi090.stdout:9/644: mknod d0/d69/d7d/d5b/cc9 0 2024-04-24T08:25:07.967 INFO:tasks.workunit.client.1.smithi149.stdout:7/971: unlink d9/dc/d11/d83/da6/d67/lee 0 2024-04-24T08:25:07.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/713: dwrite d0/feb [0,4194304] 0 2024-04-24T08:25:07.970 INFO:tasks.workunit.client.0.smithi090.stdout:3/593: mkdir d2/d25/d33/dc8 0 2024-04-24T08:25:07.976 INFO:tasks.workunit.client.0.smithi090.stdout:9/645: truncate d0/dc/d24/d2d/d49/fb3 4845414 0 2024-04-24T08:25:07.981 INFO:tasks.workunit.client.0.smithi090.stdout:3/594: creat d2/d15/d36/d6f/d7a/fc9 x:0 0 0 2024-04-24T08:25:07.982 INFO:tasks.workunit.client.0.smithi090.stdout:7/731: creat db/d17/d29/d32/d5a/ff8 x:0 0 0 2024-04-24T08:25:07.985 INFO:tasks.workunit.client.0.smithi090.stdout:9/646: truncate d0/f53 4173501 0 2024-04-24T08:25:07.987 INFO:tasks.workunit.client.0.smithi090.stdout:7/732: readlink db/d17/d39/d59/l74 0 2024-04-24T08:25:07.990 INFO:tasks.workunit.client.0.smithi090.stdout:8/631: truncate d2/da/d13/d64/f86 2761792 0 2024-04-24T08:25:07.990 INFO:tasks.workunit.client.0.smithi090.stdout:4/590: dwrite d3/d24/f5d [0,4194304] 0 2024-04-24T08:25:07.990 INFO:tasks.workunit.client.0.smithi090.stdout:3/595: read d2/df/d1a/d37/f49 [1892508,21536] 0 2024-04-24T08:25:07.991 INFO:tasks.workunit.client.0.smithi090.stdout:5/706: dread d8/d59/dc7/d7a/d50/f5e [0,4194304] 0 2024-04-24T08:25:07.993 INFO:tasks.workunit.client.0.smithi090.stdout:6/699: dread d8/dc/d19/d8e/d38/f49 [0,4194304] 0 2024-04-24T08:25:07.993 INFO:tasks.workunit.client.0.smithi090.stdout:9/647: write d0/dc/d24/d2d/d49/fb3 [30786,128189] 0 2024-04-24T08:25:07.995 INFO:tasks.workunit.client.0.smithi090.stdout:3/596: read - d2/d90/fa1 zero size 2024-04-24T08:25:07.996 INFO:tasks.workunit.client.1.smithi149.stdout:7/972: dwrite d9/dc/d11/d83/da6/f140 [0,4194304] 0 2024-04-24T08:25:07.997 INFO:tasks.workunit.client.0.smithi090.stdout:4/591: mknod d3/d25/cb3 0 2024-04-24T08:25:08.000 INFO:tasks.workunit.client.0.smithi090.stdout:8/632: dread - d2/da/d13/d60/d76/fca zero size 2024-04-24T08:25:08.001 INFO:tasks.workunit.client.0.smithi090.stdout:9/648: chown d0/dc/d24/d7c/d4d/f52 40 1 2024-04-24T08:25:08.001 INFO:tasks.workunit.client.0.smithi090.stdout:1/660: sync 2024-04-24T08:25:08.003 INFO:tasks.workunit.client.0.smithi090.stdout:2/714: dwrite d0/d2e/f37 [0,4194304] 0 2024-04-24T08:25:08.003 INFO:tasks.workunit.client.0.smithi090.stdout:5/707: rename d8/da/d14/l95 to d8/d76/dad/le7 0 2024-04-24T08:25:08.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/649: write d0/d69/d7d/fc4 [945546,29159] 0 2024-04-24T08:25:08.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/715: chown d0/d7/d67/d7f/d31/ddb 5 1 2024-04-24T08:25:08.007 INFO:tasks.workunit.client.0.smithi090.stdout:2/716: fsync d0/d14/d17/f62 0 2024-04-24T08:25:08.009 INFO:tasks.workunit.client.0.smithi090.stdout:3/597: mkdir d2/df/d1a/d37/d53/d60/d9b/dca 0 2024-04-24T08:25:08.012 INFO:tasks.workunit.client.0.smithi090.stdout:3/598: chown d2/df/d1a/d37/d53/c62 29761348 1 2024-04-24T08:25:08.013 INFO:tasks.workunit.client.1.smithi149.stdout:7/973: mkdir d9/dc/de/d142 0 2024-04-24T08:25:08.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/592: creat d3/df/d69/d8b/db2/d6b/fb4 x:0 0 0 2024-04-24T08:25:08.017 INFO:tasks.workunit.client.0.smithi090.stdout:5/708: mknod d8/d7c/ce8 0 2024-04-24T08:25:08.036 INFO:tasks.workunit.client.0.smithi090.stdout:9/650: symlink d0/da5/lca 0 2024-04-24T08:25:08.037 INFO:tasks.workunit.client.1.smithi149.stdout:7/974: dwrite d9/d43/f7e [4194304,4194304] 0 2024-04-24T08:25:08.039 INFO:tasks.workunit.client.0.smithi090.stdout:9/651: read - d0/dc/d24/d7c/d4d/d32/d9f/d78/f91 zero size 2024-04-24T08:25:08.042 INFO:tasks.workunit.client.0.smithi090.stdout:5/709: rename d8/d59/dc7/d7a/fe5 to d8/da/d14/fe9 0 2024-04-24T08:25:08.042 INFO:tasks.workunit.client.0.smithi090.stdout:9/652: readlink d0/dc/d24/d7c/d4d/l70 0 2024-04-24T08:25:08.042 INFO:tasks.workunit.client.0.smithi090.stdout:2/717: getdents d0/d14/d17/dc2/dc9 0 2024-04-24T08:25:08.044 INFO:tasks.workunit.client.0.smithi090.stdout:6/700: link d8/dc/d19/l2d d8/d1d/d2f/d85/ld5 0 2024-04-24T08:25:08.044 INFO:tasks.workunit.client.0.smithi090.stdout:9/653: chown d0/dc/fd 99778871 1 2024-04-24T08:25:08.046 INFO:tasks.workunit.client.0.smithi090.stdout:8/633: dwrite d2/f59 [0,4194304] 0 2024-04-24T08:25:08.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/710: symlink d8/d59/dc7/d7a/da1/dae/lea 0 2024-04-24T08:25:08.052 INFO:tasks.workunit.client.0.smithi090.stdout:9/654: mkdir d0/dc/d24/d2d/d49/d62/dcb 0 2024-04-24T08:25:08.058 INFO:tasks.workunit.client.0.smithi090.stdout:4/593: rename d3/df to d3/d18/d5c/db5 0 2024-04-24T08:25:08.058 INFO:tasks.workunit.client.0.smithi090.stdout:6/701: creat d8/d1d/d1f/d3e/fd6 x:0 0 0 2024-04-24T08:25:08.060 INFO:tasks.workunit.client.0.smithi090.stdout:5/711: chown d8/d59/dc7/d7a/d50/c77 131509044 1 2024-04-24T08:25:08.061 INFO:tasks.workunit.client.0.smithi090.stdout:9/655: mknod d0/da5/ccc 0 2024-04-24T08:25:08.062 INFO:tasks.workunit.client.0.smithi090.stdout:3/599: write d2/df/f19 [793548,69689] 0 2024-04-24T08:25:08.067 INFO:tasks.workunit.client.0.smithi090.stdout:0/652: sync 2024-04-24T08:25:08.067 INFO:tasks.workunit.client.0.smithi090.stdout:2/718: dwrite d0/d14/d36/dce/dee/fcc [0,4194304] 0 2024-04-24T08:25:08.069 INFO:tasks.workunit.client.0.smithi090.stdout:9/656: truncate d0/dc/d24/d7c/f86 716991 0 2024-04-24T08:25:08.070 INFO:tasks.workunit.client.0.smithi090.stdout:9/657: stat d0/l8 0 2024-04-24T08:25:08.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/712: rename d8/da/d3a/d2d/lc1 to d8/da/d3a/d96/dcb/d21/d4c/leb 0 2024-04-24T08:25:08.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/719: fdatasync d0/d7/d67/d7f/d31/d46/fb7 0 2024-04-24T08:25:08.078 INFO:tasks.workunit.client.0.smithi090.stdout:9/658: fsync d0/dc/d24/d7c/f86 0 2024-04-24T08:25:08.080 INFO:tasks.workunit.client.0.smithi090.stdout:0/653: read - d1a/d4b/d53/fbb zero size 2024-04-24T08:25:08.083 INFO:tasks.workunit.client.0.smithi090.stdout:4/594: mkdir d3/d18/d5c/db5/db6 0 2024-04-24T08:25:08.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/720: fdatasync d0/d7/de1/fa1 0 2024-04-24T08:25:08.084 INFO:tasks.workunit.client.0.smithi090.stdout:5/713: mkdir d8/da/d3a/d2d/d37/dec 0 2024-04-24T08:25:08.086 INFO:tasks.workunit.client.0.smithi090.stdout:6/702: dread d8/dc/d19/f64 [0,4194304] 0 2024-04-24T08:25:08.088 INFO:tasks.workunit.client.0.smithi090.stdout:1/661: dwrite d3/d5/f8d [0,4194304] 0 2024-04-24T08:25:08.089 INFO:tasks.workunit.client.0.smithi090.stdout:2/721: chown d0/d7/fd 65 1 2024-04-24T08:25:08.092 INFO:tasks.workunit.client.0.smithi090.stdout:8/634: dwrite d2/da/d13/d64/d50/f57 [0,4194304] 0 2024-04-24T08:25:08.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/714: creat d8/d59/dc7/d7a/fed x:0 0 0 2024-04-24T08:25:08.094 INFO:tasks.workunit.client.0.smithi090.stdout:6/703: read - d8/d1d/d1f/f9c zero size 2024-04-24T08:25:08.094 INFO:tasks.workunit.client.0.smithi090.stdout:4/595: creat d3/d8e/fb7 x:0 0 0 2024-04-24T08:25:08.098 INFO:tasks.workunit.client.0.smithi090.stdout:8/635: dread - d2/d62/d83/da3/fbd zero size 2024-04-24T08:25:08.099 INFO:tasks.workunit.client.0.smithi090.stdout:3/600: dwrite d2/d25/d33/f81 [0,4194304] 0 2024-04-24T08:25:08.102 INFO:tasks.workunit.client.0.smithi090.stdout:2/722: rmdir d0/d7/d28/daa/def/db4/dcd 39 2024-04-24T08:25:08.102 INFO:tasks.workunit.client.0.smithi090.stdout:8/636: readlink d2/da/d13/d64/l96 0 2024-04-24T08:25:08.103 INFO:tasks.workunit.client.0.smithi090.stdout:4/596: symlink d3/d18/d5c/db5/d69/lb8 0 2024-04-24T08:25:08.106 INFO:tasks.workunit.client.0.smithi090.stdout:9/659: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd 0 2024-04-24T08:25:08.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/654: link d1a/d4b/d53/d79/d69/d8a/fbe d1a/d4b/d53/d79/d69/d78/fd7 0 2024-04-24T08:25:08.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/704: creat d8/dc/d19/d8e/d38/fd7 x:0 0 0 2024-04-24T08:25:08.109 INFO:tasks.workunit.client.0.smithi090.stdout:8/637: rmdir d2/da/d13/d64/dcc 39 2024-04-24T08:25:08.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/733: sync 2024-04-24T08:25:08.112 INFO:tasks.workunit.client.0.smithi090.stdout:4/597: mknod d3/d8e/cb9 0 2024-04-24T08:25:08.113 INFO:tasks.workunit.client.0.smithi090.stdout:6/705: dread - d8/dc/d19/d8e/fd3 zero size 2024-04-24T08:25:08.116 INFO:tasks.workunit.client.0.smithi090.stdout:1/662: getdents d3/d25 0 2024-04-24T08:25:08.116 INFO:tasks.workunit.client.0.smithi090.stdout:2/723: mknod d0/d14/d36/db9/cf1 0 2024-04-24T08:25:08.119 INFO:tasks.workunit.client.0.smithi090.stdout:5/715: rename d8/d76/fb2 to d8/da/d3a/d96/dcb/d21/fee 0 2024-04-24T08:25:08.121 INFO:tasks.workunit.client.0.smithi090.stdout:6/706: fsync d8/dc/d19/d8e/fa6 0 2024-04-24T08:25:08.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/734: rename db/d17/d39/d86/f92 to db/d17/d39/d86/ff9 0 2024-04-24T08:25:08.122 INFO:tasks.workunit.client.0.smithi090.stdout:3/601: dwrite d2/f24 [0,4194304] 0 2024-04-24T08:25:08.123 INFO:tasks.workunit.client.0.smithi090.stdout:0/655: read d1a/f30 [6393996,83693] 0 2024-04-24T08:25:08.124 INFO:tasks.workunit.client.0.smithi090.stdout:1/663: mkdir d3/d5/da/d97/ded 0 2024-04-24T08:25:08.125 INFO:tasks.workunit.client.0.smithi090.stdout:7/735: chown db/c5b 58766213 1 2024-04-24T08:25:08.126 INFO:tasks.workunit.client.0.smithi090.stdout:9/660: link d0/dc/d24/d7c/d4d/f84 d0/dc/d24/d7c/d4d/d32/fce 0 2024-04-24T08:25:08.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/638: dread d2/d62/d71/fa7 [0,4194304] 0 2024-04-24T08:25:08.126 INFO:tasks.workunit.client.0.smithi090.stdout:4/598: read d3/d18/d5c/db5/d69/d8b/db2/d49/f53 [692114,64227] 0 2024-04-24T08:25:08.129 INFO:tasks.workunit.client.0.smithi090.stdout:2/724: rmdir d0/d5e/de2/de9 0 2024-04-24T08:25:08.132 INFO:tasks.workunit.client.0.smithi090.stdout:5/716: rename d8/da/d14/d2f/d44/f5b to d8/da/d3a/fef 0 2024-04-24T08:25:08.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/639: rmdir d2/d62/d83/da3 39 2024-04-24T08:25:08.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/602: truncate d2/df/d1a/d37/f49 4517554 0 2024-04-24T08:25:08.140 INFO:tasks.workunit.client.0.smithi090.stdout:5/717: symlink d8/d76/dad/lf0 0 2024-04-24T08:25:08.141 INFO:tasks.workunit.client.0.smithi090.stdout:0/656: truncate d1a/d4b/d53/d57/fa4 639342 0 2024-04-24T08:25:08.144 INFO:tasks.workunit.client.0.smithi090.stdout:4/599: creat d3/fba x:0 0 0 2024-04-24T08:25:08.146 INFO:tasks.workunit.client.0.smithi090.stdout:2/725: getdents d0/d14/d17/dc2/dcb 0 2024-04-24T08:25:08.147 INFO:tasks.workunit.client.0.smithi090.stdout:8/640: mknod d2/da/d13/d64/db2/cda 0 2024-04-24T08:25:08.149 INFO:tasks.workunit.client.0.smithi090.stdout:6/707: dwrite d8/d1d/d2f/f88 [4194304,4194304] 0 2024-04-24T08:25:08.149 INFO:tasks.workunit.client.0.smithi090.stdout:5/718: creat d8/d7c/ff1 x:0 0 0 2024-04-24T08:25:08.155 INFO:tasks.workunit.client.0.smithi090.stdout:7/736: link db/d17/d39/l4b db/d17/d29/d38/d99/lfa 0 2024-04-24T08:25:08.157 INFO:tasks.workunit.client.0.smithi090.stdout:4/600: stat d3/d24/d92/ca3 0 2024-04-24T08:25:08.158 INFO:tasks.workunit.client.0.smithi090.stdout:2/726: dread - d0/d14/f53 zero size 2024-04-24T08:25:08.159 INFO:tasks.workunit.client.0.smithi090.stdout:1/664: dwrite d3/d25/d30/d9d/fd3 [4194304,4194304] 0 2024-04-24T08:25:08.160 INFO:tasks.workunit.client.0.smithi090.stdout:8/641: fdatasync d2/da/d13/d64/d24/f21 0 2024-04-24T08:25:08.161 INFO:tasks.workunit.client.0.smithi090.stdout:5/719: symlink d8/d59/dc7/d7a/lf2 0 2024-04-24T08:25:08.165 INFO:tasks.workunit.client.0.smithi090.stdout:9/661: truncate d0/dc/d24/d7c/d4d/f5f 3157984 0 2024-04-24T08:25:08.171 INFO:tasks.workunit.client.0.smithi090.stdout:7/737: mkdir db/d17/d29/d38/d99/dfb 0 2024-04-24T08:25:08.172 INFO:tasks.workunit.client.0.smithi090.stdout:0/657: getdents d1a/d4b/d53/d79/d35/d62 0 2024-04-24T08:25:08.174 INFO:tasks.workunit.client.0.smithi090.stdout:3/603: dwrite d2/df/d1a/d37/f8a [0,4194304] 0 2024-04-24T08:25:08.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/662: fsync d0/dc/d24/d7c/d4d/d32/fce 0 2024-04-24T08:25:08.179 INFO:tasks.workunit.client.0.smithi090.stdout:2/727: truncate d0/d7/d28/daa/db1/ff0 1350726 0 2024-04-24T08:25:08.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/665: write f0 [4455868,58087] 0 2024-04-24T08:25:08.188 INFO:tasks.workunit.client.0.smithi090.stdout:8/642: rename d2/l5 to d2/da/d13/d64/db2/ldb 0 2024-04-24T08:25:08.190 INFO:tasks.workunit.client.0.smithi090.stdout:4/601: dwrite d3/d18/d5c/f74 [0,4194304] 0 2024-04-24T08:25:08.194 INFO:tasks.workunit.client.0.smithi090.stdout:2/728: fdatasync d0/d7/d28/daa/db1/fe5 0 2024-04-24T08:25:08.196 INFO:tasks.workunit.client.0.smithi090.stdout:1/666: mkdir d3/dee 0 2024-04-24T08:25:08.196 INFO:tasks.workunit.client.0.smithi090.stdout:8/643: rmdir d2/d62/d6b 39 2024-04-24T08:25:08.197 INFO:tasks.workunit.client.0.smithi090.stdout:4/602: rmdir d3/d18/d5c/db5 39 2024-04-24T08:25:08.201 INFO:tasks.workunit.client.0.smithi090.stdout:0/658: rename d1a/d4b/d53/d79/d35/f55 to d1a/d4b/fd8 0 2024-04-24T08:25:08.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/604: dwrite d2/fa7 [0,4194304] 0 2024-04-24T08:25:08.209 INFO:tasks.workunit.client.0.smithi090.stdout:7/738: dwrite db/d17/d29/d32/d5a/f8d [0,4194304] 0 2024-04-24T08:25:08.210 INFO:tasks.workunit.client.0.smithi090.stdout:1/667: mkdir d3/d68/def 0 2024-04-24T08:25:08.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/605: fdatasync d2/df/d1a/d37/f55 0 2024-04-24T08:25:08.218 INFO:tasks.workunit.client.0.smithi090.stdout:6/708: dread d8/d1d/d29/f40 [0,4194304] 0 2024-04-24T08:25:08.221 INFO:tasks.workunit.client.0.smithi090.stdout:5/720: dread d8/da/d3a/d96/dcb/d21/f41 [0,4194304] 0 2024-04-24T08:25:08.224 INFO:tasks.workunit.client.0.smithi090.stdout:9/663: rename d0/d19/f56 to d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf 0 2024-04-24T08:25:08.228 INFO:tasks.workunit.client.0.smithi090.stdout:2/729: getdents d0/d7/d67/d7f 0 2024-04-24T08:25:08.228 INFO:tasks.workunit.client.0.smithi090.stdout:8/644: link d2/da/lc d2/da/d13/d64/db2/dbb/ldc 0 2024-04-24T08:25:08.230 INFO:tasks.workunit.client.0.smithi090.stdout:7/739: read - db/d17/d39/fbb zero size 2024-04-24T08:25:08.231 INFO:tasks.workunit.client.0.smithi090.stdout:3/606: mkdir d2/df/d1a/d37/d59/dcb 0 2024-04-24T08:25:08.234 INFO:tasks.workunit.client.0.smithi090.stdout:3/607: chown d2/fa7 7828 1 2024-04-24T08:25:08.235 INFO:tasks.workunit.client.0.smithi090.stdout:5/721: chown d8/da/d3a/d96/dcb/f22 64052 1 2024-04-24T08:25:08.235 INFO:tasks.workunit.client.0.smithi090.stdout:0/659: rename d1a/d4b/d53/d79/c39 to d1a/d4b/d53/d79/d69/d78/d9b/cd9 0 2024-04-24T08:25:08.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/709: write d8/dc/d19/f27 [3899830,28970] 0 2024-04-24T08:25:08.238 INFO:tasks.workunit.client.0.smithi090.stdout:7/740: creat db/de9/ffc x:0 0 0 2024-04-24T08:25:08.243 INFO:tasks.workunit.client.0.smithi090.stdout:6/710: unlink d8/d1d/f1e 0 2024-04-24T08:25:08.245 INFO:tasks.workunit.client.0.smithi090.stdout:4/603: dwrite d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/f35 [4194304,4194304] 0 2024-04-24T08:25:08.245 INFO:tasks.workunit.client.0.smithi090.stdout:9/664: rename d0/dc/f3e to d0/fd0 0 2024-04-24T08:25:08.246 INFO:tasks.workunit.client.0.smithi090.stdout:2/730: dread d0/d14/d36/fca [0,4194304] 0 2024-04-24T08:25:08.250 INFO:tasks.workunit.client.0.smithi090.stdout:0/660: truncate d1a/f3f 468915 0 2024-04-24T08:25:08.250 INFO:tasks.workunit.client.0.smithi090.stdout:1/668: dwrite d3/d68/d7f/fda [0,4194304] 0 2024-04-24T08:25:08.257 INFO:tasks.workunit.client.0.smithi090.stdout:5/722: truncate d8/da/d14/d2f/f47 4466180 0 2024-04-24T08:25:08.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/731: symlink d0/d5e/lf2 0 2024-04-24T08:25:08.257 INFO:tasks.workunit.client.0.smithi090.stdout:6/711: read d8/d1d/d1f/d3e/d8f/fd0 [910942,79114] 0 2024-04-24T08:25:08.258 INFO:tasks.workunit.client.0.smithi090.stdout:4/604: creat d3/d25/fbb x:0 0 0 2024-04-24T08:25:08.259 INFO:tasks.workunit.client.0.smithi090.stdout:1/669: symlink d3/d5/da/d97/d9c/lf0 0 2024-04-24T08:25:08.261 INFO:tasks.workunit.client.0.smithi090.stdout:5/723: creat d8/d7c/da9/ff3 x:0 0 0 2024-04-24T08:25:08.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/712: write d8/d1d/d1f/d3e/d44/fc0 [169632,89315] 0 2024-04-24T08:25:08.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/661: symlink d1a/d4b/d53/d79/d82/d9d/lda 0 2024-04-24T08:25:08.264 INFO:tasks.workunit.client.0.smithi090.stdout:7/741: dwrite db/d17/d29/d32/d5a/d75/f81 [0,4194304] 0 2024-04-24T08:25:08.266 INFO:tasks.workunit.client.0.smithi090.stdout:5/724: read d8/da/d3a/d96/dcb/f1f [1123876,19345] 0 2024-04-24T08:25:08.271 INFO:tasks.workunit.client.0.smithi090.stdout:7/742: dread - db/d17/d29/d38/d99/fef zero size 2024-04-24T08:25:08.271 INFO:tasks.workunit.client.0.smithi090.stdout:9/665: getdents d0/da5 0 2024-04-24T08:25:08.273 INFO:tasks.workunit.client.0.smithi090.stdout:5/725: stat d8/da/d3a/d96/dcb/f60 0 2024-04-24T08:25:08.274 INFO:tasks.workunit.client.0.smithi090.stdout:1/670: creat d3/d1d/ff1 x:0 0 0 2024-04-24T08:25:08.276 INFO:tasks.workunit.client.0.smithi090.stdout:6/713: creat d8/d1d/d1f/d3e/d44/d90/da2/fd8 x:0 0 0 2024-04-24T08:25:08.276 INFO:tasks.workunit.client.0.smithi090.stdout:0/662: mknod d1a/d4b/d53/d79/d69/d90/dbc/cdb 0 2024-04-24T08:25:08.283 INFO:tasks.workunit.client.0.smithi090.stdout:2/732: write d0/d2e/f49 [40983,88865] 0 2024-04-24T08:25:08.283 INFO:tasks.workunit.client.0.smithi090.stdout:1/671: read d3/d5/da/dc/f3a [3395143,22598] 0 2024-04-24T08:25:08.284 INFO:tasks.workunit.client.0.smithi090.stdout:4/605: dwrite d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/f51 [0,4194304] 0 2024-04-24T08:25:08.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/726: fdatasync d8/d59/dc7/d7a/d50/f6c 0 2024-04-24T08:25:08.287 INFO:tasks.workunit.client.0.smithi090.stdout:9/666: rename d0/d69 to d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1 0 2024-04-24T08:25:08.288 INFO:tasks.workunit.client.0.smithi090.stdout:1/672: write d3/d5/da/dc/f3a [4181156,106461] 0 2024-04-24T08:25:08.293 INFO:tasks.workunit.client.0.smithi090.stdout:6/714: truncate d8/dc/d19/f3d 834558 0 2024-04-24T08:25:08.294 INFO:tasks.workunit.client.0.smithi090.stdout:2/733: creat d0/d7/d28/ff3 x:0 0 0 2024-04-24T08:25:08.296 INFO:tasks.workunit.client.0.smithi090.stdout:2/734: write d0/d14/f1d [776466,12666] 0 2024-04-24T08:25:08.298 INFO:tasks.workunit.client.0.smithi090.stdout:2/735: fdatasync d0/d7/d67/d7f/d31/d90/f9d 0 2024-04-24T08:25:08.299 INFO:tasks.workunit.client.0.smithi090.stdout:1/673: fsync d3/d5/d3e/f76 0 2024-04-24T08:25:08.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/667: fdatasync d0/dc/d24/d7c/f36 0 2024-04-24T08:25:08.302 INFO:tasks.workunit.client.1.smithi149.stdout:2/802: sync 2024-04-24T08:25:08.304 INFO:tasks.workunit.client.0.smithi090.stdout:4/606: creat d3/d18/fbc x:0 0 0 2024-04-24T08:25:08.306 INFO:tasks.workunit.client.0.smithi090.stdout:1/674: unlink d3/d5/da/dc/d80/c7a 0 2024-04-24T08:25:08.308 INFO:tasks.workunit.client.1.smithi149.stdout:2/803: mkdir d0/d6/db/d39/d3d/d44/df0 0 2024-04-24T08:25:08.309 INFO:tasks.workunit.client.0.smithi090.stdout:4/607: write d3/d18/d1d/f46 [7045726,58934] 0 2024-04-24T08:25:08.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/675: chown d3/d1d/d54/f74 12889 1 2024-04-24T08:25:08.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/608: readlink d3/d18/d5c/db5/d69/d8b/db2/la5 0 2024-04-24T08:25:08.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/663: dread d1a/d4b/d53/d79/d35/d68/f76 [0,4194304] 0 2024-04-24T08:25:08.316 INFO:tasks.workunit.client.0.smithi090.stdout:5/727: dwrite d8/d59/dc7/d7a/d50/f68 [0,4194304] 0 2024-04-24T08:25:08.316 INFO:tasks.workunit.client.0.smithi090.stdout:9/668: link d0/dc/d24/d2d/d49/fb3 d0/dc/d24/fd2 0 2024-04-24T08:25:08.317 INFO:tasks.workunit.client.0.smithi090.stdout:1/676: mkdir d3/d68/df2 0 2024-04-24T08:25:08.321 INFO:tasks.workunit.client.0.smithi090.stdout:9/669: write d0/d2a/f98 [150227,73592] 0 2024-04-24T08:25:08.321 INFO:tasks.workunit.client.0.smithi090.stdout:0/664: creat d1a/d4b/d53/d79/d69/d90/fdc x:0 0 0 2024-04-24T08:25:08.324 INFO:tasks.workunit.client.0.smithi090.stdout:5/728: truncate d8/da/d14/d2f/d44/f97 472135 0 2024-04-24T08:25:08.333 INFO:tasks.workunit.client.0.smithi090.stdout:7/743: dread db/d17/d29/f42 [0,4194304] 0 2024-04-24T08:25:08.333 INFO:tasks.workunit.client.0.smithi090.stdout:4/609: getdents d3/d25 0 2024-04-24T08:25:08.335 INFO:tasks.workunit.client.0.smithi090.stdout:0/665: dread - d1a/d4b/d53/d79/d69/d8a/fd5 zero size 2024-04-24T08:25:08.336 INFO:tasks.workunit.client.0.smithi090.stdout:1/677: write d3/d5/da/f4d [143045,40238] 0 2024-04-24T08:25:08.339 INFO:tasks.workunit.client.1.smithi149.stdout:4/890: sync 2024-04-24T08:25:08.339 INFO:tasks.workunit.client.1.smithi149.stdout:6/799: sync 2024-04-24T08:25:08.339 INFO:tasks.workunit.client.1.smithi149.stdout:1/982: sync 2024-04-24T08:25:08.340 INFO:tasks.workunit.client.0.smithi090.stdout:0/666: read d1a/f1e [2049551,70577] 0 2024-04-24T08:25:08.342 INFO:tasks.workunit.client.1.smithi149.stdout:2/804: dread d0/d6/db/d28/f38 [4194304,4194304] 0 2024-04-24T08:25:08.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/805: chown d0/d6/db/d27/d81/db4 3 1 2024-04-24T08:25:08.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/715: dread d8/d1d/d1f/d3e/d8f/fcc [0,4194304] 0 2024-04-24T08:25:08.346 INFO:tasks.workunit.client.1.smithi149.stdout:4/891: truncate d2/d24/d2e/d38/f9c 2401747 0 2024-04-24T08:25:08.351 INFO:tasks.workunit.client.1.smithi149.stdout:2/806: rename d0/d6/db/de4 to d0/d6/db/d39/d78/d7c/df1 0 2024-04-24T08:25:08.357 INFO:tasks.workunit.client.1.smithi149.stdout:2/807: mknod d0/d6/db/d27/d81/db4/cf2 0 2024-04-24T08:25:08.373 INFO:tasks.workunit.client.0.smithi090.stdout:0/667: creat d1a/d4b/d53/d79/d35/db8/fdd x:0 0 0 2024-04-24T08:25:08.374 INFO:tasks.workunit.client.0.smithi090.stdout:6/716: creat d8/dc/d19/d8e/d81/fd9 x:0 0 0 2024-04-24T08:25:08.376 INFO:tasks.workunit.client.0.smithi090.stdout:9/670: truncate d0/d19/f46 3105279 0 2024-04-24T08:25:08.377 INFO:tasks.workunit.client.0.smithi090.stdout:7/744: mknod db/d17/d29/d38/d99/dfb/cfd 0 2024-04-24T08:25:08.379 INFO:tasks.workunit.client.0.smithi090.stdout:1/678: symlink d3/d68/def/lf3 0 2024-04-24T08:25:08.380 INFO:tasks.workunit.client.0.smithi090.stdout:0/668: creat d1a/d4b/d53/d79/d58/db4/fde x:0 0 0 2024-04-24T08:25:08.380 INFO:tasks.workunit.client.0.smithi090.stdout:1/679: dread - d3/d5/d3e/f7c zero size 2024-04-24T08:25:08.381 INFO:tasks.workunit.client.0.smithi090.stdout:3/608: sync 2024-04-24T08:25:08.381 INFO:tasks.workunit.client.0.smithi090.stdout:8/645: sync 2024-04-24T08:25:08.383 INFO:tasks.workunit.client.0.smithi090.stdout:3/609: stat d2/df/d5b/f63 0 2024-04-24T08:25:08.383 INFO:tasks.workunit.client.0.smithi090.stdout:0/669: chown d1a/d4b/d53/d79/d35/fc7 28491 1 2024-04-24T08:25:08.384 INFO:tasks.workunit.client.0.smithi090.stdout:1/680: write d3/d5/da/fc1 [882618,30729] 0 2024-04-24T08:25:08.386 INFO:tasks.workunit.client.0.smithi090.stdout:5/729: write d8/d99/f9d [1217149,51749] 0 2024-04-24T08:25:08.389 INFO:tasks.workunit.client.0.smithi090.stdout:3/610: truncate d2/f9 1303302 0 2024-04-24T08:25:08.390 INFO:tasks.workunit.client.0.smithi090.stdout:5/730: read d8/d25/fc8 [4018220,78692] 0 2024-04-24T08:25:08.390 INFO:tasks.workunit.client.0.smithi090.stdout:7/745: write f4 [1585658,75118] 0 2024-04-24T08:25:08.391 INFO:tasks.workunit.client.0.smithi090.stdout:3/611: stat d2/d15 0 2024-04-24T08:25:08.391 INFO:tasks.workunit.client.0.smithi090.stdout:1/681: mkdir d3/d5/da/dc/df4 0 2024-04-24T08:25:08.396 INFO:tasks.workunit.client.0.smithi090.stdout:1/682: write d3/d5/da/f4d [643660,48752] 0 2024-04-24T08:25:08.396 INFO:tasks.workunit.client.0.smithi090.stdout:5/731: read d8/d59/f61 [1504307,43338] 0 2024-04-24T08:25:08.397 INFO:tasks.workunit.client.0.smithi090.stdout:8/646: link d2/d62/d83/da3/fa5 d2/da/d13/d64/d24/db3/db4/fdd 0 2024-04-24T08:25:08.398 INFO:tasks.workunit.client.0.smithi090.stdout:4/610: dread d3/d87/f9f [0,4194304] 0 2024-04-24T08:25:08.399 INFO:tasks.workunit.client.0.smithi090.stdout:7/746: creat db/d54/d78/ffe x:0 0 0 2024-04-24T08:25:08.402 INFO:tasks.workunit.client.1.smithi149.stdout:1/983: write d0/ddf/d53/d61/f71 [2670010,105879] 0 2024-04-24T08:25:08.402 INFO:tasks.workunit.client.0.smithi090.stdout:3/612: mknod d2/df/d1a/d37/d53/ccc 0 2024-04-24T08:25:08.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/671: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f6f [0,4194304] 0 2024-04-24T08:25:08.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/984: stat d0/d1b/d2a/c30 0 2024-04-24T08:25:08.406 INFO:tasks.workunit.client.0.smithi090.stdout:5/732: fsync d8/da/d3a/d2d/d37/f88 0 2024-04-24T08:25:08.408 INFO:tasks.workunit.client.0.smithi090.stdout:8/647: symlink d2/da/d13/d64/db2/db6/lde 0 2024-04-24T08:25:08.411 INFO:tasks.workunit.client.0.smithi090.stdout:5/733: chown d8/da/d14/f17 28 1 2024-04-24T08:25:08.411 INFO:tasks.workunit.client.0.smithi090.stdout:7/747: symlink db/dab/lff 0 2024-04-24T08:25:08.414 INFO:tasks.workunit.client.1.smithi149.stdout:4/892: unlink d2/d18/d1e/d3c/d48/c6b 0 2024-04-24T08:25:08.418 INFO:tasks.workunit.client.0.smithi090.stdout:0/670: dwrite d1a/d4b/d53/d79/d3c/f84 [0,4194304] 0 2024-04-24T08:25:08.418 INFO:tasks.workunit.client.0.smithi090.stdout:4/611: rename d3/d18/d5c/db5/d69/fa8 to d3/d18/d5c/db5/d69/d8b/fbd 0 2024-04-24T08:25:08.421 INFO:tasks.workunit.client.0.smithi090.stdout:8/648: fdatasync d2/f69 0 2024-04-24T08:25:08.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/748: readlink db/d17/l9a 0 2024-04-24T08:25:08.433 INFO:tasks.workunit.client.1.smithi149.stdout:2/808: dwrite d0/d6/db/d27/f37 [0,4194304] 0 2024-04-24T08:25:08.437 INFO:tasks.workunit.client.0.smithi090.stdout:8/649: read d2/f69 [1983333,124643] 0 2024-04-24T08:25:08.440 INFO:tasks.workunit.client.0.smithi090.stdout:1/683: dread d3/d25/fd4 [0,4194304] 0 2024-04-24T08:25:08.442 INFO:tasks.workunit.client.0.smithi090.stdout:9/672: truncate d0/dc/d24/d2d/d49/fb3 4146743 0 2024-04-24T08:25:08.447 INFO:tasks.workunit.client.0.smithi090.stdout:0/671: truncate d1a/d4b/fc4 245688 0 2024-04-24T08:25:08.449 INFO:tasks.workunit.client.0.smithi090.stdout:8/650: mknod d2/da/d13/d60/d76/cdf 0 2024-04-24T08:25:08.451 INFO:tasks.workunit.client.0.smithi090.stdout:7/749: dwrite db/d17/d29/d32/f61 [0,4194304] 0 2024-04-24T08:25:08.452 INFO:tasks.workunit.client.0.smithi090.stdout:1/684: readlink d3/d5/l77 0 2024-04-24T08:25:08.458 INFO:tasks.workunit.client.0.smithi090.stdout:3/613: write d2/df/d1a/d37/d59/f61 [935130,104060] 0 2024-04-24T08:25:08.464 INFO:tasks.workunit.client.0.smithi090.stdout:8/651: mkdir d2/da/d13/d64/d24/db3/db4/de0 0 2024-04-24T08:25:08.465 INFO:tasks.workunit.client.0.smithi090.stdout:7/750: mknod db/d17/d29/d32/c100 0 2024-04-24T08:25:08.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/652: write d2/f59 [4861104,15082] 0 2024-04-24T08:25:08.467 INFO:tasks.workunit.client.0.smithi090.stdout:1/685: symlink d3/d1d/d34/d73/lf5 0 2024-04-24T08:25:08.469 INFO:tasks.workunit.client.0.smithi090.stdout:0/672: dwrite d1a/d4b/d53/d79/d82/f98 [0,4194304] 0 2024-04-24T08:25:08.469 INFO:tasks.workunit.client.0.smithi090.stdout:4/612: rename d3/l26 to d3/lbe 0 2024-04-24T08:25:08.473 INFO:tasks.workunit.client.0.smithi090.stdout:3/614: rename d2/d25/d2c/l51 to d2/df/d57/d4f/lcd 0 2024-04-24T08:25:08.477 INFO:tasks.workunit.client.0.smithi090.stdout:7/751: mkdir db/d17/d29/d32/d5a/d75/d101 0 2024-04-24T08:25:08.486 INFO:tasks.workunit.client.0.smithi090.stdout:1/686: dread d3/d5/f8d [0,4194304] 0 2024-04-24T08:25:08.489 INFO:tasks.workunit.client.0.smithi090.stdout:7/752: dread - db/d54/db1/fb4 zero size 2024-04-24T08:25:08.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/736: sync 2024-04-24T08:25:08.494 INFO:tasks.workunit.client.0.smithi090.stdout:3/615: creat d2/df/d1a/d37/d59/d87/db0/fce x:0 0 0 2024-04-24T08:25:08.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/653: dread d2/da/d13/d34/f6f [0,4194304] 0 2024-04-24T08:25:08.495 INFO:tasks.workunit.client.0.smithi090.stdout:7/753: rmdir db/de9 39 2024-04-24T08:25:08.499 INFO:tasks.workunit.client.0.smithi090.stdout:9/673: link d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d92/lc2 d0/ld3 0 2024-04-24T08:25:08.501 INFO:tasks.workunit.client.0.smithi090.stdout:2/737: symlink d0/d14/d36/lf4 0 2024-04-24T08:25:08.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/654: fdatasync d2/da/d13/d64/f16 0 2024-04-24T08:25:08.507 INFO:tasks.workunit.client.0.smithi090.stdout:2/738: truncate d0/d7/d28/daa/def/d50/d96/fcf 749073 0 2024-04-24T08:25:08.509 INFO:tasks.workunit.client.0.smithi090.stdout:3/616: dwrite d2/df/d57/d4f/f72 [0,4194304] 0 2024-04-24T08:25:08.510 INFO:tasks.workunit.client.0.smithi090.stdout:9/674: write d0/f1b [2496316,2825] 0 2024-04-24T08:25:08.513 INFO:tasks.workunit.client.0.smithi090.stdout:9/675: write d0/d19/f1e [3868904,8468] 0 2024-04-24T08:25:08.519 INFO:tasks.workunit.client.0.smithi090.stdout:7/754: creat db/d17/d29/dc0/f102 x:0 0 0 2024-04-24T08:25:08.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/613: dwrite d3/d24/f2b [4194304,4194304] 0 2024-04-24T08:25:08.527 INFO:tasks.workunit.client.0.smithi090.stdout:3/617: creat d2/df/d1a/d37/d53/d60/d9b/dca/fcf x:0 0 0 2024-04-24T08:25:08.529 INFO:tasks.workunit.client.0.smithi090.stdout:0/673: write d1a/d4b/d53/d79/d3c/f5b [364899,65001] 0 2024-04-24T08:25:08.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/618: chown d2/df/d1a/d37/d59/f7b 192438547 1 2024-04-24T08:25:08.534 INFO:tasks.workunit.client.0.smithi090.stdout:7/755: creat db/d17/d29/d38/d3e/d46/f103 x:0 0 0 2024-04-24T08:25:08.534 INFO:tasks.workunit.client.0.smithi090.stdout:3/619: readlink d2/df/d57/la2 0 2024-04-24T08:25:08.535 INFO:tasks.workunit.client.0.smithi090.stdout:2/739: link d0/d14/f53 d0/d7/d28/daa/ff5 0 2024-04-24T08:25:08.536 INFO:tasks.workunit.client.1.smithi149.stdout:7/975: sync 2024-04-24T08:25:08.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/961: sync 2024-04-24T08:25:08.537 INFO:tasks.workunit.client.1.smithi149.stdout:0/873: sync 2024-04-24T08:25:08.537 INFO:tasks.workunit.client.1.smithi149.stdout:8/887: sync 2024-04-24T08:25:08.541 INFO:tasks.workunit.client.0.smithi090.stdout:7/756: truncate db/d17/d29/d32/fe4 213897 0 2024-04-24T08:25:08.543 INFO:tasks.workunit.client.0.smithi090.stdout:3/620: chown d2/d15/f1d 3092 1 2024-04-24T08:25:08.544 INFO:tasks.workunit.client.0.smithi090.stdout:2/740: fsync d0/d7/d67/d7f/d31/d46/d60/fdf 0 2024-04-24T08:25:08.545 INFO:tasks.workunit.client.0.smithi090.stdout:0/674: unlink d1a/d4b/l9c 0 2024-04-24T08:25:08.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/675: readlink d1a/d4b/d53/d79/d35/l50 0 2024-04-24T08:25:08.550 INFO:tasks.workunit.client.0.smithi090.stdout:8/655: getdents d2/da/d13/d64/d24/db3/db4 0 2024-04-24T08:25:08.553 INFO:tasks.workunit.client.1.smithi149.stdout:9/962: mkdir d1/d2/d33/d163 0 2024-04-24T08:25:08.556 INFO:tasks.workunit.client.0.smithi090.stdout:2/741: fsync d0/f3b 0 2024-04-24T08:25:08.557 INFO:tasks.workunit.client.1.smithi149.stdout:8/888: rmdir d1/de/d18/d32/d65 39 2024-04-24T08:25:08.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/621: creat d2/df/d1a/d37/d59/fd0 x:0 0 0 2024-04-24T08:25:08.562 INFO:tasks.workunit.client.0.smithi090.stdout:7/757: symlink db/d90/l104 0 2024-04-24T08:25:08.566 INFO:tasks.workunit.client.0.smithi090.stdout:2/742: symlink d0/d14/d17/lf6 0 2024-04-24T08:25:08.571 INFO:tasks.workunit.client.1.smithi149.stdout:8/889: mkdir d1/de/d18/d32/d63/d76/d12f 0 2024-04-24T08:25:08.578 INFO:tasks.workunit.client.1.smithi149.stdout:4/893: write d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 [968646,78856] 0 2024-04-24T08:25:08.579 INFO:tasks.workunit.client.1.smithi149.stdout:8/890: creat d1/de/d18/d32/d63/d76/f130 x:0 0 0 2024-04-24T08:25:08.581 INFO:tasks.workunit.client.1.smithi149.stdout:7/976: getdents d9/d43/d4f/d101 0 2024-04-24T08:25:08.581 INFO:tasks.workunit.client.1.smithi149.stdout:8/891: chown d1/de/lb2 305906 1 2024-04-24T08:25:08.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/800: sync 2024-04-24T08:25:08.585 INFO:tasks.workunit.client.1.smithi149.stdout:6/801: stat d0/d1a/d28/f9c 0 2024-04-24T08:25:08.585 INFO:tasks.workunit.client.1.smithi149.stdout:1/985: dwrite d0/d5/df/fca [0,4194304] 0 2024-04-24T08:25:08.587 INFO:tasks.workunit.client.0.smithi090.stdout:3/622: mknod d2/d15/d36/d6f/d92/cd1 0 2024-04-24T08:25:08.587 INFO:tasks.workunit.client.0.smithi090.stdout:7/758: unlink db/d17/d29/d32/fca 0 2024-04-24T08:25:08.590 INFO:tasks.workunit.client.1.smithi149.stdout:4/894: creat d2/d10/d1a/dac/ddd/f118 x:0 0 0 2024-04-24T08:25:08.590 INFO:tasks.workunit.client.1.smithi149.stdout:7/977: creat d9/dc/d11/d83/f143 x:0 0 0 2024-04-24T08:25:08.592 INFO:tasks.workunit.client.0.smithi090.stdout:8/656: creat d2/d62/d6b/d9c/dd8/fe1 x:0 0 0 2024-04-24T08:25:08.592 INFO:tasks.workunit.client.0.smithi090.stdout:9/676: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f9d [0,4194304] 0 2024-04-24T08:25:08.593 INFO:tasks.workunit.client.1.smithi149.stdout:9/963: write d1/d2/d15/d1c/d42/f7f [357885,48524] 0 2024-04-24T08:25:08.595 INFO:tasks.workunit.client.0.smithi090.stdout:4/614: dwrite d3/d18/d5c/f7f [0,4194304] 0 2024-04-24T08:25:08.598 INFO:tasks.workunit.client.1.smithi149.stdout:1/986: mknod d0/d5/d4f/d60/c13e 0 2024-04-24T08:25:08.599 INFO:tasks.workunit.client.0.smithi090.stdout:2/743: creat d0/d14/d36/ff7 x:0 0 0 2024-04-24T08:25:08.601 INFO:tasks.workunit.client.0.smithi090.stdout:3/623: rmdir d2/df/d1a/d37/d59/d98 39 2024-04-24T08:25:08.603 INFO:tasks.workunit.client.1.smithi149.stdout:0/874: truncate d2/d9/df/d56/f5f 1938636 0 2024-04-24T08:25:08.604 INFO:tasks.workunit.client.0.smithi090.stdout:2/744: chown d0/d7/d67/d7f/d31/l82 400148 1 2024-04-24T08:25:08.604 INFO:tasks.workunit.client.1.smithi149.stdout:4/895: mknod d2/d10/d91/d9f/c119 0 2024-04-24T08:25:08.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/759: creat db/d17/d29/d38/d99/f105 x:0 0 0 2024-04-24T08:25:08.610 INFO:tasks.workunit.client.1.smithi149.stdout:2/809: dwrite d0/d6/fa0 [0,4194304] 0 2024-04-24T08:25:08.610 INFO:tasks.workunit.client.0.smithi090.stdout:3/624: truncate d2/d15/d36/f5f 528506 0 2024-04-24T08:25:08.613 INFO:tasks.workunit.client.1.smithi149.stdout:7/978: symlink d9/dc/d2e/d3c/l144 0 2024-04-24T08:25:08.615 INFO:tasks.workunit.client.0.smithi090.stdout:8/657: symlink d2/da/d13/d64/dcc/le2 0 2024-04-24T08:25:08.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/615: mknod d3/cbf 0 2024-04-24T08:25:08.622 INFO:tasks.workunit.client.1.smithi149.stdout:4/896: creat d2/d10/d1a/f11a x:0 0 0 2024-04-24T08:25:08.625 INFO:tasks.workunit.client.0.smithi090.stdout:3/625: rename d2/f17 to d2/df/d1a/d37/d59/dcb/fd2 0 2024-04-24T08:25:08.626 INFO:tasks.workunit.client.0.smithi090.stdout:7/760: fdatasync db/f11 0 2024-04-24T08:25:08.627 INFO:tasks.workunit.client.0.smithi090.stdout:3/626: chown d2/df/d1a/d37/d59/d87 60615 1 2024-04-24T08:25:08.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/745: creat d0/d7/de1/ff8 x:0 0 0 2024-04-24T08:25:08.633 INFO:tasks.workunit.client.1.smithi149.stdout:0/875: dread d2/f8 [4194304,4194304] 0 2024-04-24T08:25:08.637 INFO:tasks.workunit.client.0.smithi090.stdout:2/746: read d0/d7/d67/d7f/f5d [2610762,46205] 0 2024-04-24T08:25:08.638 INFO:tasks.workunit.client.1.smithi149.stdout:8/892: creat d1/de/d18/d32/d65/f131 x:0 0 0 2024-04-24T08:25:08.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/616: symlink d3/d18/d5c/db5/d69/d8b/db2/lc0 0 2024-04-24T08:25:08.643 INFO:tasks.workunit.client.0.smithi090.stdout:3/627: dwrite d2/df/d1a/d37/d59/f61 [0,4194304] 0 2024-04-24T08:25:08.650 INFO:tasks.workunit.client.1.smithi149.stdout:8/893: chown d1/d1f/ffe 8 1 2024-04-24T08:25:08.653 INFO:tasks.workunit.client.0.smithi090.stdout:3/628: unlink d2/df/d21/f27 0 2024-04-24T08:25:08.655 INFO:tasks.workunit.client.0.smithi090.stdout:3/629: fsync d2/df/d1a/f28 0 2024-04-24T08:25:08.656 INFO:tasks.workunit.client.0.smithi090.stdout:3/630: fsync d2/df/fa5 0 2024-04-24T08:25:08.662 INFO:tasks.workunit.client.1.smithi149.stdout:7/979: rename d9/d43/d95/c115 to d9/dc/d11/d83/da6/d67/d69/c145 0 2024-04-24T08:25:08.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/658: rename d2/da/d13/d64/dcc/le2 to d2/le3 0 2024-04-24T08:25:08.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/659: write d2/f12 [4956100,25692] 0 2024-04-24T08:25:08.669 INFO:tasks.workunit.client.1.smithi149.stdout:6/802: dwrite d0/f11 [4194304,4194304] 0 2024-04-24T08:25:08.671 INFO:tasks.workunit.client.1.smithi149.stdout:9/964: dread d1/d2/d34/ff2 [0,4194304] 0 2024-04-24T08:25:08.681 INFO:tasks.workunit.client.1.smithi149.stdout:1/987: dread d0/d1b/db5/f2b [0,4194304] 0 2024-04-24T08:25:08.682 INFO:tasks.workunit.client.1.smithi149.stdout:7/980: truncate d9/dc/d2e/d51/f59 1652772 0 2024-04-24T08:25:08.688 INFO:tasks.workunit.client.1.smithi149.stdout:9/965: creat d1/d2/d33/d60/f164 x:0 0 0 2024-04-24T08:25:08.689 INFO:tasks.workunit.client.0.smithi090.stdout:3/631: dread d2/df/f3c [0,4194304] 0 2024-04-24T08:25:08.689 INFO:tasks.workunit.client.0.smithi090.stdout:7/761: write db/f15 [2572712,34679] 0 2024-04-24T08:25:08.689 INFO:tasks.workunit.client.1.smithi149.stdout:8/894: dread - d1/de/d18/d32/d65/d8b/d7d/ddc/ff0 zero size 2024-04-24T08:25:08.694 INFO:tasks.workunit.client.1.smithi149.stdout:6/803: symlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/l10e 0 2024-04-24T08:25:08.697 INFO:tasks.workunit.client.1.smithi149.stdout:8/895: dread - d1/d1f/d22/f115 zero size 2024-04-24T08:25:08.697 INFO:tasks.workunit.client.0.smithi090.stdout:2/747: rmdir d0/d14/d17/dc2/dc9 0 2024-04-24T08:25:08.702 INFO:tasks.workunit.client.1.smithi149.stdout:1/988: unlink d0/ddf/da0/db8/dec/f103 0 2024-04-24T08:25:08.703 INFO:tasks.workunit.client.0.smithi090.stdout:3/632: symlink d2/df/d1a/d37/d53/d60/ld3 0 2024-04-24T08:25:08.706 INFO:tasks.workunit.client.0.smithi090.stdout:2/748: chown d0/d7/d28/daa/def/d50/d96/cb6 2 1 2024-04-24T08:25:08.706 INFO:tasks.workunit.client.1.smithi149.stdout:2/810: dwrite d0/d6/db/d39/d78/d8a/f99 [0,4194304] 0 2024-04-24T08:25:08.708 INFO:tasks.workunit.client.0.smithi090.stdout:7/762: dwrite db/d17/d29/d38/d3e/d46/fe3 [0,4194304] 0 2024-04-24T08:25:08.708 INFO:tasks.workunit.client.0.smithi090.stdout:4/617: rename d3/d18/d5c/f4a to d3/fc1 0 2024-04-24T08:25:08.708 INFO:tasks.workunit.client.1.smithi149.stdout:4/897: dread d2/d18/d1e/d3c/d59/fb1 [0,4194304] 0 2024-04-24T08:25:08.708 INFO:tasks.workunit.client.1.smithi149.stdout:9/966: rmdir d1/d2/d33/dea/d133/dad 39 2024-04-24T08:25:08.711 INFO:tasks.workunit.client.0.smithi090.stdout:8/660: creat d2/d62/fe4 x:0 0 0 2024-04-24T08:25:08.713 INFO:tasks.workunit.client.1.smithi149.stdout:4/898: dread - d2/d10/d1a/d34/f62 zero size 2024-04-24T08:25:08.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/717: sync 2024-04-24T08:25:08.714 INFO:tasks.workunit.client.0.smithi090.stdout:5/734: sync 2024-04-24T08:25:08.718 INFO:tasks.workunit.client.1.smithi149.stdout:6/804: chown d0/f3 0 1 2024-04-24T08:25:08.719 INFO:tasks.workunit.client.1.smithi149.stdout:0/876: dwrite d2/d9/f9d [0,4194304] 0 2024-04-24T08:25:08.724 INFO:tasks.workunit.client.0.smithi090.stdout:3/633: chown d2/df/l6c 14 1 2024-04-24T08:25:08.724 INFO:tasks.workunit.client.0.smithi090.stdout:2/749: symlink d0/d7/d28/daa/db1/lf9 0 2024-04-24T08:25:08.725 INFO:tasks.workunit.client.1.smithi149.stdout:6/805: stat d0/d1a/d28/d31/d4c/d80/d6f/dd7/lf1 0 2024-04-24T08:25:08.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/718: dread d8/d1d/d1f/d3e/d8f/fcc [0,4194304] 0 2024-04-24T08:25:08.726 INFO:tasks.workunit.client.0.smithi090.stdout:7/763: readlink db/d17/d29/d38/d99/lfa 0 2024-04-24T08:25:08.727 INFO:tasks.workunit.client.1.smithi149.stdout:9/967: chown d1/d2/d34/f70 1 1 2024-04-24T08:25:08.729 INFO:tasks.workunit.client.0.smithi090.stdout:4/618: creat d3/d24/d92/fc2 x:0 0 0 2024-04-24T08:25:08.729 INFO:tasks.workunit.client.1.smithi149.stdout:7/981: creat d9/dc/d11/d83/da6/d67/f146 x:0 0 0 2024-04-24T08:25:08.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/661: unlink d2/da/d13/d64/l96 0 2024-04-24T08:25:08.733 INFO:tasks.workunit.client.1.smithi149.stdout:2/811: creat d0/d6/db/d28/db0/ff3 x:0 0 0 2024-04-24T08:25:08.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/896: creat d1/de/d18/d32/d63/d76/d12f/f132 x:0 0 0 2024-04-24T08:25:08.733 INFO:tasks.workunit.client.1.smithi149.stdout:9/968: dread - d1/de/d12/f153 zero size 2024-04-24T08:25:08.735 INFO:tasks.workunit.client.1.smithi149.stdout:8/897: chown d1/d1f/d25/d7a/f12d 381844343 1 2024-04-24T08:25:08.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/898: stat d1/d1f/d25/d7a/f12d 0 2024-04-24T08:25:08.739 INFO:tasks.workunit.client.0.smithi090.stdout:3/634: stat d2/l18 0 2024-04-24T08:25:08.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/877: creat d2/d9/df/d2f/d6c/d72/f117 x:0 0 0 2024-04-24T08:25:08.745 INFO:tasks.workunit.client.1.smithi149.stdout:0/878: stat d2/d9/df/d2f/d6c/d72/f117 0 2024-04-24T08:25:08.748 INFO:tasks.workunit.client.1.smithi149.stdout:4/899: link d2/d24/d2e/d3f/d94/dfd/f10a d2/d18/db9/dd8/f11b 0 2024-04-24T08:25:08.749 INFO:tasks.workunit.client.1.smithi149.stdout:4/900: stat d2/d18/d1e/d3c/f80 0 2024-04-24T08:25:08.756 INFO:tasks.workunit.client.1.smithi149.stdout:7/982: dread d9/dc/d11/d83/da6/f140 [0,4194304] 0 2024-04-24T08:25:08.758 INFO:tasks.workunit.client.0.smithi090.stdout:5/735: rename d8/d76/dad/le7 to d8/da/lf4 0 2024-04-24T08:25:08.759 INFO:tasks.workunit.client.1.smithi149.stdout:8/899: mknod d1/d1f/d22/c133 0 2024-04-24T08:25:08.760 INFO:tasks.workunit.client.0.smithi090.stdout:5/736: fdatasync d8/da/d14/d2f/f66 0 2024-04-24T08:25:08.761 INFO:tasks.workunit.client.0.smithi090.stdout:3/635: dwrite d2/d25/d2c/fb2 [0,4194304] 0 2024-04-24T08:25:08.763 INFO:tasks.workunit.client.1.smithi149.stdout:6/806: mkdir d0/d1a/d28/d31/d4c/d80/d6f/d10f 0 2024-04-24T08:25:08.764 INFO:tasks.workunit.client.1.smithi149.stdout:4/901: mknod d2/d10/c11c 0 2024-04-24T08:25:08.765 INFO:tasks.workunit.client.0.smithi090.stdout:7/764: creat db/d17/d29/d32/d5a/d75/d101/f106 x:0 0 0 2024-04-24T08:25:08.766 INFO:tasks.workunit.client.1.smithi149.stdout:2/812: dread d0/d6/db/d27/fc3 [0,4194304] 0 2024-04-24T08:25:08.769 INFO:tasks.workunit.client.0.smithi090.stdout:4/619: dread d3/d18/d1d/f46 [4194304,4194304] 0 2024-04-24T08:25:08.775 INFO:tasks.workunit.client.1.smithi149.stdout:9/969: rmdir d1/d2/d33/dea/d133/dad 39 2024-04-24T08:25:08.782 INFO:tasks.workunit.client.1.smithi149.stdout:1/989: dwrite d0/f84 [4194304,4194304] 0 2024-04-24T08:25:08.782 INFO:tasks.workunit.client.1.smithi149.stdout:7/983: mkdir d9/dc/d7b/dc2/d147 0 2024-04-24T08:25:08.784 INFO:tasks.workunit.client.0.smithi090.stdout:8/662: write d2/f54 [1516093,75898] 0 2024-04-24T08:25:08.786 INFO:tasks.workunit.client.1.smithi149.stdout:1/990: stat d0/ddf/da0/db8/dec/f100 0 2024-04-24T08:25:08.786 INFO:tasks.workunit.client.1.smithi149.stdout:7/984: fdatasync d9/dc/d2e/d11e/d133/f141 0 2024-04-24T08:25:08.787 INFO:tasks.workunit.client.0.smithi090.stdout:6/719: dwrite d8/dc/fad [0,4194304] 0 2024-04-24T08:25:08.788 INFO:tasks.workunit.client.1.smithi149.stdout:8/900: mkdir d1/d1f/d22/d134 0 2024-04-24T08:25:08.789 INFO:tasks.workunit.client.1.smithi149.stdout:1/991: write d0/ddf/feb [4632956,113774] 0 2024-04-24T08:25:08.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/737: rename d8/da/d3a/d96/dcb/f2b to d8/d59/dc7/d7a/da1/ff5 0 2024-04-24T08:25:08.794 INFO:tasks.workunit.client.1.smithi149.stdout:0/879: mknod d2/d9/d1a/c118 0 2024-04-24T08:25:08.795 INFO:tasks.workunit.client.0.smithi090.stdout:1/687: sync 2024-04-24T08:25:08.798 INFO:tasks.workunit.client.0.smithi090.stdout:4/620: symlink d3/d18/d5c/db5/d69/d8b/db2/d6b/lc3 0 2024-04-24T08:25:08.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/750: dwrite d0/d14/d17/f18 [0,4194304] 0 2024-04-24T08:25:08.801 INFO:tasks.workunit.client.0.smithi090.stdout:8/663: creat d2/d62/d83/da3/fe5 x:0 0 0 2024-04-24T08:25:08.803 INFO:tasks.workunit.client.0.smithi090.stdout:8/664: fdatasync d2/da/d13/d64/d50/f57 0 2024-04-24T08:25:08.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/902: unlink d2/d18/db9/dd8/f42 0 2024-04-24T08:25:08.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/880: truncate d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/ffc 648734 0 2024-04-24T08:25:08.814 INFO:tasks.workunit.client.0.smithi090.stdout:7/765: creat db/dbd/f107 x:0 0 0 2024-04-24T08:25:08.815 INFO:tasks.workunit.client.0.smithi090.stdout:3/636: dwrite d2/d15/d36/f83 [4194304,4194304] 0 2024-04-24T08:25:08.817 INFO:tasks.workunit.client.0.smithi090.stdout:1/688: creat d3/d1d/d34/d73/ff6 x:0 0 0 2024-04-24T08:25:08.818 INFO:tasks.workunit.client.0.smithi090.stdout:2/751: chown d0/d5e/le3 650234806 1 2024-04-24T08:25:08.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/881: rmdir d2/d9/d1a/d36/d85/db5 39 2024-04-24T08:25:08.825 INFO:tasks.workunit.client.1.smithi149.stdout:4/903: rename d2/d10/d91/ca5 to d2/d10/d1a/c11d 0 2024-04-24T08:25:08.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/813: creat d0/ff4 x:0 0 0 2024-04-24T08:25:08.830 INFO:tasks.workunit.client.1.smithi149.stdout:0/882: stat d2/d9/df/d40/lc8 0 2024-04-24T08:25:08.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/637: mknod d2/df/d1a/d37/d59/dcb/cd4 0 2024-04-24T08:25:08.832 INFO:tasks.workunit.client.0.smithi090.stdout:8/665: getdents d2/d97 0 2024-04-24T08:25:08.835 INFO:tasks.workunit.client.0.smithi090.stdout:6/720: rename d8/d1d/d2f/l50 to d8/dc/d19/lda 0 2024-04-24T08:25:08.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/621: creat d3/fc4 x:0 0 0 2024-04-24T08:25:08.839 INFO:tasks.workunit.client.0.smithi090.stdout:6/721: read d8/dc/d19/d8e/d81/d96/fbb [419808,118670] 0 2024-04-24T08:25:08.841 INFO:tasks.workunit.client.1.smithi149.stdout:4/904: creat d2/d18/da4/f11e x:0 0 0 2024-04-24T08:25:08.848 INFO:tasks.workunit.client.0.smithi090.stdout:3/638: symlink d2/df/d5b/ld5 0 2024-04-24T08:25:08.849 INFO:tasks.workunit.client.0.smithi090.stdout:8/666: creat d2/da/d13/d60/fe6 x:0 0 0 2024-04-24T08:25:08.851 INFO:tasks.workunit.client.0.smithi090.stdout:4/622: mknod d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/cc5 0 2024-04-24T08:25:08.853 INFO:tasks.workunit.client.0.smithi090.stdout:2/752: rmdir d0/d7/d67/d7f/d31/d46/d60/dd9 0 2024-04-24T08:25:08.855 INFO:tasks.workunit.client.1.smithi149.stdout:7/985: dwrite d9/dc/d11/d83/da6/f140 [4194304,4194304] 0 2024-04-24T08:25:08.857 INFO:tasks.workunit.client.0.smithi090.stdout:4/623: read d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/f35 [2294833,18434] 0 2024-04-24T08:25:08.860 INFO:tasks.workunit.client.1.smithi149.stdout:1/992: dread d0/d1b/d5d/fdc [0,4194304] 0 2024-04-24T08:25:08.860 INFO:tasks.workunit.client.1.smithi149.stdout:6/807: dread d0/d1a/d28/d31/d4c/d80/d6f/dd7/ff6 [0,4194304] 0 2024-04-24T08:25:08.865 INFO:tasks.workunit.client.0.smithi090.stdout:2/753: fsync d0/f55 0 2024-04-24T08:25:08.873 INFO:tasks.workunit.client.0.smithi090.stdout:7/766: rename db/dbd/fe0 to db/d54/f108 0 2024-04-24T08:25:08.875 INFO:tasks.workunit.client.1.smithi149.stdout:1/993: unlink d0/d1b/d5d/de5/f111 0 2024-04-24T08:25:08.880 INFO:tasks.workunit.client.1.smithi149.stdout:0/883: dwrite d2/d9/df/d2f/d6c/d72/f117 [0,4194304] 0 2024-04-24T08:25:08.881 INFO:tasks.workunit.client.1.smithi149.stdout:7/986: creat d9/dc/d2e/d11e/d133/f148 x:0 0 0 2024-04-24T08:25:08.881 INFO:tasks.workunit.client.0.smithi090.stdout:6/722: rmdir d8/d1d/d2f/d85/d9f 0 2024-04-24T08:25:08.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/624: unlink d3/d18/d5c/db5/l20 0 2024-04-24T08:25:08.890 INFO:tasks.workunit.client.0.smithi090.stdout:3/639: creat d2/df/db5/fd6 x:0 0 0 2024-04-24T08:25:08.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/625: write d3/d18/f5b [909821,70637] 0 2024-04-24T08:25:08.892 INFO:tasks.workunit.client.1.smithi149.stdout:9/970: dwrite d1/d2/d33/dea/d126/fb1 [0,4194304] 0 2024-04-24T08:25:08.892 INFO:tasks.workunit.client.0.smithi090.stdout:6/723: truncate d8/d1d/d1f/d3e/d4f/d7e/fbe 677786 0 2024-04-24T08:25:08.894 INFO:tasks.workunit.client.0.smithi090.stdout:5/738: dwrite d8/d56/f5f [0,4194304] 0 2024-04-24T08:25:08.896 INFO:tasks.workunit.client.0.smithi090.stdout:2/754: stat d0/d14/f53 0 2024-04-24T08:25:08.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/901: dwrite d1/de/fc8 [0,4194304] 0 2024-04-24T08:25:08.920 INFO:tasks.workunit.client.1.smithi149.stdout:2/814: dwrite d0/d6/db/d39/d3d/d58/f5e [0,4194304] 0 2024-04-24T08:25:08.925 INFO:tasks.workunit.client.1.smithi149.stdout:4/905: dread d2/d10/d1a/f36 [0,4194304] 0 2024-04-24T08:25:08.925 INFO:tasks.workunit.client.1.smithi149.stdout:4/906: chown d2/d18/d1e/d3c/d48/d69/d7d/d8d/fd4 125330 1 2024-04-24T08:25:08.926 INFO:tasks.workunit.client.1.smithi149.stdout:1/994: rename d0/d5/db/d18/d4d/d50/f134 to d0/d1b/d5d/dd6/ddb/d127/d12d/f13f 0 2024-04-24T08:25:08.927 INFO:tasks.workunit.client.0.smithi090.stdout:0/676: sync 2024-04-24T08:25:08.928 INFO:tasks.workunit.client.0.smithi090.stdout:9/677: sync 2024-04-24T08:25:08.930 INFO:tasks.workunit.client.1.smithi149.stdout:1/995: dread d0/d1b/d5d/fdc [0,4194304] 0 2024-04-24T08:25:08.931 INFO:tasks.workunit.client.0.smithi090.stdout:9/678: write d0/d19/f25 [8219278,83309] 0 2024-04-24T08:25:08.956 INFO:tasks.workunit.client.0.smithi090.stdout:1/689: sync 2024-04-24T08:25:08.956 INFO:tasks.workunit.client.1.smithi149.stdout:1/996: dwrite d0/d1b/d5d/dd6/f12f [0,4194304] 0 2024-04-24T08:25:08.960 INFO:tasks.workunit.client.0.smithi090.stdout:1/690: write d3/d5/da/fc1 [1750926,85757] 0 2024-04-24T08:25:08.961 INFO:tasks.workunit.client.0.smithi090.stdout:3/640: fsync d2/df/d5b/f63 0 2024-04-24T08:25:08.963 INFO:tasks.workunit.client.0.smithi090.stdout:3/641: fdatasync d2/d25/d33/f81 0 2024-04-24T08:25:08.967 INFO:tasks.workunit.client.1.smithi149.stdout:6/808: dwrite d0/f1 [0,4194304] 0 2024-04-24T08:25:08.967 INFO:tasks.workunit.client.0.smithi090.stdout:3/642: truncate d2/df/d1a/d37/d53/f6b 248351 0 2024-04-24T08:25:08.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/755: mkdir d0/d14/d17/dfa 0 2024-04-24T08:25:08.985 INFO:tasks.workunit.client.0.smithi090.stdout:2/756: chown d0/d14/d17/c8e 66 1 2024-04-24T08:25:08.988 INFO:tasks.workunit.client.1.smithi149.stdout:7/987: write d9/d43/d4f/fd1 [333995,103885] 0 2024-04-24T08:25:08.989 INFO:tasks.workunit.client.0.smithi090.stdout:9/679: fsync d0/dc/d24/d2d/f80 0 2024-04-24T08:25:08.991 INFO:tasks.workunit.client.1.smithi149.stdout:7/988: chown d9/dc/de/d44 296 1 2024-04-24T08:25:08.993 INFO:tasks.workunit.client.0.smithi090.stdout:7/767: mkdir db/dab/d109 0 2024-04-24T08:25:08.995 INFO:tasks.workunit.client.0.smithi090.stdout:7/768: write db/d17/d29/d32/d5a/d75/f8a [985503,16072] 0 2024-04-24T08:25:08.996 INFO:tasks.workunit.client.0.smithi090.stdout:8/667: link d2/da/d13/l1d d2/da/d13/d64/db2/db6/dd0/le7 0 2024-04-24T08:25:08.997 INFO:tasks.workunit.client.0.smithi090.stdout:7/769: chown db/d17/d29/d38/d99/f105 1843669 1 2024-04-24T08:25:08.997 INFO:tasks.workunit.client.1.smithi149.stdout:0/884: truncate d2/d9/d1a/f6b 2738181 0 2024-04-24T08:25:08.998 INFO:tasks.workunit.client.0.smithi090.stdout:8/668: write d2/d62/d6b/f7d [522227,50266] 0 2024-04-24T08:25:09.000 INFO:tasks.workunit.client.1.smithi149.stdout:9/971: dread d1/d2/d15/f81 [0,4194304] 0 2024-04-24T08:25:09.001 INFO:tasks.workunit.client.1.smithi149.stdout:6/809: dread - d0/d1a/d28/d31/d4c/d76/fe1 zero size 2024-04-24T08:25:09.003 INFO:tasks.workunit.client.0.smithi090.stdout:4/626: getdents d3/d25 0 2024-04-24T08:25:09.004 INFO:tasks.workunit.client.0.smithi090.stdout:4/627: chown d3/d18/d5c/l9d 126 1 2024-04-24T08:25:09.007 INFO:tasks.workunit.client.1.smithi149.stdout:4/907: mknod d2/c11f 0 2024-04-24T08:25:09.008 INFO:tasks.workunit.client.0.smithi090.stdout:3/643: unlink d2/d15/f31 0 2024-04-24T08:25:09.009 INFO:tasks.workunit.client.1.smithi149.stdout:7/989: readlink d9/d43/lca 0 2024-04-24T08:25:09.010 INFO:tasks.workunit.client.0.smithi090.stdout:6/724: mkdir d8/d1d/ddb 0 2024-04-24T08:25:09.011 INFO:tasks.workunit.client.1.smithi149.stdout:1/997: mknod d0/ddf/d53/c140 0 2024-04-24T08:25:09.012 INFO:tasks.workunit.client.1.smithi149.stdout:1/998: stat d0/ddf/fc0 0 2024-04-24T08:25:09.015 INFO:tasks.workunit.client.1.smithi149.stdout:0/885: dread - d2/d9/d1a/d36/d85/db5/def/ff3 zero size 2024-04-24T08:25:09.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/677: rename d1a/d4b/fc4 to d1a/d4b/d53/d79/d35/fdf 0 2024-04-24T08:25:09.018 INFO:tasks.workunit.client.1.smithi149.stdout:6/810: rmdir d0/d1a/d28/d4a 39 2024-04-24T08:25:09.021 INFO:tasks.workunit.client.1.smithi149.stdout:2/815: fsync d0/d6/db/d39/d3d/d44/f96 0 2024-04-24T08:25:09.022 INFO:tasks.workunit.client.0.smithi090.stdout:9/680: symlink d0/dc/d24/d2d/d49/d62/ld4 0 2024-04-24T08:25:09.023 INFO:tasks.workunit.client.1.smithi149.stdout:8/902: dwrite d1/de/d18/daf/fcf [0,4194304] 0 2024-04-24T08:25:09.025 INFO:tasks.workunit.client.1.smithi149.stdout:2/816: chown d0/d6/db/d28/d7f/d8f/db9/ddf/de3 0 1 2024-04-24T08:25:09.028 INFO:tasks.workunit.client.1.smithi149.stdout:1/999: symlink d0/d5/db/d18/d4d/de8/dfb/l141 0 2024-04-24T08:25:09.028 INFO:tasks.workunit.client.0.smithi090.stdout:7/770: mknod db/d17/d39/d86/c10a 0 2024-04-24T08:25:09.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/771: write db/d17/d39/f3c [2869183,91066] 0 2024-04-24T08:25:09.032 INFO:tasks.workunit.client.1.smithi149.stdout:0/886: stat d2/d9/df/c3e 0 2024-04-24T08:25:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:1/691: symlink d3/d5/da/dc/df4/lf7 0 2024-04-24T08:25:09.035 INFO:tasks.workunit.client.0.smithi090.stdout:6/725: mknod d8/d1d/d29/cdc 0 2024-04-24T08:25:09.036 INFO:tasks.workunit.client.0.smithi090.stdout:5/739: link d8/da/d3a/d96/dcb/d21/f48 d8/d76/dad/ff6 0 2024-04-24T08:25:09.037 INFO:tasks.workunit.client.0.smithi090.stdout:1/692: write d3/d1d/f55 [161899,61519] 0 2024-04-24T08:25:09.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/644: dread d2/d15/d36/d6f/d92/fac [0,4194304] 0 2024-04-24T08:25:09.038 INFO:tasks.workunit.client.0.smithi090.stdout:4/628: rename d3/d24/d92/ca3 to d3/d24/cc6 0 2024-04-24T08:25:09.040 INFO:tasks.workunit.client.1.smithi149.stdout:9/972: creat d1/d2/d15/d1c/f165 x:0 0 0 2024-04-24T08:25:09.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/645: readlink d2/df/d57/d4f/lb4 0 2024-04-24T08:25:09.042 INFO:tasks.workunit.client.0.smithi090.stdout:0/678: creat d1a/d4b/d53/d57/fe0 x:0 0 0 2024-04-24T08:25:09.043 INFO:tasks.workunit.client.1.smithi149.stdout:2/817: chown d0/d6/db/d39/d3d/f6b 108 1 2024-04-24T08:25:09.044 INFO:tasks.workunit.client.0.smithi090.stdout:0/679: chown d1a/d4b/d53/d79/d58/db4/fde 1672179 1 2024-04-24T08:25:09.044 INFO:tasks.workunit.client.1.smithi149.stdout:7/990: write d9/dc/d11/d83/da6/d67/d69/dad/f110 [947296,100722] 0 2024-04-24T08:25:09.045 INFO:tasks.workunit.client.0.smithi090.stdout:9/681: symlink d0/dc/d24/d7c/ld5 0 2024-04-24T08:25:09.046 INFO:tasks.workunit.client.0.smithi090.stdout:0/680: chown d1a/d4b/d53/d79/d82/f98 11525 1 2024-04-24T08:25:09.047 INFO:tasks.workunit.client.1.smithi149.stdout:0/887: symlink d2/d9/df/d56/l119 0 2024-04-24T08:25:09.049 INFO:tasks.workunit.client.1.smithi149.stdout:0/888: chown d2/d9/df/d48/dba 4000341 1 2024-04-24T08:25:09.053 INFO:tasks.workunit.client.1.smithi149.stdout:8/903: symlink d1/dce/de9/l135 0 2024-04-24T08:25:09.055 INFO:tasks.workunit.client.0.smithi090.stdout:8/669: dwrite d2/da/d13/d34/f3c [0,4194304] 0 2024-04-24T08:25:09.057 INFO:tasks.workunit.client.1.smithi149.stdout:9/973: mknod d1/d2/d15/d1c/dca/d128/c166 0 2024-04-24T08:25:09.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/818: write d0/d6/fae [19754,69333] 0 2024-04-24T08:25:09.059 INFO:tasks.workunit.client.0.smithi090.stdout:6/726: write d8/d1d/d29/f40 [3451445,102273] 0 2024-04-24T08:25:09.060 INFO:tasks.workunit.client.1.smithi149.stdout:7/991: mknod d9/dc/d11/d83/da6/d67/d69/dad/c149 0 2024-04-24T08:25:09.060 INFO:tasks.workunit.client.1.smithi149.stdout:4/908: dread d2/d10/fa6 [0,4194304] 0 2024-04-24T08:25:09.061 INFO:tasks.workunit.client.0.smithi090.stdout:5/740: creat d8/da/d3a/d96/dcb/ff7 x:0 0 0 2024-04-24T08:25:09.062 INFO:tasks.workunit.client.1.smithi149.stdout:0/889: creat d2/d3c/dc7/f11a x:0 0 0 2024-04-24T08:25:09.064 INFO:tasks.workunit.client.1.smithi149.stdout:2/819: stat d0/d6/db/d27/d81/d9f/dba 0 2024-04-24T08:25:09.067 INFO:tasks.workunit.client.1.smithi149.stdout:6/811: dwrite d0/f3 [0,4194304] 0 2024-04-24T08:25:09.092 INFO:tasks.workunit.client.0.smithi090.stdout:9/682: unlink d0/dc/d24/d7c/f94 0 2024-04-24T08:25:09.093 INFO:tasks.workunit.client.1.smithi149.stdout:0/890: fsync d2/d17/f98 0 2024-04-24T08:25:09.094 INFO:tasks.workunit.client.0.smithi090.stdout:9/683: write d0/d2a/f98 [810251,104406] 0 2024-04-24T08:25:09.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/681: chown d1a/d4b/d53/d79/d3c/c95 8707977 1 2024-04-24T08:25:09.097 INFO:tasks.workunit.client.1.smithi149.stdout:2/820: mknod d0/d6/db/d39/d3d/d58/d98/cf5 0 2024-04-24T08:25:09.098 INFO:tasks.workunit.client.1.smithi149.stdout:4/909: dread d2/f7 [0,4194304] 0 2024-04-24T08:25:09.100 INFO:tasks.workunit.client.1.smithi149.stdout:6/812: fdatasync d0/d1a/d28/d31/d4c/d4e/d62/d66/f71 0 2024-04-24T08:25:09.102 INFO:tasks.workunit.client.0.smithi090.stdout:8/670: rmdir d2 39 2024-04-24T08:25:09.102 INFO:tasks.workunit.client.0.smithi090.stdout:3/646: write d2/f2f [481485,61678] 0 2024-04-24T08:25:09.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/757: link d0/d2e/l44 d0/d7/d67/lfb 0 2024-04-24T08:25:09.105 INFO:tasks.workunit.client.1.smithi149.stdout:7/992: truncate d9/d43/d4f/da5/f10e 2786457 0 2024-04-24T08:25:09.105 INFO:tasks.workunit.client.0.smithi090.stdout:3/647: write d2/d25/d33/f81 [1222620,56451] 0 2024-04-24T08:25:09.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/741: creat d8/d7c/ff8 x:0 0 0 2024-04-24T08:25:09.106 INFO:tasks.workunit.client.1.smithi149.stdout:7/993: chown d9/fd 217733 1 2024-04-24T08:25:09.108 INFO:tasks.workunit.client.1.smithi149.stdout:9/974: dread d1/d2/d33/d60/f77 [0,4194304] 0 2024-04-24T08:25:09.108 INFO:tasks.workunit.client.0.smithi090.stdout:5/742: chown d8/da/d3a/d96/da2 449456 1 2024-04-24T08:25:09.109 INFO:tasks.workunit.client.1.smithi149.stdout:0/891: rename d2/d9/d1a/d36/l8d to d2/d9/de6/l11b 0 2024-04-24T08:25:09.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/743: dread - d8/da/d3a/d96/dcb/f65 zero size 2024-04-24T08:25:09.110 INFO:tasks.workunit.client.0.smithi090.stdout:6/727: creat d8/d1d/d1f/d3e/d44/d90/da2/fdd x:0 0 0 2024-04-24T08:25:09.111 INFO:tasks.workunit.client.1.smithi149.stdout:4/910: creat d2/d24/d2e/d3f/f120 x:0 0 0 2024-04-24T08:25:09.112 INFO:tasks.workunit.client.0.smithi090.stdout:5/744: stat d8/da/d3a/d96/dcb/f60 0 2024-04-24T08:25:09.112 INFO:tasks.workunit.client.1.smithi149.stdout:6/813: fdatasync d0/d1a/f2e 0 2024-04-24T08:25:09.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/745: chown d8/d25/l90 64 1 2024-04-24T08:25:09.113 INFO:tasks.workunit.client.0.smithi090.stdout:4/629: symlink d3/d18/d5c/db5/d69/d8b/db2/lc7 0 2024-04-24T08:25:09.115 INFO:tasks.workunit.client.0.smithi090.stdout:4/630: dread - d3/d18/d5c/db5/d69/d8b/fac zero size 2024-04-24T08:25:09.115 INFO:tasks.workunit.client.1.smithi149.stdout:2/821: rename d0/d6/db/d39/d3d/d58/f5e to d0/d6/db/d28/d7f/d9b/ff6 0 2024-04-24T08:25:09.118 INFO:tasks.workunit.client.0.smithi090.stdout:4/631: readlink d3/d18/d5c/db5/d69/d8b/db2/d49/l67 0 2024-04-24T08:25:09.120 INFO:tasks.workunit.client.1.smithi149.stdout:9/975: rmdir d1/d2/d33/dea/d126 39 2024-04-24T08:25:09.120 INFO:tasks.workunit.client.1.smithi149.stdout:0/892: creat d2/d9/df/d48/dba/f11c x:0 0 0 2024-04-24T08:25:09.121 INFO:tasks.workunit.client.0.smithi090.stdout:0/682: mknod d1a/d4b/d53/d79/d69/d8a/ce1 0 2024-04-24T08:25:09.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/684: symlink d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/ld6 0 2024-04-24T08:25:09.127 INFO:tasks.workunit.client.0.smithi090.stdout:7/772: creat db/d17/d39/f10b x:0 0 0 2024-04-24T08:25:09.128 INFO:tasks.workunit.client.1.smithi149.stdout:4/911: dwrite d2/d24/d2e/d38/fec [4194304,4194304] 0 2024-04-24T08:25:09.138 INFO:tasks.workunit.client.1.smithi149.stdout:0/893: dwrite d2/d9/df/fec [0,4194304] 0 2024-04-24T08:25:09.140 INFO:tasks.workunit.client.0.smithi090.stdout:3/648: fsync d2/df/d1a/d37/d59/f7b 0 2024-04-24T08:25:09.142 INFO:tasks.workunit.client.1.smithi149.stdout:2/822: creat d0/d6/db/d28/d7f/ff7 x:0 0 0 2024-04-24T08:25:09.145 INFO:tasks.workunit.client.0.smithi090.stdout:6/728: creat d8/d1d/d1f/d3e/d44/fde x:0 0 0 2024-04-24T08:25:09.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/746: fsync d8/f10 0 2024-04-24T08:25:09.147 INFO:tasks.workunit.client.1.smithi149.stdout:6/814: write d0/d1a/d28/d31/d4c/d80/f74 [4185555,47703] 0 2024-04-24T08:25:09.151 INFO:tasks.workunit.client.0.smithi090.stdout:1/693: link d3/d1d/d34/d5a/f5b d3/d1d/d34/d73/dd8/ff8 0 2024-04-24T08:25:09.152 INFO:tasks.workunit.client.1.smithi149.stdout:4/912: mkdir d2/d24/d2e/d121 0 2024-04-24T08:25:09.153 INFO:tasks.workunit.client.0.smithi090.stdout:4/632: rename d3/d18/d5c/db5/d69/d8b/db2/d49/l50 to d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/lc8 0 2024-04-24T08:25:09.155 INFO:tasks.workunit.client.0.smithi090.stdout:5/747: truncate d8/d59/dc7/d7a/d50/d9f/fd9 605532 0 2024-04-24T08:25:09.156 INFO:tasks.workunit.client.1.smithi149.stdout:7/994: rename d9/d43/l62 to d9/dc/d11/d83/l14a 0 2024-04-24T08:25:09.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/685: mknod d0/dc/d24/d2d/cd7 0 2024-04-24T08:25:09.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/633: write d3/d18/d5c/db5/d69/d8b/db2/d6b/fb4 [512074,87948] 0 2024-04-24T08:25:09.161 INFO:tasks.workunit.client.1.smithi149.stdout:0/894: creat d2/d9/d1a/d36/d85/db5/def/f11d x:0 0 0 2024-04-24T08:25:09.163 INFO:tasks.workunit.client.0.smithi090.stdout:7/773: creat db/d17/d29/d32/d5a/d75/f10c x:0 0 0 2024-04-24T08:25:09.168 INFO:tasks.workunit.client.0.smithi090.stdout:2/758: dread d0/d7/d28/daa/def/d50/d96/fa2 [0,4194304] 0 2024-04-24T08:25:09.168 INFO:tasks.workunit.client.1.smithi149.stdout:6/815: mknod d0/d1a/d28/d31/d38/d54/c110 0 2024-04-24T08:25:09.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/649: chown d2/df/d21/c8b 4 1 2024-04-24T08:25:09.172 INFO:tasks.workunit.client.1.smithi149.stdout:7/995: symlink d9/dc/d11/d83/da6/d67/dba/l14b 0 2024-04-24T08:25:09.174 INFO:tasks.workunit.client.1.smithi149.stdout:2/823: mkdir d0/d6/db/d39/d3d/d44/df0/df8 0 2024-04-24T08:25:09.175 INFO:tasks.workunit.client.0.smithi090.stdout:3/650: write d2/df/d1a/f28 [5677945,50961] 0 2024-04-24T08:25:09.176 INFO:tasks.workunit.client.1.smithi149.stdout:0/895: dread d2/d9/d1a/f25 [0,4194304] 0 2024-04-24T08:25:09.176 INFO:tasks.workunit.client.1.smithi149.stdout:9/976: link d1/d2/d33/d5d/d83/dbf/dc4/f13c d1/d2/d15/d1c/d42/df3/f167 0 2024-04-24T08:25:09.177 INFO:tasks.workunit.client.0.smithi090.stdout:5/748: dwrite d8/d59/dc7/d7a/d50/f68 [0,4194304] 0 2024-04-24T08:25:09.178 INFO:tasks.workunit.client.0.smithi090.stdout:5/749: fsync d8/f9e 0 2024-04-24T08:25:09.181 INFO:tasks.workunit.client.0.smithi090.stdout:5/750: chown d8/d7c/da9/ff3 7409 1 2024-04-24T08:25:09.182 INFO:tasks.workunit.client.0.smithi090.stdout:8/671: mkdir d2/da/d13/d64/d24/db3/db4/dcd/de8 0 2024-04-24T08:25:09.182 INFO:tasks.workunit.client.0.smithi090.stdout:7/774: dwrite db/d17/d29/d38/f4e [0,4194304] 0 2024-04-24T08:25:09.183 INFO:tasks.workunit.client.0.smithi090.stdout:7/775: stat db/d17/d39 0 2024-04-24T08:25:09.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/672: read - d2/da/d13/d60/fe6 zero size 2024-04-24T08:25:09.184 INFO:tasks.workunit.client.0.smithi090.stdout:5/751: write d8/da/d14/d2f/f66 [842064,27402] 0 2024-04-24T08:25:09.184 INFO:tasks.workunit.client.0.smithi090.stdout:4/634: truncate d3/d18/d1d/f1f 3918309 0 2024-04-24T08:25:09.187 INFO:tasks.workunit.client.0.smithi090.stdout:5/752: dread - d8/da/d14/fd3 zero size 2024-04-24T08:25:09.188 INFO:tasks.workunit.client.1.smithi149.stdout:7/996: fdatasync d9/d43/d4f/d66/d96/ffe 0 2024-04-24T08:25:09.191 INFO:tasks.workunit.client.0.smithi090.stdout:4/635: truncate d3/d18/d5c/db5/d7c/fa4 469562 0 2024-04-24T08:25:09.192 INFO:tasks.workunit.client.1.smithi149.stdout:2/824: truncate d0/f2 5178856 0 2024-04-24T08:25:09.192 INFO:tasks.workunit.client.0.smithi090.stdout:6/729: write d8/f1b [585819,70792] 0 2024-04-24T08:25:09.194 INFO:tasks.workunit.client.0.smithi090.stdout:6/730: chown d8/dc/f10 20601 1 2024-04-24T08:25:09.194 INFO:tasks.workunit.client.1.smithi149.stdout:9/977: mkdir d1/d2/d15/d1c/dca/d168 0 2024-04-24T08:25:09.197 INFO:tasks.workunit.client.1.smithi149.stdout:4/913: write d2/f14 [3715312,16404] 0 2024-04-24T08:25:09.198 INFO:tasks.workunit.client.1.smithi149.stdout:0/896: truncate d2/d9/df/d2f/d6c/d72/f99 1639537 0 2024-04-24T08:25:09.199 INFO:tasks.workunit.client.0.smithi090.stdout:1/694: write d3/d1d/d34/f98 [323990,55032] 0 2024-04-24T08:25:09.200 INFO:tasks.workunit.client.1.smithi149.stdout:6/816: creat d0/d1a/d28/d31/d4c/df8/df9/f111 x:0 0 0 2024-04-24T08:25:09.201 INFO:tasks.workunit.client.1.smithi149.stdout:7/997: fsync d9/dc/de/f3a 0 2024-04-24T08:25:09.202 INFO:tasks.workunit.client.1.smithi149.stdout:0/897: write d2/d9/d1a/d36/d85/ffa [1163929,4479] 0 2024-04-24T08:25:09.203 INFO:tasks.workunit.client.1.smithi149.stdout:2/825: rename d0/d6/db/f4a to d0/d6/db/d39/d3d/d44/df0/ff9 0 2024-04-24T08:25:09.205 INFO:tasks.workunit.client.1.smithi149.stdout:9/978: mkdir d1/da/dd/d79/d169 0 2024-04-24T08:25:09.206 INFO:tasks.workunit.client.0.smithi090.stdout:7/776: creat db/d17/d29/d32/d5a/d75/d101/f10d x:0 0 0 2024-04-24T08:25:09.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/914: dread - d2/d18/db9/ffa zero size 2024-04-24T08:25:09.209 INFO:tasks.workunit.client.0.smithi090.stdout:4/636: creat d3/d18/d5c/db5/d69/d8b/fc9 x:0 0 0 2024-04-24T08:25:09.214 INFO:tasks.workunit.client.0.smithi090.stdout:2/759: link d0/d14/d36/dce/dee/fcc d0/d5e/ffc 0 2024-04-24T08:25:09.216 INFO:tasks.workunit.client.0.smithi090.stdout:0/683: getdents d1a/d4b/d53/d79/d69/d90/dcc 0 2024-04-24T08:25:09.216 INFO:tasks.workunit.client.1.smithi149.stdout:0/898: read d2/d9/df/d2f/f8f [265779,15706] 0 2024-04-24T08:25:09.217 INFO:tasks.workunit.client.1.smithi149.stdout:7/998: truncate d9/dc/d2e/f41 1392311 0 2024-04-24T08:25:09.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/695: chown d3/d25/d30/d9d/da2/fb5 1 1 2024-04-24T08:25:09.221 INFO:tasks.workunit.client.1.smithi149.stdout:7/999: truncate d9/dc/d2e/d3c/ffa 1447493 0 2024-04-24T08:25:09.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/731: dwrite d8/d1d/d1f/d3e/fca [4194304,4194304] 0 2024-04-24T08:25:09.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/753: link d8/d59/dc7/d7a/da1/ff5 d8/d59/dc7/d7b/ff9 0 2024-04-24T08:25:09.225 INFO:tasks.workunit.client.0.smithi090.stdout:6/732: truncate d8/d1d/fcf 760331 0 2024-04-24T08:25:09.229 INFO:tasks.workunit.client.1.smithi149.stdout:6/817: fsync d0/d1a/d28/d31/d4c/d80/d6f/dd7/fed 0 2024-04-24T08:25:09.232 INFO:tasks.workunit.client.1.smithi149.stdout:4/915: dread d2/fc [0,4194304] 0 2024-04-24T08:25:09.234 INFO:tasks.workunit.client.0.smithi090.stdout:4/637: rename d3/d18/d5c/db5/d69/d8b/db2/f4d to d3/d87/da6/fca 0 2024-04-24T08:25:09.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/673: dwrite d2/da/d13/f52 [0,4194304] 0 2024-04-24T08:25:09.237 INFO:tasks.workunit.client.0.smithi090.stdout:8/674: write d2/da/d13/f7e [1455954,127944] 0 2024-04-24T08:25:09.238 INFO:tasks.workunit.client.0.smithi090.stdout:2/760: mknod d0/d7/d28/daa/db1/cfd 0 2024-04-24T08:25:09.244 INFO:tasks.workunit.client.0.smithi090.stdout:0/684: creat d1a/d4b/d53/d79/d58/db4/fe2 x:0 0 0 2024-04-24T08:25:09.251 INFO:tasks.workunit.client.1.smithi149.stdout:6/818: rename d0/d1a/d28/d31/d4c/d102 to d0/db7/d112 0 2024-04-24T08:25:09.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/754: dread - d8/da/d3a/d2d/d37/f88 zero size 2024-04-24T08:25:09.253 INFO:tasks.workunit.client.0.smithi090.stdout:5/755: dread - d8/d76/dad/fdb zero size 2024-04-24T08:25:09.253 INFO:tasks.workunit.client.0.smithi090.stdout:6/733: rmdir d8/dc/d19/d8e/d38 39 2024-04-24T08:25:09.257 INFO:tasks.workunit.client.1.smithi149.stdout:0/899: mknod d2/d9/df/d2f/d6c/d72/db0/c11e 0 2024-04-24T08:25:09.259 INFO:tasks.workunit.client.1.smithi149.stdout:9/979: creat d1/d4c/f16a x:0 0 0 2024-04-24T08:25:09.260 INFO:tasks.workunit.client.0.smithi090.stdout:7/777: write db/d17/d39/fbb [603531,31351] 0 2024-04-24T08:25:09.260 INFO:tasks.workunit.client.1.smithi149.stdout:4/916: write d2/f7 [2687377,56236] 0 2024-04-24T08:25:09.264 INFO:tasks.workunit.client.1.smithi149.stdout:2/826: dwrite d0/d6/db/d39/d3d/fb6 [0,4194304] 0 2024-04-24T08:25:09.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/819: fdatasync d0/d1a/d28/d31/d38/f3a 0 2024-04-24T08:25:09.270 INFO:tasks.workunit.client.1.smithi149.stdout:2/827: chown d0/d6/db/d27/d64/f74 90 1 2024-04-24T08:25:09.276 INFO:tasks.workunit.client.1.smithi149.stdout:4/917: fdatasync d2/d10/f27 0 2024-04-24T08:25:09.276 INFO:tasks.workunit.client.1.smithi149.stdout:2/828: read d0/d6/fae [422331,94993] 0 2024-04-24T08:25:09.277 INFO:tasks.workunit.client.0.smithi090.stdout:4/638: dwrite d3/d8e/f97 [0,4194304] 0 2024-04-24T08:25:09.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/761: mkdir d0/d14/d36/dfe 0 2024-04-24T08:25:09.285 INFO:tasks.workunit.client.0.smithi090.stdout:0/685: mknod d1a/d4b/d53/d79/d58/d96/ce3 0 2024-04-24T08:25:09.287 INFO:tasks.workunit.client.1.smithi149.stdout:6/820: rename d0/d1a/d28/ffe to d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/f113 0 2024-04-24T08:25:09.287 INFO:tasks.workunit.client.1.smithi149.stdout:0/900: dwrite d2/d9/df/d48/dba/ff7 [0,4194304] 0 2024-04-24T08:25:09.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/651: getdents d2/d15/d36/d6f 0 2024-04-24T08:25:09.292 INFO:tasks.workunit.client.1.smithi149.stdout:0/901: stat d2/d9/df/d2f/d6c/dc1 0 2024-04-24T08:25:09.298 INFO:tasks.workunit.client.1.smithi149.stdout:0/902: stat d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6 0 2024-04-24T08:25:09.302 INFO:tasks.workunit.client.0.smithi090.stdout:8/675: symlink d2/da/d13/d64/db2/db6/dd0/le9 0 2024-04-24T08:25:09.303 INFO:tasks.workunit.client.1.smithi149.stdout:2/829: read d0/f35 [2986001,7154] 0 2024-04-24T08:25:09.304 INFO:tasks.workunit.client.0.smithi090.stdout:8/676: chown d2/da/d13/d34/f66 158914 1 2024-04-24T08:25:09.305 INFO:tasks.workunit.client.0.smithi090.stdout:2/762: mkdir d0/d14/d17/dc2/dff 0 2024-04-24T08:25:09.306 INFO:tasks.workunit.client.0.smithi090.stdout:0/686: fsync d1a/d4b/d53/d79/d3c/fad 0 2024-04-24T08:25:09.308 INFO:tasks.workunit.client.0.smithi090.stdout:3/652: mkdir d2/df/db5/dd7 0 2024-04-24T08:25:09.310 INFO:tasks.workunit.client.0.smithi090.stdout:1/696: creat d3/d25/d30/d9d/ff9 x:0 0 0 2024-04-24T08:25:09.312 INFO:tasks.workunit.client.1.smithi149.stdout:4/918: mknod d2/d18/d1e/c122 0 2024-04-24T08:25:09.313 INFO:tasks.workunit.client.0.smithi090.stdout:4/639: mkdir d3/d18/d1d/dcb 0 2024-04-24T08:25:09.314 INFO:tasks.workunit.client.1.smithi149.stdout:4/919: truncate d2/f107 1008372 0 2024-04-24T08:25:09.315 INFO:tasks.workunit.client.0.smithi090.stdout:8/677: dread - d2/da/f70 zero size 2024-04-24T08:25:09.316 INFO:tasks.workunit.client.0.smithi090.stdout:8/678: write d2/f10 [3687774,26637] 0 2024-04-24T08:25:09.321 INFO:tasks.workunit.client.0.smithi090.stdout:0/687: mknod d1a/d4b/d53/d79/d69/d78/ce4 0 2024-04-24T08:25:09.321 INFO:tasks.workunit.client.0.smithi090.stdout:5/756: dwrite d8/da/d14/d2f/d81/fb8 [0,4194304] 0 2024-04-24T08:25:09.323 INFO:tasks.workunit.client.1.smithi149.stdout:2/830: rename d0/d6/db/d39/d3d/d44/ld9 to d0/d6/lfa 0 2024-04-24T08:25:09.323 INFO:tasks.workunit.client.1.smithi149.stdout:9/980: dwrite d1/de/fdb [0,4194304] 0 2024-04-24T08:25:09.326 INFO:tasks.workunit.client.1.smithi149.stdout:6/821: getdents d0/d1a/d28/d31/d4c/d4e 0 2024-04-24T08:25:09.328 INFO:tasks.workunit.client.1.smithi149.stdout:9/981: write d1/da/f8 [516094,115558] 0 2024-04-24T08:25:09.337 INFO:tasks.workunit.client.1.smithi149.stdout:9/982: fsync d1/d2/f123 0 2024-04-24T08:25:09.339 INFO:tasks.workunit.client.0.smithi090.stdout:2/763: creat d0/d14/d17/dc2/dcb/f100 x:0 0 0 2024-04-24T08:25:09.341 INFO:tasks.workunit.client.1.smithi149.stdout:6/822: read d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 [605999,73115] 0 2024-04-24T08:25:09.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/757: creat d8/da/d14/d2f/d44/ffa x:0 0 0 2024-04-24T08:25:09.342 INFO:tasks.workunit.client.1.smithi149.stdout:0/903: dread d2/d17/fd1 [0,4194304] 0 2024-04-24T08:25:09.343 INFO:tasks.workunit.client.0.smithi090.stdout:9/686: sync 2024-04-24T08:25:09.343 INFO:tasks.workunit.client.0.smithi090.stdout:5/758: write d8/d7c/da9/ff3 [186027,68734] 0 2024-04-24T08:25:09.345 INFO:tasks.workunit.client.1.smithi149.stdout:6/823: chown d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f88 43 1 2024-04-24T08:25:09.347 INFO:tasks.workunit.client.1.smithi149.stdout:4/920: link d2/d24/d2e/f4a d2/d10/f123 0 2024-04-24T08:25:09.347 INFO:tasks.workunit.client.0.smithi090.stdout:3/653: dwrite d2/df/d1a/f4c [0,4194304] 0 2024-04-24T08:25:09.347 INFO:tasks.workunit.client.1.smithi149.stdout:0/904: write d2/d9/d1a/d36/d85/db5/def/ff3 [890973,91338] 0 2024-04-24T08:25:09.349 INFO:tasks.workunit.client.1.smithi149.stdout:2/831: unlink d0/d6/db/d39/d3d/d44/d57/la2 0 2024-04-24T08:25:09.349 INFO:tasks.workunit.client.1.smithi149.stdout:0/905: chown d2/d3c/dc7 573305 1 2024-04-24T08:25:09.351 INFO:tasks.workunit.client.1.smithi149.stdout:6/824: rmdir d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8 39 2024-04-24T08:25:09.352 INFO:tasks.workunit.client.1.smithi149.stdout:0/906: fsync d2/d9/df/d48/dba/f110 0 2024-04-24T08:25:09.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/983: fdatasync d1/d2/d33/dea/d126/d137/f14a 0 2024-04-24T08:25:09.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/640: rmdir d3/d18 39 2024-04-24T08:25:09.366 INFO:tasks.workunit.client.0.smithi090.stdout:9/687: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f85 [0,4194304] 0 2024-04-24T08:25:09.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/832: unlink d0/d6/lc 0 2024-04-24T08:25:09.371 INFO:tasks.workunit.client.0.smithi090.stdout:1/697: dwrite d3/d25/faf [0,4194304] 0 2024-04-24T08:25:09.374 INFO:tasks.workunit.client.1.smithi149.stdout:4/921: truncate d2/d24/d2e/f4a 345243 0 2024-04-24T08:25:09.377 INFO:tasks.workunit.client.1.smithi149.stdout:6/825: link d0/d1a/c6d d0/d1a/d28/d31/d4c/d4e/d62/d66/d10c/c114 0 2024-04-24T08:25:09.384 INFO:tasks.workunit.client.1.smithi149.stdout:4/922: readlink d2/d18/db9/dd8/l53 0 2024-04-24T08:25:09.385 INFO:tasks.workunit.client.1.smithi149.stdout:0/907: dwrite d2/d9/f89 [0,4194304] 0 2024-04-24T08:25:09.387 INFO:tasks.workunit.client.1.smithi149.stdout:6/826: mkdir d0/d1a/d28/d31/d4c/df8/df9/d115 0 2024-04-24T08:25:09.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/908: mknod d2/d17/d41/d68/c11f 0 2024-04-24T08:25:09.399 INFO:tasks.workunit.client.1.smithi149.stdout:9/984: write d1/d2/d33/f120 [479240,21314] 0 2024-04-24T08:25:09.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/827: mknod d0/d1a/d28/d31/d33/c116 0 2024-04-24T08:25:09.401 INFO:tasks.workunit.client.1.smithi149.stdout:6/828: readlink d0/d1a/d28/lbe 0 2024-04-24T08:25:09.404 INFO:tasks.workunit.client.1.smithi149.stdout:0/909: mkdir d2/d9/d1a/d36/d85/db5/def/d120 0 2024-04-24T08:25:09.405 INFO:tasks.workunit.client.0.smithi090.stdout:6/734: sync 2024-04-24T08:25:09.408 INFO:tasks.workunit.client.1.smithi149.stdout:9/985: symlink d1/d2/d15/d1c/dca/l16b 0 2024-04-24T08:25:09.409 INFO:tasks.workunit.client.1.smithi149.stdout:2/833: write d0/d6/db/d39/f4e [5270967,3740] 0 2024-04-24T08:25:09.409 INFO:tasks.workunit.client.1.smithi149.stdout:0/910: readlink d2/d9/l105 0 2024-04-24T08:25:09.410 INFO:tasks.workunit.client.1.smithi149.stdout:9/986: dread - d1/da/d13/f142 zero size 2024-04-24T08:25:09.411 INFO:tasks.workunit.client.1.smithi149.stdout:4/923: getdents d2/d10/d91/d9f 0 2024-04-24T08:25:09.412 INFO:tasks.workunit.client.1.smithi149.stdout:9/987: chown d1/d2/d33/d60/d10b 205990729 1 2024-04-24T08:25:09.413 INFO:tasks.workunit.client.1.smithi149.stdout:6/829: rename d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/lc8 to d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/l117 0 2024-04-24T08:25:09.414 INFO:tasks.workunit.client.1.smithi149.stdout:9/988: truncate d1/d4c/f16a 808542 0 2024-04-24T08:25:09.415 INFO:tasks.workunit.client.1.smithi149.stdout:9/989: chown d1/de/f22 2 1 2024-04-24T08:25:09.418 INFO:tasks.workunit.client.0.smithi090.stdout:6/735: dwrite d8/d1d/d29/f40 [0,4194304] 0 2024-04-24T08:25:09.421 INFO:tasks.workunit.client.1.smithi149.stdout:6/830: symlink d0/db7/d112/l118 0 2024-04-24T08:25:09.431 INFO:tasks.workunit.client.1.smithi149.stdout:9/990: creat d1/d2/d33/f16c x:0 0 0 2024-04-24T08:25:09.432 INFO:tasks.workunit.client.0.smithi090.stdout:0/688: symlink d1a/d4b/d53/d79/d3c/le5 0 2024-04-24T08:25:09.435 INFO:tasks.workunit.client.0.smithi090.stdout:5/759: creat d8/da/d3a/d96/dcb/d21/d4c/ffb x:0 0 0 2024-04-24T08:25:09.437 INFO:tasks.workunit.client.0.smithi090.stdout:3/654: read d2/d15/d36/d6f/f9a [8539,16127] 0 2024-04-24T08:25:09.441 INFO:tasks.workunit.client.0.smithi090.stdout:5/760: truncate d8/d59/dc7/d7b/fd7 830389 0 2024-04-24T08:25:09.441 INFO:tasks.workunit.client.0.smithi090.stdout:1/698: symlink d3/d5/da/dc/lfa 0 2024-04-24T08:25:09.446 INFO:tasks.workunit.client.0.smithi090.stdout:2/764: link d0/d7/d67/d7f/d31/d90/f9d d0/d7/d28/daa/ddc/f101 0 2024-04-24T08:25:09.448 INFO:tasks.workunit.client.0.smithi090.stdout:3/655: mknod d2/df/d57/cd8 0 2024-04-24T08:25:09.450 INFO:tasks.workunit.client.0.smithi090.stdout:4/641: truncate d3/d18/d5c/f3d 1657267 0 2024-04-24T08:25:09.452 INFO:tasks.workunit.client.0.smithi090.stdout:3/656: dread d2/d15/d36/d6f/f9a [0,4194304] 0 2024-04-24T08:25:09.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/679: link d2/l29 d2/d62/d6b/d9c/dd8/lea 0 2024-04-24T08:25:09.453 INFO:tasks.workunit.client.0.smithi090.stdout:1/699: stat d3/d1d/lca 0 2024-04-24T08:25:09.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/689: mknod d1a/d4b/d53/ce6 0 2024-04-24T08:25:09.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/642: creat d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/fcc x:0 0 0 2024-04-24T08:25:09.464 INFO:tasks.workunit.client.0.smithi090.stdout:4/643: fdatasync d3/d8e/f97 0 2024-04-24T08:25:09.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/680: rmdir d2/da/d13/d64 39 2024-04-24T08:25:09.467 INFO:tasks.workunit.client.0.smithi090.stdout:0/690: mkdir d1a/d4b/d53/d79/d58/de7 0 2024-04-24T08:25:09.468 INFO:tasks.workunit.client.0.smithi090.stdout:1/700: dread d3/d5/f8d [0,4194304] 0 2024-04-24T08:25:09.468 INFO:tasks.workunit.client.0.smithi090.stdout:5/761: dread d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:25:09.469 INFO:tasks.workunit.client.0.smithi090.stdout:2/765: truncate d0/d7/d67/d7f/d31/f41 232188 0 2024-04-24T08:25:09.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/766: chown d0/d14 1158 1 2024-04-24T08:25:09.471 INFO:tasks.workunit.client.0.smithi090.stdout:2/767: fdatasync d0/d7/de1/fae 0 2024-04-24T08:25:09.471 INFO:tasks.workunit.client.0.smithi090.stdout:9/688: link d0/dc/d24/d7c/d4d/d32/d9f/lae d0/dc/ld8 0 2024-04-24T08:25:09.481 INFO:tasks.workunit.client.0.smithi090.stdout:3/657: link d2/d15/d36/d6f/d7a/fc9 d2/df/d1a/d37/d53/d60/d9b/dca/fd9 0 2024-04-24T08:25:09.483 INFO:tasks.workunit.client.0.smithi090.stdout:3/658: write d2/d15/f1d [3783186,58846] 0 2024-04-24T08:25:09.485 INFO:tasks.workunit.client.0.smithi090.stdout:3/659: read d2/df/d1a/f4c [4687846,38889] 0 2024-04-24T08:25:09.488 INFO:tasks.workunit.client.0.smithi090.stdout:2/768: symlink d0/d7/d67/d7f/d31/d90/l102 0 2024-04-24T08:25:09.492 INFO:tasks.workunit.client.0.smithi090.stdout:2/769: write d0/d7/d67/d7f/d31/d46/d60/fdf [718563,39517] 0 2024-04-24T08:25:09.494 INFO:tasks.workunit.client.0.smithi090.stdout:0/691: symlink d1a/le8 0 2024-04-24T08:25:09.497 INFO:tasks.workunit.client.1.smithi149.stdout:6/831: fsync d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fe8 0 2024-04-24T08:25:09.500 INFO:tasks.workunit.client.1.smithi149.stdout:2/834: unlink d0/d6/db/d39/d3d/d58/d98/cf5 0 2024-04-24T08:25:09.501 INFO:tasks.workunit.client.1.smithi149.stdout:2/835: stat d0/d6/db/d27/d81 0 2024-04-24T08:25:09.503 INFO:tasks.workunit.client.1.smithi149.stdout:9/991: fsync d1/d2/d33/f4e 0 2024-04-24T08:25:09.506 INFO:tasks.workunit.client.0.smithi090.stdout:9/689: fdatasync d0/d19/f46 0 2024-04-24T08:25:09.507 INFO:tasks.workunit.client.1.smithi149.stdout:0/911: link d2/d17/f5e d2/d9/d1a/f121 0 2024-04-24T08:25:09.509 INFO:tasks.workunit.client.1.smithi149.stdout:0/912: chown d2/d9/df/d2f/d6c/dce/l111 386 1 2024-04-24T08:25:09.513 INFO:tasks.workunit.client.0.smithi090.stdout:5/762: creat d8/d59/dc7/d7a/d50/ffc x:0 0 0 2024-04-24T08:25:09.515 INFO:tasks.workunit.client.0.smithi090.stdout:0/692: stat d1a/d4b/d53/f6a 0 2024-04-24T08:25:09.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/778: sync 2024-04-24T08:25:09.517 INFO:tasks.workunit.client.1.smithi149.stdout:4/924: rename d2/d24/c9a to d2/d18/d1e/c124 0 2024-04-24T08:25:09.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/992: mknod d1/da/dd/d79/db6/d115/c16d 0 2024-04-24T08:25:09.521 INFO:tasks.workunit.client.1.smithi149.stdout:9/993: chown d1/da/d13/d74/dbb/fcd 2012692 1 2024-04-24T08:25:09.525 INFO:tasks.workunit.client.0.smithi090.stdout:4/644: getdents d3/d18/d5c/db5 0 2024-04-24T08:25:09.528 INFO:tasks.workunit.client.0.smithi090.stdout:9/690: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9 0 2024-04-24T08:25:09.530 INFO:tasks.workunit.client.0.smithi090.stdout:5/763: mknod d8/da/d14/d2f/d81/da6/cfd 0 2024-04-24T08:25:09.536 INFO:tasks.workunit.client.1.smithi149.stdout:2/836: creat d0/d6/db/d27/d81/d9f/ffb x:0 0 0 2024-04-24T08:25:09.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/660: creat d2/d15/fda x:0 0 0 2024-04-24T08:25:09.539 INFO:tasks.workunit.client.1.smithi149.stdout:6/832: getdents d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec 0 2024-04-24T08:25:09.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/645: creat d3/d18/d5c/db5/d7c/fcd x:0 0 0 2024-04-24T08:25:09.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/681: dwrite d2/da/d13/d64/d50/f5d [4194304,4194304] 0 2024-04-24T08:25:09.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:09 smithi090 ceph-mon[25016]: pgmap v27: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 246 MiB/s wr, 474 op/s 2024-04-24T08:25:09.543 INFO:tasks.workunit.client.0.smithi090.stdout:4/646: write d3/d18/fbc [231861,29447] 0 2024-04-24T08:25:09.546 INFO:tasks.workunit.client.0.smithi090.stdout:8/682: write d2/da/d13/d64/d24/f5b [3049229,15159] 0 2024-04-24T08:25:09.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/683: chown d2 0 1 2024-04-24T08:25:09.547 INFO:tasks.workunit.client.1.smithi149.stdout:4/925: rename d2/fc2 to d2/d18/f125 0 2024-04-24T08:25:09.550 INFO:tasks.workunit.client.1.smithi149.stdout:6/833: rmdir d0/d1a/d28/d31/d4c/d76 39 2024-04-24T08:25:09.555 INFO:tasks.workunit.client.0.smithi090.stdout:2/770: dread d0/d7/d67/d7f/d31/f41 [0,4194304] 0 2024-04-24T08:25:09.559 INFO:tasks.workunit.client.1.smithi149.stdout:6/834: truncate d0/d1a/d28/f95 628188 0 2024-04-24T08:25:09.560 INFO:tasks.workunit.client.0.smithi090.stdout:5/764: mkdir d8/da/d3a/d2d/d37/dec/dfe 0 2024-04-24T08:25:09.560 INFO:tasks.workunit.client.1.smithi149.stdout:9/994: write d1/d2/d15/d1c/d42/dae/fd8 [714368,44215] 0 2024-04-24T08:25:09.566 INFO:tasks.workunit.client.0.smithi090.stdout:7/779: dwrite db/d17/d29/d32/d5a/d75/fa4 [0,4194304] 0 2024-04-24T08:25:09.567 INFO:tasks.workunit.client.1.smithi149.stdout:0/913: dwrite d2/d9/d1a/f6b [0,4194304] 0 2024-04-24T08:25:09.567 INFO:tasks.workunit.client.1.smithi149.stdout:6/835: truncate d0/d1a/d28/d31/f37 4779774 0 2024-04-24T08:25:09.567 INFO:tasks.workunit.client.0.smithi090.stdout:0/693: link d1a/d4b/fa0 d1a/d4b/d53/d79/d69/d90/dcc/fe9 0 2024-04-24T08:25:09.568 INFO:tasks.workunit.client.0.smithi090.stdout:7/780: stat db/d17/d39/c5d 0 2024-04-24T08:25:09.568 INFO:tasks.workunit.client.1.smithi149.stdout:2/837: write d0/f77 [5006049,51179] 0 2024-04-24T08:25:09.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/995: creat d1/d2/d15/de7/f16e x:0 0 0 2024-04-24T08:25:09.570 INFO:tasks.workunit.client.1.smithi149.stdout:6/836: chown d0/db7/fb9 55329 1 2024-04-24T08:25:09.571 INFO:tasks.workunit.client.1.smithi149.stdout:8/904: sync 2024-04-24T08:25:09.573 INFO:tasks.workunit.client.0.smithi090.stdout:8/684: mkdir d2/d62/d83/da3/deb 0 2024-04-24T08:25:09.578 INFO:tasks.workunit.client.0.smithi090.stdout:9/691: write d0/fd0 [324559,52464] 0 2024-04-24T08:25:09.578 INFO:tasks.workunit.client.1.smithi149.stdout:2/838: readlink d0/d6/lfa 0 2024-04-24T08:25:09.580 INFO:tasks.workunit.client.0.smithi090.stdout:0/694: dread d1a/d4b/d53/d79/d3c/f5b [0,4194304] 0 2024-04-24T08:25:09.581 INFO:tasks.workunit.client.0.smithi090.stdout:5/765: rmdir d8/da/d14/d2f/d81 39 2024-04-24T08:25:09.583 INFO:tasks.workunit.client.0.smithi090.stdout:6/736: sync 2024-04-24T08:25:09.585 INFO:tasks.workunit.client.1.smithi149.stdout:6/837: creat d0/d1a/d28/d31/d4c/df8/df9/f119 x:0 0 0 2024-04-24T08:25:09.586 INFO:tasks.workunit.client.0.smithi090.stdout:4/647: mknod d3/d18/d1d/cce 0 2024-04-24T08:25:09.588 INFO:tasks.workunit.client.1.smithi149.stdout:8/905: mknod d1/de/d18/d32/d65/d8b/c136 0 2024-04-24T08:25:09.588 INFO:tasks.workunit.client.0.smithi090.stdout:7/781: creat db/d54/d78/f10e x:0 0 0 2024-04-24T08:25:09.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/692: mkdir d0/dc/d24/d2d/dda 0 2024-04-24T08:25:09.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/695: mkdir d1a/d4b/d53/d79/d35/dea 0 2024-04-24T08:25:09.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/696: stat d1a/d4b/d53/d79/d69/d8a/ce1 0 2024-04-24T08:25:09.596 INFO:tasks.workunit.client.1.smithi149.stdout:2/839: mkdir d0/d6/db/dfc 0 2024-04-24T08:25:09.596 INFO:tasks.workunit.client.0.smithi090.stdout:3/661: dwrite d2/df/d1a/d37/d53/d60/d9b/dca/fd9 [0,4194304] 0 2024-04-24T08:25:09.602 INFO:tasks.workunit.client.1.smithi149.stdout:0/914: write d2/d9/df/d48/f52 [270162,22421] 0 2024-04-24T08:25:09.603 INFO:tasks.workunit.client.1.smithi149.stdout:9/996: write d1/d2/d33/d60/d10b/d141/f146 [436512,38654] 0 2024-04-24T08:25:09.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/926: dwrite d2/d24/d2e/d3f/d94/dfd/f10a [0,4194304] 0 2024-04-24T08:25:09.607 INFO:tasks.workunit.client.0.smithi090.stdout:4/648: mknod d3/d18/d5c/db5/d69/d8b/db2/d49/ccf 0 2024-04-24T08:25:09.609 INFO:tasks.workunit.client.1.smithi149.stdout:8/906: rename d1/de/d18/d32/d65/f131 to d1/de/d18/d32/d65/dad/f137 0 2024-04-24T08:25:09.612 INFO:tasks.workunit.client.0.smithi090.stdout:2/771: dwrite d0/d14/f53 [0,4194304] 0 2024-04-24T08:25:09.617 INFO:tasks.workunit.client.1.smithi149.stdout:0/915: symlink d2/d9/d1a/d36/dcc/l122 0 2024-04-24T08:25:09.619 INFO:tasks.workunit.client.0.smithi090.stdout:0/697: dread d1a/d4b/d53/d79/d3c/f5b [0,4194304] 0 2024-04-24T08:25:09.621 INFO:tasks.workunit.client.0.smithi090.stdout:0/698: chown d1a/d4b/d53/d79/d35/c48 5969897 1 2024-04-24T08:25:09.622 INFO:tasks.workunit.client.1.smithi149.stdout:9/997: mknod d1/d2/dc8/dc9/c16f 0 2024-04-24T08:25:09.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/927: symlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/l126 0 2024-04-24T08:25:09.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/907: symlink d1/de/d18/d32/d65/d8b/dc9/l138 0 2024-04-24T08:25:09.631 INFO:tasks.workunit.client.1.smithi149.stdout:6/838: write d0/d1a/d28/d31/d4c/d80/f68 [3019780,84784] 0 2024-04-24T08:25:09.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/693: getdents d0/da5 0 2024-04-24T08:25:09.631 INFO:tasks.workunit.client.1.smithi149.stdout:0/916: symlink d2/d17/d41/d68/ddc/l123 0 2024-04-24T08:25:09.634 INFO:tasks.workunit.client.0.smithi090.stdout:8/685: dread d2/da/d13/f5f [0,4194304] 0 2024-04-24T08:25:09.634 INFO:tasks.workunit.client.0.smithi090.stdout:7/782: rename db/d17/d29/d38/d3e/lcf to db/d54/d78/l10f 0 2024-04-24T08:25:09.638 INFO:tasks.workunit.client.0.smithi090.stdout:2/772: dwrite d0/d7/f95 [0,4194304] 0 2024-04-24T08:25:09.639 INFO:tasks.workunit.client.0.smithi090.stdout:5/766: link d8/da/d3a/d96/dcb/d21/d4c/cd8 d8/da/d3a/d96/dcb/de3/cff 0 2024-04-24T08:25:09.643 INFO:tasks.workunit.client.1.smithi149.stdout:8/908: rmdir d1/d1f/d25/d7a 39 2024-04-24T08:25:09.645 INFO:tasks.workunit.client.1.smithi149.stdout:9/998: creat d1/d2/d33/dea/d133/dad/f170 x:0 0 0 2024-04-24T08:25:09.646 INFO:tasks.workunit.client.0.smithi090.stdout:4/649: mkdir d3/dd0 0 2024-04-24T08:25:09.646 INFO:tasks.workunit.client.0.smithi090.stdout:0/699: symlink d1a/d4b/d53/d79/d69/d8a/leb 0 2024-04-24T08:25:09.651 INFO:tasks.workunit.client.1.smithi149.stdout:6/839: truncate d0/d1a/d28/d31/d33/f9b 1024493 0 2024-04-24T08:25:09.654 INFO:tasks.workunit.client.0.smithi090.stdout:7/783: read - db/de9/ff5 zero size 2024-04-24T08:25:09.656 INFO:tasks.workunit.client.0.smithi090.stdout:3/662: dwrite d2/d25/d33/f80 [0,4194304] 0 2024-04-24T08:25:09.658 INFO:tasks.workunit.client.0.smithi090.stdout:2/773: rename d0/d7/d28/daa/def/d50/c80 to d0/d14/d36/c103 0 2024-04-24T08:25:09.662 INFO:tasks.workunit.client.0.smithi090.stdout:2/774: write d0/d2e/f49 [2485656,82124] 0 2024-04-24T08:25:09.666 INFO:tasks.workunit.client.0.smithi090.stdout:2/775: chown d0/d7/dda 11 1 2024-04-24T08:25:09.667 INFO:tasks.workunit.client.1.smithi149.stdout:6/840: rename d0/d1a/d28/d31/d38/l3b to d0/d1a/d28/d31/d4c/d4e/d62/l11a 0 2024-04-24T08:25:09.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/767: dwrite d8/d59/dc7/d7a/d50/f79 [0,4194304] 0 2024-04-24T08:25:09.668 INFO:tasks.workunit.client.1.smithi149.stdout:0/917: link d2/d9/df/d2f/c35 d2/d9/de6/c124 0 2024-04-24T08:25:09.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/686: creat d2/d62/d83/da3/deb/fec x:0 0 0 2024-04-24T08:25:09.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/663: mknod d2/df/d57/d4f/cdb 0 2024-04-24T08:25:09.673 INFO:tasks.workunit.client.1.smithi149.stdout:4/928: dwrite d2/d24/da7/fd0 [4194304,4194304] 0 2024-04-24T08:25:09.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/841: symlink d0/d1a/d28/d31/d38/l11b 0 2024-04-24T08:25:09.679 INFO:tasks.workunit.client.1.smithi149.stdout:2/840: dread d0/d6/db/d39/d3d/d44/df0/ff9 [0,4194304] 0 2024-04-24T08:25:09.681 INFO:tasks.workunit.client.1.smithi149.stdout:0/918: rmdir d2/d9/d1a/d36/d85/db5/def 39 2024-04-24T08:25:09.684 INFO:tasks.workunit.client.0.smithi090.stdout:3/664: chown d2/d25/d33/c46 12647875 1 2024-04-24T08:25:09.687 INFO:tasks.workunit.client.0.smithi090.stdout:8/687: readlink d2/d62/d6b/d9c/dd8/lea 0 2024-04-24T08:25:09.688 INFO:tasks.workunit.client.1.smithi149.stdout:0/919: mknod d2/d9/df/d56/c125 0 2024-04-24T08:25:09.690 INFO:tasks.workunit.client.0.smithi090.stdout:9/694: dwrite d0/dc/d24/d2d/d49/d81/f9a [0,4194304] 0 2024-04-24T08:25:09.695 INFO:tasks.workunit.client.1.smithi149.stdout:2/841: rename d0/d6/db/d27/c7b to d0/d6/db/d27/d81/db4/cfd 0 2024-04-24T08:25:09.696 INFO:tasks.workunit.client.0.smithi090.stdout:9/695: dread - d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/fb7 zero size 2024-04-24T08:25:09.696 INFO:tasks.workunit.client.1.smithi149.stdout:8/909: dwrite d1/d1f/f2c [0,4194304] 0 2024-04-24T08:25:09.700 INFO:tasks.workunit.client.1.smithi149.stdout:2/842: dread - d0/d6/db/d27/d81/d9f/ffb zero size 2024-04-24T08:25:09.700 INFO:tasks.workunit.client.0.smithi090.stdout:2/776: write d0/f22 [2266051,95014] 0 2024-04-24T08:25:09.703 INFO:tasks.workunit.client.1.smithi149.stdout:6/842: getdents d0/d1a/d28/d31/d4c/d80/d6f/d97 0 2024-04-24T08:25:09.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/929: dwrite d2/d10/d91/f112 [0,4194304] 0 2024-04-24T08:25:09.706 INFO:tasks.workunit.client.1.smithi149.stdout:6/843: read d0/f11 [6673891,56225] 0 2024-04-24T08:25:09.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/665: creat d2/d15/da6/fdc x:0 0 0 2024-04-24T08:25:09.707 INFO:tasks.workunit.client.0.smithi090.stdout:0/700: write d1a/d4b/d53/d79/d35/d68/f7b [408740,805] 0 2024-04-24T08:25:09.709 INFO:tasks.workunit.client.0.smithi090.stdout:4/650: dwrite d3/f13 [0,4194304] 0 2024-04-24T08:25:09.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/910: mknod d1/dce/c139 0 2024-04-24T08:25:09.714 INFO:tasks.workunit.client.1.smithi149.stdout:0/920: creat d2/d9/df/d48/f126 x:0 0 0 2024-04-24T08:25:09.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/688: mknod d2/d97/ced 0 2024-04-24T08:25:09.714 INFO:tasks.workunit.client.1.smithi149.stdout:4/930: symlink d2/d24/d7f/l127 0 2024-04-24T08:25:09.715 INFO:tasks.workunit.client.0.smithi090.stdout:9/696: fdatasync d0/dc/d24/d2d/d49/d62/f68 0 2024-04-24T08:25:09.721 INFO:tasks.workunit.client.0.smithi090.stdout:2/777: dwrite d0/d14/d36/dce/dee/fd3 [0,4194304] 0 2024-04-24T08:25:09.723 INFO:tasks.workunit.client.1.smithi149.stdout:2/843: mknod d0/d6/db/d39/cfe 0 2024-04-24T08:25:09.725 INFO:tasks.workunit.client.0.smithi090.stdout:4/651: write d3/d87/f9f [3007277,63600] 0 2024-04-24T08:25:09.730 INFO:tasks.workunit.client.0.smithi090.stdout:5/768: getdents d8/d59/dc7/d7a/d50/d9f 0 2024-04-24T08:25:09.735 INFO:tasks.workunit.client.1.smithi149.stdout:6/844: mkdir d0/d1a/d28/d31/d4c/d80/d11c 0 2024-04-24T08:25:09.743 INFO:tasks.workunit.client.1.smithi149.stdout:4/931: creat d2/d10/d91/f128 x:0 0 0 2024-04-24T08:25:09.744 INFO:tasks.workunit.client.0.smithi090.stdout:8/689: readlink d2/da/d13/d64/db2/db6/dd0/le7 0 2024-04-24T08:25:09.746 INFO:tasks.workunit.client.1.smithi149.stdout:2/844: fdatasync d0/d6/db/d39/d78/d7c/f97 0 2024-04-24T08:25:09.752 INFO:tasks.workunit.client.1.smithi149.stdout:6/845: symlink d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/l11d 0 2024-04-24T08:25:09.756 INFO:tasks.workunit.client.0.smithi090.stdout:5/769: creat d8/d59/dc7/d7a/f100 x:0 0 0 2024-04-24T08:25:09.759 INFO:tasks.workunit.client.1.smithi149.stdout:8/911: creat d1/de/d18/d32/f13a x:0 0 0 2024-04-24T08:25:09.762 INFO:tasks.workunit.client.0.smithi090.stdout:9/697: write d0/f59 [1762465,5186] 0 2024-04-24T08:25:09.767 INFO:tasks.workunit.client.1.smithi149.stdout:0/921: creat d2/d9/df/f127 x:0 0 0 2024-04-24T08:25:09.767 INFO:tasks.workunit.client.1.smithi149.stdout:6/846: dread d0/f3f [0,4194304] 0 2024-04-24T08:25:09.768 INFO:tasks.workunit.client.0.smithi090.stdout:1/701: sync 2024-04-24T08:25:09.769 INFO:tasks.workunit.client.0.smithi090.stdout:5/770: unlink d8/da/d14/d2f/d44/f49 0 2024-04-24T08:25:09.769 INFO:tasks.workunit.client.0.smithi090.stdout:9/698: truncate d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d92/fc1 487389 0 2024-04-24T08:25:09.770 INFO:tasks.workunit.client.1.smithi149.stdout:6/847: chown d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f81 15099 1 2024-04-24T08:25:09.770 INFO:tasks.workunit.client.0.smithi090.stdout:1/702: chown d3/d5/da/dc/l20 1278166150 1 2024-04-24T08:25:09.782 INFO:tasks.workunit.client.1.smithi149.stdout:2/845: creat d0/d6/db/d39/d3d/d44/d75/fff x:0 0 0 2024-04-24T08:25:09.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/778: link d0/d7/d28/l69 d0/d14/d17/dc2/dcb/l104 0 2024-04-24T08:25:09.783 INFO:tasks.workunit.client.0.smithi090.stdout:5/771: dread - d8/da/d14/d2f/d44/f73 zero size 2024-04-24T08:25:09.785 INFO:tasks.workunit.client.1.smithi149.stdout:8/912: symlink d1/de/d18/d32/d63/d9e/dec/l13b 0 2024-04-24T08:25:09.785 INFO:tasks.workunit.client.0.smithi090.stdout:4/652: dread d3/d24/f5d [0,4194304] 0 2024-04-24T08:25:09.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/701: dread f8 [0,4194304] 0 2024-04-24T08:25:09.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:09 smithi149 ceph-mon[31282]: pgmap v27: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 246 MiB/s wr, 474 op/s 2024-04-24T08:25:09.786 INFO:tasks.workunit.client.0.smithi090.stdout:2/779: chown d0/d7/d28/daa/def/d50/d96 30516 1 2024-04-24T08:25:09.787 INFO:tasks.workunit.client.0.smithi090.stdout:3/666: getdents d2/d15/d36/d6f 0 2024-04-24T08:25:09.787 INFO:tasks.workunit.client.0.smithi090.stdout:5/772: write d8/d76/dad/fd1 [941876,49915] 0 2024-04-24T08:25:09.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/690: dread d2/da/d13/d64/f8e [0,4194304] 0 2024-04-24T08:25:09.787 INFO:tasks.workunit.client.0.smithi090.stdout:9/699: truncate d0/fb 3953076 0 2024-04-24T08:25:09.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/703: creat d3/d5/da/d97/d9c/dc4/ffb x:0 0 0 2024-04-24T08:25:09.788 INFO:tasks.workunit.client.1.smithi149.stdout:6/848: rename d0/d1a/d28/d31/d4c/d80/d6f/d10f to d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/d11e 0 2024-04-24T08:25:09.793 INFO:tasks.workunit.client.0.smithi090.stdout:8/691: symlink d2/da/d13/d64/d24/lee 0 2024-04-24T08:25:09.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/704: rmdir d3/d5/da/d94/da6 39 2024-04-24T08:25:09.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/667: dwrite d2/df/db5/fd6 [0,4194304] 0 2024-04-24T08:25:09.803 INFO:tasks.workunit.client.0.smithi090.stdout:4/653: creat d3/fd1 x:0 0 0 2024-04-24T08:25:09.803 INFO:tasks.workunit.client.0.smithi090.stdout:9/700: dwrite d0/dc/d24/d2d/f3f [0,4194304] 0 2024-04-24T08:25:09.805 INFO:tasks.workunit.client.0.smithi090.stdout:5/773: write d8/d76/dad/ff6 [155164,94940] 0 2024-04-24T08:25:09.807 INFO:tasks.workunit.client.0.smithi090.stdout:2/780: truncate d0/d7/fd 2149157 0 2024-04-24T08:25:09.809 INFO:tasks.workunit.client.0.smithi090.stdout:2/781: chown d0/d14/d36 1315720 1 2024-04-24T08:25:09.816 INFO:tasks.workunit.client.0.smithi090.stdout:8/692: getdents d2/da/d13/d64/dcc 0 2024-04-24T08:25:09.817 INFO:tasks.workunit.client.0.smithi090.stdout:5/774: read d8/d99/f9d [2167484,78778] 0 2024-04-24T08:25:09.819 INFO:tasks.workunit.client.0.smithi090.stdout:4/654: write d3/d18/d5c/f66 [240665,121012] 0 2024-04-24T08:25:09.820 INFO:tasks.workunit.client.0.smithi090.stdout:2/782: rmdir d0/d7/d28/daa/db1 39 2024-04-24T08:25:09.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/775: chown d8/da/d14/d2f/d44/c83 614966 1 2024-04-24T08:25:09.824 INFO:tasks.workunit.client.0.smithi090.stdout:8/693: truncate d2/da/d13/d34/fc3 1016448 0 2024-04-24T08:25:09.825 INFO:tasks.workunit.client.0.smithi090.stdout:5/776: fsync d8/da/d3a/f43 0 2024-04-24T08:25:09.828 INFO:tasks.workunit.client.0.smithi090.stdout:8/694: mkdir d2/d97/def 0 2024-04-24T08:25:09.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/695: write d2/da/f82 [334274,50717] 0 2024-04-24T08:25:09.833 INFO:tasks.workunit.client.0.smithi090.stdout:1/705: dwrite d3/d5/da/f64 [0,4194304] 0 2024-04-24T08:25:09.833 INFO:tasks.workunit.client.0.smithi090.stdout:0/702: dread d1a/d4b/d53/f7f [0,4194304] 0 2024-04-24T08:25:09.834 INFO:tasks.workunit.client.0.smithi090.stdout:4/655: rename d3/d18/d5c/db5/d69/d8b/db2/d49/l67 to d3/d87/da6/ld2 0 2024-04-24T08:25:09.835 INFO:tasks.workunit.client.0.smithi090.stdout:5/777: creat d8/da/d3a/d2d/d37/dec/dfe/f101 x:0 0 0 2024-04-24T08:25:09.836 INFO:tasks.workunit.client.0.smithi090.stdout:8/696: chown d2/f10 673 1 2024-04-24T08:25:09.844 INFO:tasks.workunit.client.0.smithi090.stdout:2/783: dwrite d0/d14/d17/f62 [0,4194304] 0 2024-04-24T08:25:09.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/701: dwrite d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf [0,4194304] 0 2024-04-24T08:25:09.848 INFO:tasks.workunit.client.0.smithi090.stdout:2/784: stat d0/d7/d28/daa 0 2024-04-24T08:25:09.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/706: rename d3/d5/da/f6a to d3/d5/d3e/da3/ffc 0 2024-04-24T08:25:09.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/668: dread d2/d25/d33/f43 [0,4194304] 0 2024-04-24T08:25:09.859 INFO:tasks.workunit.client.0.smithi090.stdout:5/778: dwrite d8/d59/fdf [0,4194304] 0 2024-04-24T08:25:09.862 INFO:tasks.workunit.client.0.smithi090.stdout:1/707: truncate d3/d25/d30/f79 911086 0 2024-04-24T08:25:09.862 INFO:tasks.workunit.client.0.smithi090.stdout:3/669: unlink d2/df/d1a/d37/d59/d98/cb1 0 2024-04-24T08:25:09.863 INFO:tasks.workunit.client.0.smithi090.stdout:2/785: truncate d0/d7/d67/d7f/d31/d46/d60/f74 7997204 0 2024-04-24T08:25:09.877 INFO:tasks.workunit.client.0.smithi090.stdout:1/708: truncate d3/d25/d30/dc5/fcf 753829 0 2024-04-24T08:25:09.881 INFO:tasks.workunit.client.0.smithi090.stdout:3/670: mknod d2/df/db5/dd7/cdd 0 2024-04-24T08:25:09.881 INFO:tasks.workunit.client.0.smithi090.stdout:2/786: symlink d0/d7/d28/daa/ddc/l105 0 2024-04-24T08:25:09.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/709: fsync d3/d5/da/dc/d80/fbc 0 2024-04-24T08:25:09.881 INFO:tasks.workunit.client.0.smithi090.stdout:9/702: link d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/cb1 d0/da4/cdb 0 2024-04-24T08:25:09.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/710: creat d3/d68/df2/ffd x:0 0 0 2024-04-24T08:25:09.883 INFO:tasks.workunit.client.0.smithi090.stdout:5/779: dwrite d8/d25/f26 [0,4194304] 0 2024-04-24T08:25:09.884 INFO:tasks.workunit.client.0.smithi090.stdout:3/671: rename d2/d25/d33/l89 to d2/df/d1a/d37/d53/d60/lde 0 2024-04-24T08:25:09.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/780: stat d8/d9c 0 2024-04-24T08:25:09.888 INFO:tasks.workunit.client.0.smithi090.stdout:4/656: dread d3/d18/d1d/fab [0,4194304] 0 2024-04-24T08:25:09.890 INFO:tasks.workunit.client.0.smithi090.stdout:1/711: rmdir d3/d1d/d54 39 2024-04-24T08:25:09.890 INFO:tasks.workunit.client.0.smithi090.stdout:9/703: unlink d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/l4c 0 2024-04-24T08:25:09.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/781: write d8/da/d3a/d2d/d37/dec/dfe/f101 [841574,5396] 0 2024-04-24T08:25:09.891 INFO:tasks.workunit.client.0.smithi090.stdout:3/672: creat d2/d15/da6/fdf x:0 0 0 2024-04-24T08:25:09.894 INFO:tasks.workunit.client.0.smithi090.stdout:8/697: dread d2/f59 [0,4194304] 0 2024-04-24T08:25:09.913 INFO:tasks.workunit.client.0.smithi090.stdout:5/782: symlink d8/d25/l102 0 2024-04-24T08:25:09.914 INFO:tasks.workunit.client.0.smithi090.stdout:9/704: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/ddc 0 2024-04-24T08:25:09.914 INFO:tasks.workunit.client.0.smithi090.stdout:9/705: stat d0/dc/d24/d7c/ld5 0 2024-04-24T08:25:09.914 INFO:tasks.workunit.client.0.smithi090.stdout:2/787: write d0/d5e/f9a [416817,89304] 0 2024-04-24T08:25:09.914 INFO:tasks.workunit.client.0.smithi090.stdout:3/673: creat d2/d15/fe0 x:0 0 0 2024-04-24T08:25:09.914 INFO:tasks.workunit.client.0.smithi090.stdout:8/698: creat d2/da/d13/ff0 x:0 0 0 2024-04-24T08:25:09.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/706: creat d0/dc/d24/d7c/d4d/d8c/fdd x:0 0 0 2024-04-24T08:25:09.916 INFO:tasks.workunit.client.0.smithi090.stdout:4/657: dwrite d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/f51 [0,4194304] 0 2024-04-24T08:25:09.917 INFO:tasks.workunit.client.0.smithi090.stdout:1/712: dwrite d3/d5/da/d97/d9c/fe2 [0,4194304] 0 2024-04-24T08:25:09.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/788: chown d0/d7/d67/d7f/d31/cdd 112266 1 2024-04-24T08:25:09.917 INFO:tasks.workunit.client.0.smithi090.stdout:3/674: stat d2/df/d1a/f4c 0 2024-04-24T08:25:09.917 INFO:tasks.workunit.client.0.smithi090.stdout:1/713: chown d3/d5/da 479398 1 2024-04-24T08:25:09.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/789: unlink d0/d14/f2b 0 2024-04-24T08:25:09.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/790: readlink d0/d7/d67/d7f/d31/d46/d60/lab 0 2024-04-24T08:25:09.920 INFO:tasks.workunit.client.0.smithi090.stdout:5/783: dwrite d8/d59/dc7/d7b/fd7 [0,4194304] 0 2024-04-24T08:25:09.921 INFO:tasks.workunit.client.0.smithi090.stdout:2/791: fdatasync d0/d14/f1d 0 2024-04-24T08:25:09.922 INFO:tasks.workunit.client.0.smithi090.stdout:3/675: creat d2/df/d1a/d37/d53/fe1 x:0 0 0 2024-04-24T08:25:09.925 INFO:tasks.workunit.client.0.smithi090.stdout:4/658: symlink d3/d18/d5c/db5/d77/ld3 0 2024-04-24T08:25:09.927 INFO:tasks.workunit.client.0.smithi090.stdout:4/659: dread - d3/d18/d5c/db5/f9a zero size 2024-04-24T08:25:09.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/707: rmdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96 39 2024-04-24T08:25:09.932 INFO:tasks.workunit.client.0.smithi090.stdout:1/714: truncate d3/d5/d58/f60 4563126 0 2024-04-24T08:25:09.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/699: truncate d2/f9d 3363312 0 2024-04-24T08:25:09.940 INFO:tasks.workunit.client.0.smithi090.stdout:1/715: rmdir d3/d1d/d34/d73/dd8 39 2024-04-24T08:25:09.940 INFO:tasks.workunit.client.0.smithi090.stdout:6/737: sync 2024-04-24T08:25:09.942 INFO:tasks.workunit.client.0.smithi090.stdout:2/792: mknod d0/d7/d28/daa/def/db4/dcd/c106 0 2024-04-24T08:25:09.942 INFO:tasks.workunit.client.0.smithi090.stdout:4/660: symlink d3/d18/d5c/db5/d69/d8b/db2/ld4 0 2024-04-24T08:25:09.943 INFO:tasks.workunit.client.0.smithi090.stdout:1/716: read - d3/d5/fa5 zero size 2024-04-24T08:25:09.944 INFO:tasks.workunit.client.0.smithi090.stdout:8/700: chown d2/da/d13/d64/d24/db3/db4/dcd/de8 21135 1 2024-04-24T08:25:09.944 INFO:tasks.workunit.client.0.smithi090.stdout:3/676: dwrite d2/d15/d36/d6f/d92/fa9 [0,4194304] 0 2024-04-24T08:25:09.945 INFO:tasks.workunit.client.0.smithi090.stdout:6/738: readlink d8/d1d/l37 0 2024-04-24T08:25:09.945 INFO:tasks.workunit.client.0.smithi090.stdout:5/784: getdents d8/d59/dc7/d7a/dbc 0 2024-04-24T08:25:09.947 INFO:tasks.workunit.client.0.smithi090.stdout:4/661: creat d3/d8e/fd5 x:0 0 0 2024-04-24T08:25:09.950 INFO:tasks.workunit.client.0.smithi090.stdout:8/701: write d2/da/d13/d64/db2/fd7 [779469,23740] 0 2024-04-24T08:25:09.951 INFO:tasks.workunit.client.0.smithi090.stdout:1/717: mkdir d3/d1d/dfe 0 2024-04-24T08:25:09.954 INFO:tasks.workunit.client.0.smithi090.stdout:5/785: dread - d8/da/d14/d2f/d81/f98 zero size 2024-04-24T08:25:09.956 INFO:tasks.workunit.client.0.smithi090.stdout:4/662: creat d3/d18/fd6 x:0 0 0 2024-04-24T08:25:09.964 INFO:tasks.workunit.client.0.smithi090.stdout:4/663: stat d3/d18/d5c/db5/d69/d8b/db2/d6b/fb4 0 2024-04-24T08:25:09.967 INFO:tasks.workunit.client.0.smithi090.stdout:9/708: write d0/d19/f46 [2247561,125268] 0 2024-04-24T08:25:09.970 INFO:tasks.workunit.client.0.smithi090.stdout:6/739: dwrite d8/dc/f10 [0,4194304] 0 2024-04-24T08:25:09.974 INFO:tasks.workunit.client.0.smithi090.stdout:4/664: write d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/fcc [374743,128874] 0 2024-04-24T08:25:09.974 INFO:tasks.workunit.client.0.smithi090.stdout:1/718: write d3/d5/f33 [1323172,117702] 0 2024-04-24T08:25:09.977 INFO:tasks.workunit.client.0.smithi090.stdout:9/709: truncate d0/dc/f2c 804318 0 2024-04-24T08:25:09.978 INFO:tasks.workunit.client.0.smithi090.stdout:0/703: sync 2024-04-24T08:25:09.979 INFO:tasks.workunit.client.0.smithi090.stdout:7/784: sync 2024-04-24T08:25:09.981 INFO:tasks.workunit.client.1.smithi149.stdout:4/932: link d2/d24/d2e/d38/f5c d2/d24/d2e/d3f/f129 0 2024-04-24T08:25:09.982 INFO:tasks.workunit.client.1.smithi149.stdout:2/846: rename d0/d6/fae to d0/d6/db/d39/d78/d7c/df1/f100 0 2024-04-24T08:25:09.984 INFO:tasks.workunit.client.0.smithi090.stdout:7/785: dread - db/d54/d78/ffe zero size 2024-04-24T08:25:09.984 INFO:tasks.workunit.client.0.smithi090.stdout:9/710: write d0/dc/d24/d7c/d4d/d32/d9f/f9b [2483367,94162] 0 2024-04-24T08:25:09.986 INFO:tasks.workunit.client.0.smithi090.stdout:4/665: rmdir d3/d18/d5c/db5/d69/d8b/db2/d49 39 2024-04-24T08:25:09.986 INFO:tasks.workunit.client.1.smithi149.stdout:6/849: chown d0/d1a/d28/d31/d4c/d8a/fb5 7416316 1 2024-04-24T08:25:09.988 INFO:tasks.workunit.client.0.smithi090.stdout:3/677: dread d2/df/d57/d4f/f72 [0,4194304] 0 2024-04-24T08:25:09.988 INFO:tasks.workunit.client.1.smithi149.stdout:4/933: truncate d2/d24/d2e/d38/fd1 833565 0 2024-04-24T08:25:09.991 INFO:tasks.workunit.client.0.smithi090.stdout:6/740: link d8/d1d/f6f d8/d1d/d1f/d8c/fdf 0 2024-04-24T08:25:09.991 INFO:tasks.workunit.client.1.smithi149.stdout:8/913: dread d1/d1f/d5e/f73 [0,4194304] 0 2024-04-24T08:25:09.992 INFO:tasks.workunit.client.1.smithi149.stdout:4/934: chown d2/d24/d2e/f5b 138593245 1 2024-04-24T08:25:09.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/704: readlink d1a/d4b/d53/d79/d35/d62/l66 0 2024-04-24T08:25:09.994 INFO:tasks.workunit.client.1.smithi149.stdout:6/850: rename d0/d1a/d28/d31/d4c/d8a to d0/d1a/d28/d4a/d11f 0 2024-04-24T08:25:09.998 INFO:tasks.workunit.client.0.smithi090.stdout:9/711: write d0/dc/d24/d7c/d4d/d32/d9f/f9b [2282913,51448] 0 2024-04-24T08:25:10.000 INFO:tasks.workunit.client.0.smithi090.stdout:7/786: fsync db/d90/faf 0 2024-04-24T08:25:10.002 INFO:tasks.workunit.client.0.smithi090.stdout:4/666: mknod d3/d18/d1d/dcb/cd7 0 2024-04-24T08:25:10.006 INFO:tasks.workunit.client.0.smithi090.stdout:5/786: dwrite d8/d59/dc7/d7b/ff9 [4194304,4194304] 0 2024-04-24T08:25:10.006 INFO:tasks.workunit.client.0.smithi090.stdout:0/705: dread d1a/d4b/d53/d79/d35/d68/f7b [0,4194304] 0 2024-04-24T08:25:10.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/706: rmdir d1a/d4b/d53/d57/dc8 0 2024-04-24T08:25:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:7/787: truncate db/d17/d29/d32/d5a/d75/f10c 535592 0 2024-04-24T08:25:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:4/667: stat d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/c28 0 2024-04-24T08:25:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:6/741: link d8/dc/d19/d8e/d38/cb7 d8/dc/d19/d8e/d38/ce0 0 2024-04-24T08:25:10.023 INFO:tasks.workunit.client.0.smithi090.stdout:8/702: dread d2/da/d13/d64/d24/f79 [0,4194304] 0 2024-04-24T08:25:10.023 INFO:tasks.workunit.client.0.smithi090.stdout:5/787: link d8/d59/dc7/d7a/da1/dae/fce d8/da/d14/d2f/d81/da6/f103 0 2024-04-24T08:25:10.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/742: mknod d8/d1d/d1f/ce1 0 2024-04-24T08:25:10.025 INFO:tasks.workunit.client.0.smithi090.stdout:7/788: read db/d17/d39/f3c [1077506,86044] 0 2024-04-24T08:25:10.026 INFO:tasks.workunit.client.0.smithi090.stdout:4/668: symlink d3/d18/d5c/db5/d69/ld8 0 2024-04-24T08:25:10.027 INFO:tasks.workunit.client.0.smithi090.stdout:1/719: dwrite d3/d1d/fc9 [0,4194304] 0 2024-04-24T08:25:10.029 INFO:tasks.workunit.client.0.smithi090.stdout:0/707: rename d1a/f30 to d1a/d4b/d53/d79/d35/d8e/fec 0 2024-04-24T08:25:10.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/793: dread d0/d7/f32 [0,4194304] 0 2024-04-24T08:25:10.029 INFO:tasks.workunit.client.0.smithi090.stdout:8/703: unlink d2/d62/d83/da3/fa5 0 2024-04-24T08:25:10.031 INFO:tasks.workunit.client.0.smithi090.stdout:4/669: unlink d3/d18/d5c/c75 0 2024-04-24T08:25:10.031 INFO:tasks.workunit.client.0.smithi090.stdout:8/704: chown d2/d62/d6b/c85 1683 1 2024-04-24T08:25:10.034 INFO:tasks.workunit.client.0.smithi090.stdout:1/720: fdatasync d3/d5/da/dc/f2b 0 2024-04-24T08:25:10.034 INFO:tasks.workunit.client.0.smithi090.stdout:6/743: creat d8/d1d/d1f/d3e/d44/fe2 x:0 0 0 2024-04-24T08:25:10.039 INFO:tasks.workunit.client.0.smithi090.stdout:6/744: creat d8/d1d/d29/fe3 x:0 0 0 2024-04-24T08:25:10.040 INFO:tasks.workunit.client.0.smithi090.stdout:8/705: rename d2/da/d13/d34/f72 to d2/ff1 0 2024-04-24T08:25:10.042 INFO:tasks.workunit.client.0.smithi090.stdout:6/745: chown d8/dc/d19/l22 1810181 1 2024-04-24T08:25:10.045 INFO:tasks.workunit.client.0.smithi090.stdout:8/706: fdatasync d2/da/d13/d34/f6f 0 2024-04-24T08:25:10.047 INFO:tasks.workunit.client.0.smithi090.stdout:8/707: truncate d2/f54 5169030 0 2024-04-24T08:25:10.047 INFO:tasks.workunit.client.0.smithi090.stdout:6/746: mkdir d8/d1d/d1f/d3e/d4f/d7e/daa/de4 0 2024-04-24T08:25:10.049 INFO:tasks.workunit.client.0.smithi090.stdout:8/708: rmdir d2/d62/d83 39 2024-04-24T08:25:10.051 INFO:tasks.workunit.client.0.smithi090.stdout:8/709: dread - d2/da/d13/d64/db2/dbb/fc0 zero size 2024-04-24T08:25:10.063 INFO:tasks.workunit.client.0.smithi090.stdout:6/747: dwrite d8/dc/d19/d8e/d81/d96/fbd [0,4194304] 0 2024-04-24T08:25:10.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/748: rename d8/dc/d19/d8e/d38/ce0 to d8/d1d/d1f/d3e/ce5 0 2024-04-24T08:25:10.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/749: write d8/d1d/d1f/d3e/f6a [265929,71332] 0 2024-04-24T08:25:10.075 INFO:tasks.workunit.client.0.smithi090.stdout:2/794: dread d0/d7/d28/daa/db1/fc5 [0,4194304] 0 2024-04-24T08:25:10.080 INFO:tasks.workunit.client.0.smithi090.stdout:6/750: readlink d8/dc/d19/l2d 0 2024-04-24T08:25:10.081 INFO:tasks.workunit.client.0.smithi090.stdout:6/751: chown d8/d1d/d1f/f2e 3793 1 2024-04-24T08:25:10.082 INFO:tasks.workunit.client.0.smithi090.stdout:2/795: rename d0/d7/de1/fa1 to d0/d7/d28/daa/def/d50/f107 0 2024-04-24T08:25:10.084 INFO:tasks.workunit.client.0.smithi090.stdout:6/752: fdatasync d8/d1d/d2f/d85/fc5 0 2024-04-24T08:25:10.086 INFO:tasks.workunit.client.0.smithi090.stdout:0/708: dread d1a/d4b/d53/d57/fa4 [0,4194304] 0 2024-04-24T08:25:10.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/753: creat d8/dc/d19/d8e/d81/d96/fe6 x:0 0 0 2024-04-24T08:25:10.092 INFO:tasks.workunit.client.0.smithi090.stdout:6/754: stat d8/d1d/d1f/d3e/d8f/fcc 0 2024-04-24T08:25:10.092 INFO:tasks.workunit.client.0.smithi090.stdout:0/709: creat d1a/d4b/d53/d79/d58/db4/fed x:0 0 0 2024-04-24T08:25:10.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/710: fsync d1a/d4b/d53/d79/d35/d62/fab 0 2024-04-24T08:25:10.099 INFO:tasks.workunit.client.0.smithi090.stdout:6/755: unlink d8/d1d/d29/l45 0 2024-04-24T08:25:10.100 INFO:tasks.workunit.client.0.smithi090.stdout:3/678: dwrite d2/fa [0,4194304] 0 2024-04-24T08:25:10.105 INFO:tasks.workunit.client.1.smithi149.stdout:4/935: dwrite d2/d24/d2e/d3f/ff8 [0,4194304] 0 2024-04-24T08:25:10.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/756: truncate d8/d1d/d1f/d3e/d44/d90/fc7 4657586 0 2024-04-24T08:25:10.108 INFO:tasks.workunit.client.0.smithi090.stdout:3/679: mknod d2/d15/d36/d6f/d7a/ce2 0 2024-04-24T08:25:10.110 INFO:tasks.workunit.client.0.smithi090.stdout:0/711: link d1a/l8f d1a/d4b/d53/d79/d69/d78/lee 0 2024-04-24T08:25:10.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/788: dread d8/f9e [0,4194304] 0 2024-04-24T08:25:10.115 INFO:tasks.workunit.client.0.smithi090.stdout:9/712: dwrite d0/dc/d24/d2d/d49/d81/d87/f8a [0,4194304] 0 2024-04-24T08:25:10.117 INFO:tasks.workunit.client.0.smithi090.stdout:9/713: fdatasync d0/dc/f1f 0 2024-04-24T08:25:10.119 INFO:tasks.workunit.client.0.smithi090.stdout:5/789: truncate d8/d59/dc7/d7a/da1/dae/fce 39431 0 2024-04-24T08:25:10.120 INFO:tasks.workunit.client.0.smithi090.stdout:9/714: chown d0/d2a/fb8 117630050 1 2024-04-24T08:25:10.120 INFO:tasks.workunit.client.0.smithi090.stdout:1/721: dread d3/d5/da/dc/fab [0,4194304] 0 2024-04-24T08:25:10.121 INFO:tasks.workunit.client.0.smithi090.stdout:3/680: mkdir d2/df/d1a/d37/de3 0 2024-04-24T08:25:10.123 INFO:tasks.workunit.client.0.smithi090.stdout:5/790: stat d8/d59/dc7/d7a/dbc 0 2024-04-24T08:25:10.125 INFO:tasks.workunit.client.0.smithi090.stdout:0/712: fsync d1a/f1b 0 2024-04-24T08:25:10.126 INFO:tasks.workunit.client.0.smithi090.stdout:9/715: stat d0/dc/d24/d2d/d49/d62/c8d 0 2024-04-24T08:25:10.131 INFO:tasks.workunit.client.0.smithi090.stdout:5/791: write d8/da/d14/d2f/d81/fb8 [3126324,14652] 0 2024-04-24T08:25:10.133 INFO:tasks.workunit.client.0.smithi090.stdout:4/670: sync 2024-04-24T08:25:10.134 INFO:tasks.workunit.client.0.smithi090.stdout:3/681: read d2/df/d1a/f6d [170770,60014] 0 2024-04-24T08:25:10.135 INFO:tasks.workunit.client.0.smithi090.stdout:7/789: dwrite db/d17/d29/d38/d99/fad [0,4194304] 0 2024-04-24T08:25:10.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/710: truncate d2/da/f35 3029998 0 2024-04-24T08:25:10.138 INFO:tasks.workunit.client.0.smithi090.stdout:7/790: stat db/d17/d29/d38/cb2 0 2024-04-24T08:25:10.193 INFO:tasks.workunit.client.1.smithi149.stdout:9/999: sync 2024-04-24T08:25:10.223 INFO:tasks.workunit.client.0.smithi090.stdout:2/796: write d0/f55 [1437486,26071] 0 2024-04-24T08:25:10.226 INFO:tasks.workunit.client.0.smithi090.stdout:8/711: stat d2/d62/d83/da3/deb/fec 0 2024-04-24T08:25:10.226 INFO:tasks.workunit.client.0.smithi090.stdout:1/722: link d3/d5/d3e/da3/fb2 d3/d5/da/dc/d80/fff 0 2024-04-24T08:25:10.227 INFO:tasks.workunit.client.0.smithi090.stdout:2/797: dread - d0/d14/d36/ff7 zero size 2024-04-24T08:25:10.227 INFO:tasks.workunit.client.0.smithi090.stdout:7/791: truncate db/d17/d39/fa2 597163 0 2024-04-24T08:25:10.230 INFO:tasks.workunit.client.0.smithi090.stdout:4/671: mkdir d3/dd9 0 2024-04-24T08:25:10.230 INFO:tasks.workunit.client.0.smithi090.stdout:7/792: dread - db/d17/d29/d38/d3e/ff1 zero size 2024-04-24T08:25:10.231 INFO:tasks.workunit.client.0.smithi090.stdout:3/682: link d2/df/d1a/f2b d2/d90/fe4 0 2024-04-24T08:25:10.232 INFO:tasks.workunit.client.0.smithi090.stdout:4/672: write d3/d8e/fd5 [1038818,35825] 0 2024-04-24T08:25:10.232 INFO:tasks.workunit.client.0.smithi090.stdout:0/713: rename d1a/f71 to d1a/fef 0 2024-04-24T08:25:10.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/712: mkdir d2/d62/d6b/d9c/dd8/df2 0 2024-04-24T08:25:10.237 INFO:tasks.workunit.client.0.smithi090.stdout:2/798: creat d0/d7/d67/d7f/d31/d90/f108 x:0 0 0 2024-04-24T08:25:10.238 INFO:tasks.workunit.client.1.smithi149.stdout:4/936: truncate d2/d18/d1e/d3c/d48/d69/fb6 1113087 0 2024-04-24T08:25:10.241 INFO:tasks.workunit.client.0.smithi090.stdout:4/673: unlink d3/d18/d1d/fab 0 2024-04-24T08:25:10.241 INFO:tasks.workunit.client.0.smithi090.stdout:0/714: dread d1a/d4b/d53/d79/d3c/fad [0,4194304] 0 2024-04-24T08:25:10.244 INFO:tasks.workunit.client.0.smithi090.stdout:7/793: rename db/d17/d39/f3c to db/dbd/f110 0 2024-04-24T08:25:10.247 INFO:tasks.workunit.client.0.smithi090.stdout:6/757: dwrite d8/d1d/f6f [0,4194304] 0 2024-04-24T08:25:10.248 INFO:tasks.workunit.client.0.smithi090.stdout:7/794: dread - db/d17/d29/d32/d5a/d75/f68 zero size 2024-04-24T08:25:10.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/792: write d8/f9 [755071,30369] 0 2024-04-24T08:25:10.253 INFO:tasks.workunit.client.1.smithi149.stdout:4/937: dread d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/fb3 [0,4194304] 0 2024-04-24T08:25:10.254 INFO:tasks.workunit.client.0.smithi090.stdout:2/799: creat d0/d7/d28/daa/ddc/f109 x:0 0 0 2024-04-24T08:25:10.256 INFO:tasks.workunit.client.0.smithi090.stdout:5/793: chown d8/da/d14/fd3 28 1 2024-04-24T08:25:10.260 INFO:tasks.workunit.client.0.smithi090.stdout:4/674: creat d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/fda x:0 0 0 2024-04-24T08:25:10.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/794: dread d8/da/d3a/fbd [0,4194304] 0 2024-04-24T08:25:10.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/715: rename d1a/d4b/d53/d79/d3c/lc9 to d1a/d4b/d53/d79/d69/d8a/lf0 0 2024-04-24T08:25:10.267 INFO:tasks.workunit.client.0.smithi090.stdout:7/795: mkdir db/d17/d29/d32/d5a/d75/d101/d111 0 2024-04-24T08:25:10.268 INFO:tasks.workunit.client.0.smithi090.stdout:0/716: dread - d1a/d4b/d53/d79/d69/d8a/fd5 zero size 2024-04-24T08:25:10.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/716: dwrite d0/dc/d24/fd2 [0,4194304] 0 2024-04-24T08:25:10.274 INFO:tasks.workunit.client.0.smithi090.stdout:3/683: dread d2/f24 [0,4194304] 0 2024-04-24T08:25:10.276 INFO:tasks.workunit.client.0.smithi090.stdout:3/684: chown d2/df/d1a/c1b 1 1 2024-04-24T08:25:10.280 INFO:tasks.workunit.client.0.smithi090.stdout:4/675: rename d3/d25 to d3/d18/d5c/db5/d69/d8b/db2/d49/ddb 0 2024-04-24T08:25:10.284 INFO:tasks.workunit.client.0.smithi090.stdout:1/723: link d3/d68/d7f/f9e d3/d1d/d34/d73/dd8/f100 0 2024-04-24T08:25:10.285 INFO:tasks.workunit.client.0.smithi090.stdout:9/717: creat d0/dc/d24/d2d/fde x:0 0 0 2024-04-24T08:25:10.288 INFO:tasks.workunit.client.0.smithi090.stdout:9/718: readlink d0/dc/d24/d7c/d4d/d32/d9f/d78/l99 0 2024-04-24T08:25:10.290 INFO:tasks.workunit.client.1.smithi149.stdout:4/938: dread d2/d18/ff4 [0,4194304] 0 2024-04-24T08:25:10.290 INFO:tasks.workunit.client.0.smithi090.stdout:6/758: link d8/d1d/d1f/d3e/d4f/c66 d8/d1d/ddb/ce7 0 2024-04-24T08:25:10.292 INFO:tasks.workunit.client.0.smithi090.stdout:0/717: dwrite d1a/d4b/d53/d79/d69/d8a/fd6 [0,4194304] 0 2024-04-24T08:25:10.293 INFO:tasks.workunit.client.1.smithi149.stdout:4/939: stat d2/cc1 0 2024-04-24T08:25:10.295 INFO:tasks.workunit.client.0.smithi090.stdout:1/724: mkdir d3/d68/d7f/d101 0 2024-04-24T08:25:10.296 INFO:tasks.workunit.client.0.smithi090.stdout:5/795: write d8/da/d3a/d2d/f82 [224800,121029] 0 2024-04-24T08:25:10.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/719: symlink d0/dc/d24/d7c/d4d/d32/d9f/d78/ldf 0 2024-04-24T08:25:10.301 INFO:tasks.workunit.client.1.smithi149.stdout:4/940: creat d2/d18/d1e/d3c/d48/d69/f12a x:0 0 0 2024-04-24T08:25:10.303 INFO:tasks.workunit.client.0.smithi090.stdout:2/800: creat d0/d2e/f10a x:0 0 0 2024-04-24T08:25:10.307 INFO:tasks.workunit.client.0.smithi090.stdout:1/725: fsync d3/d25/f6d 0 2024-04-24T08:25:10.307 INFO:tasks.workunit.client.0.smithi090.stdout:4/676: dwrite d3/d18/d5c/db5/d69/d8b/db2/d49/d6f/f51 [4194304,4194304] 0 2024-04-24T08:25:10.307 INFO:tasks.workunit.client.0.smithi090.stdout:5/796: truncate d8/d59/dc7/d7a/d50/f8a 459944 0 2024-04-24T08:25:10.313 INFO:tasks.workunit.client.0.smithi090.stdout:1/726: rmdir d3/d5/d58 39 2024-04-24T08:25:10.315 INFO:tasks.workunit.client.0.smithi090.stdout:3/685: dwrite d2/df/d1a/d37/d59/f7b [0,4194304] 0 2024-04-24T08:25:10.316 INFO:tasks.workunit.client.0.smithi090.stdout:6/759: link d8/d1d/d1f/d3e/d7a/f9a d8/fe8 0 2024-04-24T08:25:10.321 INFO:tasks.workunit.client.1.smithi149.stdout:4/941: write d2/d18/db9/dd8/fcf [798930,27998] 0 2024-04-24T08:25:10.327 INFO:tasks.workunit.client.1.smithi149.stdout:6/851: sync 2024-04-24T08:25:10.327 INFO:tasks.workunit.client.1.smithi149.stdout:2/847: sync 2024-04-24T08:25:10.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/914: sync 2024-04-24T08:25:10.328 INFO:tasks.workunit.client.1.smithi149.stdout:0/922: sync 2024-04-24T08:25:10.332 INFO:tasks.workunit.client.1.smithi149.stdout:8/915: creat d1/de/d18/d32/d65/dad/f13c x:0 0 0 2024-04-24T08:25:10.333 INFO:tasks.workunit.client.1.smithi149.stdout:6/852: creat d0/d1a/d28/d31/d4c/d4e/f120 x:0 0 0 2024-04-24T08:25:10.334 INFO:tasks.workunit.client.1.smithi149.stdout:2/848: mknod d0/d6/db/d39/d78/d8a/dc5/c101 0 2024-04-24T08:25:10.335 INFO:tasks.workunit.client.1.smithi149.stdout:2/849: fdatasync d0/f2d 0 2024-04-24T08:25:10.338 INFO:tasks.workunit.client.1.smithi149.stdout:0/923: getdents d2/d9/df/d2f/d6c/d72/db0 0 2024-04-24T08:25:10.343 INFO:tasks.workunit.client.1.smithi149.stdout:0/924: creat d2/d3c/d57/f128 x:0 0 0 2024-04-24T08:25:10.344 INFO:tasks.workunit.client.0.smithi090.stdout:8/713: sync 2024-04-24T08:25:10.347 INFO:tasks.workunit.client.1.smithi149.stdout:0/925: stat d2/d9/df/d2f/d6c/fbb 0 2024-04-24T08:25:10.349 INFO:tasks.workunit.client.0.smithi090.stdout:1/727: fsync d3/d5/da/d97/d9c/fa9 0 2024-04-24T08:25:10.350 INFO:tasks.workunit.client.0.smithi090.stdout:9/720: symlink d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/le0 0 2024-04-24T08:25:10.351 INFO:tasks.workunit.client.0.smithi090.stdout:2/801: mknod d0/d7/d28/daa/def/d50/d96/c10b 0 2024-04-24T08:25:10.352 INFO:tasks.workunit.client.1.smithi149.stdout:0/926: unlink d2/d9/d1a/c97 0 2024-04-24T08:25:10.352 INFO:tasks.workunit.client.1.smithi149.stdout:6/853: dread d0/f3 [0,4194304] 0 2024-04-24T08:25:10.353 INFO:tasks.workunit.client.0.smithi090.stdout:2/802: readlink d0/d7/l7e 0 2024-04-24T08:25:10.354 INFO:tasks.workunit.client.0.smithi090.stdout:6/760: dread d8/d1d/fcf [0,4194304] 0 2024-04-24T08:25:10.356 INFO:tasks.workunit.client.0.smithi090.stdout:0/718: dwrite d1a/f2c [0,4194304] 0 2024-04-24T08:25:10.358 INFO:tasks.workunit.client.1.smithi149.stdout:8/916: dwrite d1/de/fc8 [0,4194304] 0 2024-04-24T08:25:10.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/850: dwrite d0/d6/db/f72 [4194304,4194304] 0 2024-04-24T08:25:10.364 INFO:tasks.workunit.client.1.smithi149.stdout:4/942: dread d2/d10/f81 [0,4194304] 0 2024-04-24T08:25:10.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/927: mkdir d2/d9/df/d129 0 2024-04-24T08:25:10.372 INFO:tasks.workunit.client.1.smithi149.stdout:4/943: write d2/d10/d91/f9d [4568296,32097] 0 2024-04-24T08:25:10.377 INFO:tasks.workunit.client.1.smithi149.stdout:8/917: fdatasync d1/ffb 0 2024-04-24T08:25:10.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/928: unlink d2/d9/df/d2f/d6c/d72/d91/ld2 0 2024-04-24T08:25:10.394 INFO:tasks.workunit.client.1.smithi149.stdout:0/929: rename d2/d9/df/d48/dc4 to d2/d9/df/d2f/d12a 0 2024-04-24T08:25:10.396 INFO:tasks.workunit.client.1.smithi149.stdout:4/944: mknod d2/d10/d91/df7/d116/c12b 0 2024-04-24T08:25:10.403 INFO:tasks.workunit.client.1.smithi149.stdout:8/918: fsync d1/d1f/d31/fbb 0 2024-04-24T08:25:10.408 INFO:tasks.workunit.client.0.smithi090.stdout:7/796: sync 2024-04-24T08:25:10.411 INFO:tasks.workunit.client.1.smithi149.stdout:2/851: dwrite d0/d6/db/d27/d64/f6e [0,4194304] 0 2024-04-24T08:25:10.413 INFO:tasks.workunit.client.1.smithi149.stdout:0/930: link d2/d9/d1a/d36/d85/l9f d2/d9/df/d2f/d12a/l12b 0 2024-04-24T08:25:10.415 INFO:tasks.workunit.client.1.smithi149.stdout:6/854: dwrite d0/d1a/d28/d31/d33/fa8 [0,4194304] 0 2024-04-24T08:25:10.416 INFO:tasks.workunit.client.0.smithi090.stdout:3/686: stat d2/df/d57/d4f/lcd 0 2024-04-24T08:25:10.421 INFO:tasks.workunit.client.0.smithi090.stdout:5/797: mknod d8/d59/dc7/c104 0 2024-04-24T08:25:10.422 INFO:tasks.workunit.client.1.smithi149.stdout:8/919: unlink d1/d1f/d31/cdf 0 2024-04-24T08:25:10.427 INFO:tasks.workunit.client.1.smithi149.stdout:2/852: chown d0/d6/db/d39/d3d/d44/le5 2511 1 2024-04-24T08:25:10.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/853: chown d0/d6/db/d27/d81/d9f/ld1 1775618210 1 2024-04-24T08:25:10.430 INFO:tasks.workunit.client.0.smithi090.stdout:2/803: mknod d0/d7/d67/d7f/d31/d90/c10c 0 2024-04-24T08:25:10.433 INFO:tasks.workunit.client.0.smithi090.stdout:6/761: creat d8/d1d/d29/fe9 x:0 0 0 2024-04-24T08:25:10.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/728: truncate d3/d5/da/d97/d9c/dc4/fd2 262092 0 2024-04-24T08:25:10.434 INFO:tasks.workunit.client.1.smithi149.stdout:8/920: dread - d1/d1f/d5e/d92/fee zero size 2024-04-24T08:25:10.435 INFO:tasks.workunit.client.1.smithi149.stdout:4/945: write d2/d10/d1a/f3e [4332946,64468] 0 2024-04-24T08:25:10.437 INFO:tasks.workunit.client.0.smithi090.stdout:9/721: rename d0/dc/d24/d2d/d49/d62/dcb/dcd/ddc to d0/dc/d24/d2d/dda/de1 0 2024-04-24T08:25:10.438 INFO:tasks.workunit.client.1.smithi149.stdout:8/921: chown d1/de/d18/d32/d65/d8b/d7d/c112 7855515 1 2024-04-24T08:25:10.439 INFO:tasks.workunit.client.0.smithi090.stdout:9/722: chown d0/c7e 6 1 2024-04-24T08:25:10.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/729: truncate d3/d5/fa5 621217 0 2024-04-24T08:25:10.440 INFO:tasks.workunit.client.0.smithi090.stdout:0/719: creat d1a/d4b/d53/d79/d69/d90/dbc/ff1 x:0 0 0 2024-04-24T08:25:10.444 INFO:tasks.workunit.client.0.smithi090.stdout:4/677: sync 2024-04-24T08:25:10.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/931: write d2/d17/fd1 [2366923,51353] 0 2024-04-24T08:25:10.449 INFO:tasks.workunit.client.1.smithi149.stdout:0/932: fdatasync d2/d9/d1a/d36/d85/ffa 0 2024-04-24T08:25:10.451 INFO:tasks.workunit.client.0.smithi090.stdout:6/762: unlink d8/dc/d19/l95 0 2024-04-24T08:25:10.456 INFO:tasks.workunit.client.0.smithi090.stdout:1/730: dread - d3/d5/da/d94/f92 zero size 2024-04-24T08:25:10.456 INFO:tasks.workunit.client.1.smithi149.stdout:6/855: truncate d0/d1a/d28/d31/d33/fa8 991490 0 2024-04-24T08:25:10.459 INFO:tasks.workunit.client.0.smithi090.stdout:7/797: write db/d17/d29/d32/f57 [2132176,46822] 0 2024-04-24T08:25:10.461 INFO:tasks.workunit.client.0.smithi090.stdout:0/720: fdatasync d1a/d4b/d53/d79/f32 0 2024-04-24T08:25:10.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/678: rmdir d3/d24/d92 39 2024-04-24T08:25:10.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/798: link d8/da/d3a/d96/dcb/f22 d8/da/d3a/d96/dcb/de3/f105 0 2024-04-24T08:25:10.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/804: dread d0/d7/d67/d7f/d31/d46/d60/f74 [0,4194304] 0 2024-04-24T08:25:10.471 INFO:tasks.workunit.client.0.smithi090.stdout:3/687: rename d2/f9 to d2/df/d1a/d37/d4b/fe5 0 2024-04-24T08:25:10.474 INFO:tasks.workunit.client.0.smithi090.stdout:3/688: write d2/df/d1a/d37/d53/f65 [3013658,31038] 0 2024-04-24T08:25:10.477 INFO:tasks.workunit.client.0.smithi090.stdout:0/721: mkdir d1a/d4b/d53/d79/d69/d90/df2 0 2024-04-24T08:25:10.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/798: truncate db/d17/d29/d32/f3d 5364787 0 2024-04-24T08:25:10.481 INFO:tasks.workunit.client.0.smithi090.stdout:6/763: mkdir d8/dc/d19/dea 0 2024-04-24T08:25:10.491 INFO:tasks.workunit.client.0.smithi090.stdout:0/722: symlink d1a/d4b/d53/d79/d69/d78/d9b/lf3 0 2024-04-24T08:25:10.494 INFO:tasks.workunit.client.0.smithi090.stdout:5/799: dread d8/d59/dc7/d7a/da1/dae/fc9 [0,4194304] 0 2024-04-24T08:25:10.507 INFO:tasks.workunit.client.0.smithi090.stdout:8/714: rename d2/da/d13/d64/d24/db3/lbe to d2/da/d13/d64/d24/db3/db4/lf3 0 2024-04-24T08:25:10.509 INFO:tasks.workunit.client.0.smithi090.stdout:7/799: write db/d17/d29/d38/d3e/fb8 [360092,35001] 0 2024-04-24T08:25:10.510 INFO:tasks.workunit.client.0.smithi090.stdout:0/723: fdatasync d1a/d4b/fd8 0 2024-04-24T08:25:10.510 INFO:tasks.workunit.client.0.smithi090.stdout:8/715: chown d2/da/c1b 2074457 1 2024-04-24T08:25:10.513 INFO:tasks.workunit.client.0.smithi090.stdout:5/800: dread d8/f10 [0,4194304] 0 2024-04-24T08:25:10.513 INFO:tasks.workunit.client.0.smithi090.stdout:6/764: creat d8/feb x:0 0 0 2024-04-24T08:25:10.515 INFO:tasks.workunit.client.1.smithi149.stdout:2/854: rename d0/d6/db/d39/d3d/d58 to d0/d6/db/d27/d64/d66/db1/dc9/d102 0 2024-04-24T08:25:10.515 INFO:tasks.workunit.client.0.smithi090.stdout:2/805: dwrite d0/d7/de1/f76 [0,4194304] 0 2024-04-24T08:25:10.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/765: chown d8/d1d/d1f/d3e/d44/d90/da2/fa4 82610974 1 2024-04-24T08:25:10.518 INFO:tasks.workunit.client.0.smithi090.stdout:2/806: write d0/d7/d67/d7f/fe6 [1042745,42764] 0 2024-04-24T08:25:10.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/723: rename d0/da5 to d0/dc/d24/d7c/d4d/d8c/de2 0 2024-04-24T08:25:10.525 INFO:tasks.workunit.client.1.smithi149.stdout:6/856: dread - d0/d1a/d28/d31/d4c/d80/d6f/fcb zero size 2024-04-24T08:25:10.529 INFO:tasks.workunit.client.0.smithi090.stdout:1/731: rename d3/d5/da/d97/ded to d3/d25/d30/dc5/d102 0 2024-04-24T08:25:10.533 INFO:tasks.workunit.client.1.smithi149.stdout:0/933: rename d2/d9/df/d2f/d6c/dc1/d4e/d6f/d9e to d2/d9/df/d48/dba/d12c 0 2024-04-24T08:25:10.535 INFO:tasks.workunit.client.0.smithi090.stdout:9/724: fsync d0/dc/f64 0 2024-04-24T08:25:10.536 INFO:tasks.workunit.client.1.smithi149.stdout:2/855: symlink d0/d6/db/d27/l103 0 2024-04-24T08:25:10.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/724: dwrite d1a/d4b/d53/d79/d58/db4/fed [0,4194304] 0 2024-04-24T08:25:10.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/766: rename d8/dc/d19/d8e/d81/d96/d97/fae to d8/dc/d19/d8e/d81/d96/d97/fec 0 2024-04-24T08:25:10.539 INFO:tasks.workunit.client.0.smithi090.stdout:8/716: dwrite d2/da/d13/d34/fa0 [4194304,4194304] 0 2024-04-24T08:25:10.539 INFO:tasks.workunit.client.0.smithi090.stdout:9/725: read - d0/dc/f64 zero size 2024-04-24T08:25:10.540 INFO:tasks.workunit.client.1.smithi149.stdout:8/922: creat d1/d1f/d22/d134/f13d x:0 0 0 2024-04-24T08:25:10.547 INFO:tasks.workunit.client.1.smithi149.stdout:4/946: dwrite d2/d10/fa6 [0,4194304] 0 2024-04-24T08:25:10.549 INFO:tasks.workunit.client.1.smithi149.stdout:0/934: creat d2/d9/df/d48/dba/d12c/f12d x:0 0 0 2024-04-24T08:25:10.552 INFO:tasks.workunit.client.0.smithi090.stdout:1/732: dread - d3/d25/d30/d9d/fa8 zero size 2024-04-24T08:25:10.554 INFO:tasks.workunit.client.1.smithi149.stdout:2/856: mknod d0/d6/db/d39/d78/d7c/df1/c104 0 2024-04-24T08:25:10.563 INFO:tasks.workunit.client.0.smithi090.stdout:5/801: dwrite d8/da/d3a/d2d/d37/f63 [0,4194304] 0 2024-04-24T08:25:10.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/800: dwrite db/d54/d83/f8b [0,4194304] 0 2024-04-24T08:25:10.565 INFO:tasks.workunit.client.1.smithi149.stdout:4/947: mkdir d2/d10/d91/df7/d116/d12c 0 2024-04-24T08:25:10.576 INFO:tasks.workunit.client.1.smithi149.stdout:4/948: mkdir d2/d10/d91/d12d 0 2024-04-24T08:25:10.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/801: mkdir db/dbd/d112 0 2024-04-24T08:25:10.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/725: mknod d1a/d4b/d53/d79/d69/d90/df2/cf4 0 2024-04-24T08:25:10.579 INFO:tasks.workunit.client.0.smithi090.stdout:5/802: mkdir d8/d59/dc7/d7b/d106 0 2024-04-24T08:25:10.581 INFO:tasks.workunit.client.1.smithi149.stdout:8/923: getdents d1/dce 0 2024-04-24T08:25:10.584 INFO:tasks.workunit.client.0.smithi090.stdout:7/802: symlink db/d17/d29/d32/dd4/l113 0 2024-04-24T08:25:10.585 INFO:tasks.workunit.client.0.smithi090.stdout:8/717: link d2/d62/d6b/ca4 d2/da/d13/d64/db2/db6/cf4 0 2024-04-24T08:25:10.587 INFO:tasks.workunit.client.0.smithi090.stdout:8/718: write d2/da/f38 [1246308,98022] 0 2024-04-24T08:25:10.587 INFO:tasks.workunit.client.1.smithi149.stdout:8/924: mkdir d1/dce/d13e 0 2024-04-24T08:25:10.588 INFO:tasks.workunit.client.0.smithi090.stdout:8/719: fdatasync d2/f10 0 2024-04-24T08:25:10.588 INFO:tasks.workunit.client.1.smithi149.stdout:6/857: dwrite d0/d1a/d28/d31/d4c/d80/fd0 [0,4194304] 0 2024-04-24T08:25:10.589 INFO:tasks.workunit.client.0.smithi090.stdout:8/720: stat d2/da/d13/d64/d50/fb7 0 2024-04-24T08:25:10.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/721: chown d2/da/d13/d64/d50 25688018 1 2024-04-24T08:25:10.591 INFO:tasks.workunit.client.0.smithi090.stdout:9/726: link d0/dc/d24/d7c/d4d/l70 d0/d19/le3 0 2024-04-24T08:25:10.591 INFO:tasks.workunit.client.0.smithi090.stdout:5/803: chown d8/da/d3a/d96/dcb/d21/d4c/leb 6 1 2024-04-24T08:25:10.593 INFO:tasks.workunit.client.1.smithi149.stdout:8/925: dread - d1/d1f/d31/fc0 zero size 2024-04-24T08:25:10.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/722: truncate d2/da/d13/d64/db2/fd9 5206257 0 2024-04-24T08:25:10.596 INFO:tasks.workunit.client.0.smithi090.stdout:4/679: rename d3/c19 to d3/d18/d5c/cdc 0 2024-04-24T08:25:10.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/803: creat db/de9/f114 x:0 0 0 2024-04-24T08:25:10.599 INFO:tasks.workunit.client.0.smithi090.stdout:7/804: chown db/d17/d39/fbb 66885 1 2024-04-24T08:25:10.600 INFO:tasks.workunit.client.0.smithi090.stdout:7/805: readlink db/d17/l93 0 2024-04-24T08:25:10.600 INFO:tasks.workunit.client.1.smithi149.stdout:8/926: chown d1/de/d18/d32/d65/d8b/dc9/l138 111519186 1 2024-04-24T08:25:10.602 INFO:tasks.workunit.client.1.smithi149.stdout:6/858: mknod d0/d1a/d28/d31/d4c/df8/c121 0 2024-04-24T08:25:10.603 INFO:tasks.workunit.client.0.smithi090.stdout:1/733: dread d3/d25/f6d [0,4194304] 0 2024-04-24T08:25:10.605 INFO:tasks.workunit.client.1.smithi149.stdout:8/927: chown d1/de/d18/d32/d63/d76/d12f/f132 23175 1 2024-04-24T08:25:10.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/949: dwrite d2/d24/d2e/d3f/f120 [0,4194304] 0 2024-04-24T08:25:10.608 INFO:tasks.workunit.client.1.smithi149.stdout:0/935: write d2/d9/df/d2f/f8f [330227,20385] 0 2024-04-24T08:25:10.623 INFO:tasks.workunit.client.0.smithi090.stdout:3/689: rename d2/df/db5/dd7 to d2/df/d1a/d37/dc0/de6 0 2024-04-24T08:25:10.624 INFO:tasks.workunit.client.0.smithi090.stdout:3/690: fdatasync d2/d90/fa1 0 2024-04-24T08:25:10.625 INFO:tasks.workunit.client.0.smithi090.stdout:4/680: write d3/d18/d1d/f27 [2565208,93453] 0 2024-04-24T08:25:10.626 INFO:tasks.workunit.client.1.smithi149.stdout:0/936: truncate d2/d9/fd8 3914499 0 2024-04-24T08:25:10.627 INFO:tasks.workunit.client.1.smithi149.stdout:8/928: mknod d1/d1f/d5e/c13f 0 2024-04-24T08:25:10.628 INFO:tasks.workunit.client.1.smithi149.stdout:2/857: dread d0/d6/db/d28/f2c [0,4194304] 0 2024-04-24T08:25:10.629 INFO:tasks.workunit.client.0.smithi090.stdout:0/726: write d1a/d4b/d53/d79/f32 [400790,21292] 0 2024-04-24T08:25:10.629 INFO:tasks.workunit.client.0.smithi090.stdout:1/734: truncate d3/d68/d7f/fb1 235327 0 2024-04-24T08:25:10.633 INFO:tasks.workunit.client.1.smithi149.stdout:0/937: creat d2/d17/d41/d68/ddc/f12e x:0 0 0 2024-04-24T08:25:10.634 INFO:tasks.workunit.client.0.smithi090.stdout:9/727: rmdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96 39 2024-04-24T08:25:10.637 INFO:tasks.workunit.client.0.smithi090.stdout:9/728: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f6f [1284008,90234] 0 2024-04-24T08:25:10.639 INFO:tasks.workunit.client.0.smithi090.stdout:2/807: rename d0/d7/d67/d7f/d31/cb5 to d0/d7/d6f/c10d 0 2024-04-24T08:25:10.641 INFO:tasks.workunit.client.1.smithi149.stdout:2/858: dread d0/d6/fa0 [0,4194304] 0 2024-04-24T08:25:10.641 INFO:tasks.workunit.client.1.smithi149.stdout:0/938: unlink d2/d17/d41/f7d 0 2024-04-24T08:25:10.646 INFO:tasks.workunit.client.1.smithi149.stdout:6/859: dwrite d0/f4 [0,4194304] 0 2024-04-24T08:25:10.647 INFO:tasks.workunit.client.0.smithi090.stdout:4/681: creat d3/d8e/fdd x:0 0 0 2024-04-24T08:25:10.647 INFO:tasks.workunit.client.0.smithi090.stdout:7/806: write db/d54/db1/fb4 [76198,106676] 0 2024-04-24T08:25:10.647 INFO:tasks.workunit.client.0.smithi090.stdout:8/723: dwrite d2/da/d13/d60/fb1 [0,4194304] 0 2024-04-24T08:25:10.649 INFO:tasks.workunit.client.1.smithi149.stdout:4/950: truncate d2/f8a 261757 0 2024-04-24T08:25:10.655 INFO:tasks.workunit.client.1.smithi149.stdout:2/859: read d0/d6/db/d39/d3d/d44/f53 [150565,96049] 0 2024-04-24T08:25:10.656 INFO:tasks.workunit.client.1.smithi149.stdout:8/929: symlink d1/de/d18/d32/d65/de5/l140 0 2024-04-24T08:25:10.665 INFO:tasks.workunit.client.1.smithi149.stdout:6/860: fdatasync d0/d1a/d28/d31/f4b 0 2024-04-24T08:25:10.667 INFO:tasks.workunit.client.1.smithi149.stdout:4/951: mkdir d2/d18/d1e/d3c/d59/d12e 0 2024-04-24T08:25:10.668 INFO:tasks.workunit.client.0.smithi090.stdout:4/682: dwrite d3/d8e/fd5 [0,4194304] 0 2024-04-24T08:25:10.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/691: dread d2/df/d1a/f4c [0,4194304] 0 2024-04-24T08:25:10.670 INFO:tasks.workunit.client.0.smithi090.stdout:4/683: readlink d3/d18/d5c/db5/d69/lb8 0 2024-04-24T08:25:10.671 INFO:tasks.workunit.client.1.smithi149.stdout:4/952: readlink d2/d18/d1e/d3c/d48/l83 0 2024-04-24T08:25:10.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/724: dwrite d2/f54 [0,4194304] 0 2024-04-24T08:25:10.673 INFO:tasks.workunit.client.0.smithi090.stdout:2/808: creat d0/d7/d67/d7f/f10e x:0 0 0 2024-04-24T08:25:10.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/860: dwrite d0/d6/db/d27/d64/db3/fdd [0,4194304] 0 2024-04-24T08:25:10.690 INFO:tasks.workunit.client.1.smithi149.stdout:0/939: rename d2/d17 to d2/d12f 0 2024-04-24T08:25:10.696 INFO:tasks.workunit.client.0.smithi090.stdout:5/804: getdents d8/d99 0 2024-04-24T08:25:10.701 INFO:tasks.workunit.client.1.smithi149.stdout:4/953: dread d2/d18/d1e/d3c/d48/d69/d7d/d8d/faf [0,4194304] 0 2024-04-24T08:25:10.702 INFO:tasks.workunit.client.0.smithi090.stdout:2/809: dread d0/f3b [0,4194304] 0 2024-04-24T08:25:10.703 INFO:tasks.workunit.client.0.smithi090.stdout:6/767: rename d8/dc/d19/d8e/d38/l56 to d8/d1d/d1f/d3e/d4f/led 0 2024-04-24T08:25:10.704 INFO:tasks.workunit.client.0.smithi090.stdout:4/684: fsync d3/d18/d5c/f47 0 2024-04-24T08:25:10.705 INFO:tasks.workunit.client.0.smithi090.stdout:3/692: creat d2/df/d1a/d37/d53/fe7 x:0 0 0 2024-04-24T08:25:10.708 INFO:tasks.workunit.client.0.smithi090.stdout:6/768: write d8/dc/d19/d8e/fa6 [4294877,109958] 0 2024-04-24T08:25:10.709 INFO:tasks.workunit.client.1.smithi149.stdout:6/861: mkdir d0/d1a/d28/d31/d4c/d122 0 2024-04-24T08:25:10.709 INFO:tasks.workunit.client.1.smithi149.stdout:0/940: truncate d2/d9/df/d48/fb7 3181634 0 2024-04-24T08:25:10.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/807: mkdir db/d54/ddb/d115 0 2024-04-24T08:25:10.711 INFO:tasks.workunit.client.1.smithi149.stdout:2/861: mknod d0/c105 0 2024-04-24T08:25:10.712 INFO:tasks.workunit.client.0.smithi090.stdout:4/685: chown d3/d18/d5c/db5/d69/d8b/db2/d49/f4c 1666119 1 2024-04-24T08:25:10.712 INFO:tasks.workunit.client.0.smithi090.stdout:0/727: creat d1a/ff5 x:0 0 0 2024-04-24T08:25:10.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/769: dread - d8/dc/d19/d8e/d81/fd9 zero size 2024-04-24T08:25:10.713 INFO:tasks.workunit.client.0.smithi090.stdout:5/805: rmdir d8/da/d14/d2f 39 2024-04-24T08:25:10.714 INFO:tasks.workunit.client.1.smithi149.stdout:0/941: rmdir d2/d3c/d57 39 2024-04-24T08:25:10.718 INFO:tasks.workunit.client.0.smithi090.stdout:0/728: write d1a/d4b/d53/d79/d35/db8/fdd [641435,76559] 0 2024-04-24T08:25:10.721 INFO:tasks.workunit.client.0.smithi090.stdout:1/735: rename d3/d1d/d54/f67 to d3/d5/da/d97/d9c/dc4/f103 0 2024-04-24T08:25:10.722 INFO:tasks.workunit.client.1.smithi149.stdout:8/930: dread d1/de/d18/d32/d65/d8b/d7d/f96 [0,4194304] 0 2024-04-24T08:25:10.722 INFO:tasks.workunit.client.1.smithi149.stdout:2/862: creat d0/d6/db/d28/d7f/d9b/f106 x:0 0 0 2024-04-24T08:25:10.723 INFO:tasks.workunit.client.1.smithi149.stdout:0/942: truncate d2/d9/df/d48/dba/f11c 603859 0 2024-04-24T08:25:10.725 INFO:tasks.workunit.client.0.smithi090.stdout:7/808: truncate db/d17/d29/d32/f40 148663 0 2024-04-24T08:25:10.727 INFO:tasks.workunit.client.0.smithi090.stdout:6/770: symlink d8/dc/d19/d8e/d81/lee 0 2024-04-24T08:25:10.729 INFO:tasks.workunit.client.1.smithi149.stdout:0/943: unlink d2/d9/df/d56/dca/c112 0 2024-04-24T08:25:10.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/863: creat d0/d6/db/d28/d7f/d8f/db9/f107 x:0 0 0 2024-04-24T08:25:10.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/729: dwrite d0/dc/d24/d7c/d4d/f5f [0,4194304] 0 2024-04-24T08:25:10.736 INFO:tasks.workunit.client.1.smithi149.stdout:8/931: write d1/fbf [6737145,79189] 0 2024-04-24T08:25:10.739 INFO:tasks.workunit.client.0.smithi090.stdout:9/730: stat d0/d2a/f98 0 2024-04-24T08:25:10.741 INFO:tasks.workunit.client.0.smithi090.stdout:3/693: link d2/df/d1a/d37/f55 d2/df/d57/d4f/fe8 0 2024-04-24T08:25:10.741 INFO:tasks.workunit.client.0.smithi090.stdout:1/736: dwrite d3/d1d/fc9 [0,4194304] 0 2024-04-24T08:25:10.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/731: fsync d0/dc/d24/d2d/f3f 0 2024-04-24T08:25:10.749 INFO:tasks.workunit.client.0.smithi090.stdout:9/732: read d0/dc/d24/d2d/d49/d81/d87/f8a [3021491,60666] 0 2024-04-24T08:25:10.753 INFO:tasks.workunit.client.1.smithi149.stdout:4/954: dread d2/d10/d91/fc4 [0,4194304] 0 2024-04-24T08:25:10.753 INFO:tasks.workunit.client.1.smithi149.stdout:6/862: dwrite d0/d1a/d28/f2c [0,4194304] 0 2024-04-24T08:25:10.759 INFO:tasks.workunit.client.0.smithi090.stdout:6/771: creat d8/d1d/d1f/d3e/d7a/fef x:0 0 0 2024-04-24T08:25:10.761 INFO:tasks.workunit.client.1.smithi149.stdout:8/932: mknod d1/de/d18/d32/d65/d8b/d7d/dd1/c141 0 2024-04-24T08:25:10.762 INFO:tasks.workunit.client.0.smithi090.stdout:0/729: symlink d1a/d4b/d53/lf6 0 2024-04-24T08:25:10.770 INFO:tasks.workunit.client.0.smithi090.stdout:4/686: dread d3/d18/d5c/db5/d69/d8b/db2/d49/f53 [0,4194304] 0 2024-04-24T08:25:10.770 INFO:tasks.workunit.client.0.smithi090.stdout:1/737: dwrite d3/d5/fa5 [0,4194304] 0 2024-04-24T08:25:10.808 INFO:tasks.workunit.client.0.smithi090.stdout:3/694: dread d2/df/d1a/d37/f8a [0,4194304] 0 2024-04-24T08:25:10.819 INFO:tasks.workunit.client.1.smithi149.stdout:2/864: rmdir d0 39 2024-04-24T08:25:10.863 INFO:tasks.workunit.client.0.smithi090.stdout:8/725: getdents d2 0 2024-04-24T08:25:10.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/944: truncate d2/d9/f13 1110620 0 2024-04-24T08:25:10.873 INFO:tasks.workunit.client.1.smithi149.stdout:2/865: write d0/d6/db/d39/d3d/fd6 [844584,92612] 0 2024-04-24T08:25:10.878 INFO:tasks.workunit.client.0.smithi090.stdout:9/733: fsync d0/dc/d24/d7c/d4d/d32/fce 0 2024-04-24T08:25:10.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/863: symlink d0/d1a/d28/d31/d4c/d80/d11c/l123 0 2024-04-24T08:25:10.882 INFO:tasks.workunit.client.0.smithi090.stdout:9/734: write d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf [5021444,97415] 0 2024-04-24T08:25:10.884 INFO:tasks.workunit.client.1.smithi149.stdout:8/933: mkdir d1/de/d18/d142 0 2024-04-24T08:25:10.891 INFO:tasks.workunit.client.0.smithi090.stdout:0/730: creat d1a/d4b/d53/d79/d69/d90/dcc/ff7 x:0 0 0 2024-04-24T08:25:10.893 INFO:tasks.workunit.client.0.smithi090.stdout:0/731: stat d1a/d4b/d53/d79/d58/de7 0 2024-04-24T08:25:10.896 INFO:tasks.workunit.client.1.smithi149.stdout:6/864: mkdir d0/d1a/d28/d31/d38/d54/d124 0 2024-04-24T08:25:10.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/687: creat d3/d18/d5c/db5/d69/d8b/db2/d49/fde x:0 0 0 2024-04-24T08:25:10.911 INFO:tasks.workunit.client.1.smithi149.stdout:6/865: fsync d0/db7/fda 0 2024-04-24T08:25:10.911 INFO:tasks.workunit.client.0.smithi090.stdout:6/772: dread d8/dc/fa1 [0,4194304] 0 2024-04-24T08:25:10.929 INFO:tasks.workunit.client.1.smithi149.stdout:6/866: rename d0/d1a/d28/d31/d38/l5f to d0/d1a/d28/d4a/l125 0 2024-04-24T08:25:10.931 INFO:tasks.workunit.client.0.smithi090.stdout:9/735: dread - d0/dc/d24/d7c/d4d/d32/f95 zero size 2024-04-24T08:25:10.932 INFO:tasks.workunit.client.1.smithi149.stdout:2/866: getdents d0/d6/db/d28/db0 0 2024-04-24T08:25:10.933 INFO:tasks.workunit.client.0.smithi090.stdout:9/736: read d0/dc/d24/fd2 [1280682,20408] 0 2024-04-24T08:25:10.933 INFO:tasks.workunit.client.1.smithi149.stdout:2/867: stat d0/f2d 0 2024-04-24T08:25:10.939 INFO:tasks.workunit.client.1.smithi149.stdout:6/867: rename d0/d1a/d28/d31/d4c/d80/d6f/d97/fe3 to d0/db7/f126 0 2024-04-24T08:25:10.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/695: creat d2/df/d1a/d37/de3/fe9 x:0 0 0 2024-04-24T08:25:10.945 INFO:tasks.workunit.client.1.smithi149.stdout:6/868: creat d0/db7/f127 x:0 0 0 2024-04-24T08:25:10.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/696: dread - d2/df/fa5 zero size 2024-04-24T08:25:10.947 INFO:tasks.workunit.client.0.smithi090.stdout:6/773: write d8/dc/d19/d8e/d81/d96/d97/fec [2329114,128915] 0 2024-04-24T08:25:10.949 INFO:tasks.workunit.client.1.smithi149.stdout:2/868: mknod d0/d6/db/d27/d64/d66/db1/dc9/d102/c108 0 2024-04-24T08:25:10.979 INFO:tasks.workunit.client.0.smithi090.stdout:3/697: dwrite d2/df/d1a/d37/f8a [0,4194304] 0 2024-04-24T08:25:10.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/698: symlink d2/d15/d36/d6f/d9d/lea 0 2024-04-24T08:25:10.985 INFO:tasks.workunit.client.0.smithi090.stdout:3/699: readlink d2/lbe 0 2024-04-24T08:25:10.988 INFO:tasks.workunit.client.0.smithi090.stdout:3/700: write d2/d15/d36/f83 [1057516,34881] 0 2024-04-24T08:25:11.000 INFO:tasks.workunit.client.0.smithi090.stdout:9/737: dread d0/dc/d24/d2d/f65 [4194304,4194304] 0 2024-04-24T08:25:11.002 INFO:tasks.workunit.client.0.smithi090.stdout:3/701: dread d2/df/d1a/d37/f55 [0,4194304] 0 2024-04-24T08:25:11.020 INFO:tasks.workunit.client.0.smithi090.stdout:9/738: dwrite d0/dc/d24/d7c/fb0 [0,4194304] 0 2024-04-24T08:25:11.023 INFO:tasks.workunit.client.0.smithi090.stdout:9/739: readlink d0/dc/d24/d7c/d4d/d32/d9f/d78/l99 0 2024-04-24T08:25:11.025 INFO:tasks.workunit.client.1.smithi149.stdout:4/955: write d2/d10/d1a/fda [890762,108132] 0 2024-04-24T08:25:11.029 INFO:tasks.workunit.client.0.smithi090.stdout:9/740: symlink d0/dc/d24/d7c/d4d/le4 0 2024-04-24T08:25:11.040 INFO:tasks.workunit.client.1.smithi149.stdout:4/956: symlink d2/d18/d1e/d3c/d59/d12e/l12f 0 2024-04-24T08:25:11.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/957: creat d2/d24/d7f/d101/f130 x:0 0 0 2024-04-24T08:25:11.047 INFO:tasks.workunit.client.1.smithi149.stdout:8/934: write d1/de/d18/fae [28396,12252] 0 2024-04-24T08:25:11.052 INFO:tasks.workunit.client.0.smithi090.stdout:1/738: dwrite d3/d1d/d34/d5a/f5b [0,4194304] 0 2024-04-24T08:25:11.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/958: readlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/db2/l126 0 2024-04-24T08:25:11.056 INFO:tasks.workunit.client.0.smithi090.stdout:1/739: dread - d3/d5/d3e/da3/fd0 zero size 2024-04-24T08:25:11.062 INFO:tasks.workunit.client.1.smithi149.stdout:0/945: dwrite d2/d9/df/d56/f5f [0,4194304] 0 2024-04-24T08:25:11.071 INFO:tasks.workunit.client.0.smithi090.stdout:8/726: rmdir d2/d97 39 2024-04-24T08:25:11.073 INFO:tasks.workunit.client.1.smithi149.stdout:0/946: read d2/d9/df/d48/f4f [221865,32775] 0 2024-04-24T08:25:11.074 INFO:tasks.workunit.client.1.smithi149.stdout:2/869: rmdir d0/d6/db/d27/d64/d66/db1/dc9 39 2024-04-24T08:25:11.074 INFO:tasks.workunit.client.0.smithi090.stdout:0/732: write d1a/d4b/d53/d79/d35/d62/fab [422645,102963] 0 2024-04-24T08:25:11.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/741: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f85 [0,4194304] 0 2024-04-24T08:25:11.084 INFO:tasks.workunit.client.1.smithi149.stdout:2/870: rename d0/d6/f12 to d0/d6/db/d39/d78/d8a/f109 0 2024-04-24T08:25:11.085 INFO:tasks.workunit.client.0.smithi090.stdout:0/733: unlink d1a/d4b/d53/d79/d3c/f84 0 2024-04-24T08:25:11.086 INFO:tasks.workunit.client.0.smithi090.stdout:0/734: fsync d1a/d4b/d53/fbb 0 2024-04-24T08:25:11.087 INFO:tasks.workunit.client.0.smithi090.stdout:9/742: symlink d0/dc/d24/d7c/d4d/d8c/le5 0 2024-04-24T08:25:11.100 INFO:tasks.workunit.client.0.smithi090.stdout:2/810: rename d0/d5e/lac to d0/d7/d67/d7f/d31/d46/l10f 0 2024-04-24T08:25:11.100 INFO:tasks.workunit.client.1.smithi149.stdout:2/871: link d0/d6/db/d27/d64/d66/db1/dc9/d102/d98/fa5 d0/d6/db/d27/f10a 0 2024-04-24T08:25:11.103 INFO:tasks.workunit.client.1.smithi149.stdout:2/872: dread - d0/d6/db/d28/db0/ff3 zero size 2024-04-24T08:25:11.103 INFO:tasks.workunit.client.0.smithi090.stdout:5/806: rename d8/d59/dc7/d7a/d50/d9f/ca4 to d8/d25/c107 0 2024-04-24T08:25:11.106 INFO:tasks.workunit.client.0.smithi090.stdout:7/809: rename db/d17/d29/d38/d3e/f58 to db/dbd/f116 0 2024-04-24T08:25:11.107 INFO:tasks.workunit.client.1.smithi149.stdout:6/869: dwrite d0/d1a/f2b [0,4194304] 0 2024-04-24T08:25:11.107 INFO:tasks.workunit.client.0.smithi090.stdout:9/743: dread d0/d19/f1a [0,4194304] 0 2024-04-24T08:25:11.108 INFO:tasks.workunit.client.0.smithi090.stdout:7/810: write db/d54/d78/ffe [196209,66987] 0 2024-04-24T08:25:11.117 INFO:tasks.workunit.client.0.smithi090.stdout:4/688: rename d3/d18 to d3/d8e/ddf 0 2024-04-24T08:25:11.119 INFO:tasks.workunit.client.0.smithi090.stdout:4/689: write d3/d24/f2b [5371820,11481] 0 2024-04-24T08:25:11.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/811: creat db/d54/d83/f117 x:0 0 0 2024-04-24T08:25:11.123 INFO:tasks.workunit.client.0.smithi090.stdout:5/807: truncate d8/da/d14/d2f/d44/f73 451317 0 2024-04-24T08:25:11.131 INFO:tasks.workunit.client.0.smithi090.stdout:9/744: symlink d0/dc/d24/le6 0 2024-04-24T08:25:11.132 INFO:tasks.workunit.client.0.smithi090.stdout:9/745: chown d0/dc/d24/d7c/l75 98147 1 2024-04-24T08:25:11.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/811: link d0/d7/d67/d7f/d31/d90/f9d d0/d7/d28/daa/def/db4/dcd/f110 0 2024-04-24T08:25:11.137 INFO:tasks.workunit.client.1.smithi149.stdout:6/870: link d0/d1a/d28/d31/f37 d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/f128 0 2024-04-24T08:25:11.137 INFO:tasks.workunit.client.0.smithi090.stdout:2/812: stat d0/d14/d36/dce/dee/l68 0 2024-04-24T08:25:11.145 INFO:tasks.workunit.client.0.smithi090.stdout:1/740: rmdir d3/d5/da/d97 39 2024-04-24T08:25:11.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/702: truncate d2/df/d1a/d37/d59/f7b 1954935 0 2024-04-24T08:25:11.148 INFO:tasks.workunit.client.0.smithi090.stdout:1/741: dread - d3/d5/d3e/da3/fc8 zero size 2024-04-24T08:25:11.151 INFO:tasks.workunit.client.0.smithi090.stdout:3/703: fdatasync d2/df/d57/d4f/f72 0 2024-04-24T08:25:11.154 INFO:tasks.workunit.client.0.smithi090.stdout:3/704: write d2/d15/d36/d6f/d92/fa9 [1327902,56473] 0 2024-04-24T08:25:11.155 INFO:tasks.workunit.client.1.smithi149.stdout:0/947: creat d2/d9/df/d48/f130 x:0 0 0 2024-04-24T08:25:11.158 INFO:tasks.workunit.client.0.smithi090.stdout:3/705: mkdir d2/df/db5/deb 0 2024-04-24T08:25:11.160 INFO:tasks.workunit.client.0.smithi090.stdout:7/812: unlink db/d17/d29/d32/c51 0 2024-04-24T08:25:11.161 INFO:tasks.workunit.client.1.smithi149.stdout:0/948: symlink d2/d9/df/d2f/d6c/l131 0 2024-04-24T08:25:11.163 INFO:tasks.workunit.client.0.smithi090.stdout:7/813: truncate db/dbd/fd0 544851 0 2024-04-24T08:25:11.163 INFO:tasks.workunit.client.0.smithi090.stdout:3/706: symlink d2/df/d1a/d37/d53/d60/d9b/lec 0 2024-04-24T08:25:11.165 INFO:tasks.workunit.client.1.smithi149.stdout:0/949: read - d2/d12f/f107 zero size 2024-04-24T08:25:11.165 INFO:tasks.workunit.client.1.smithi149.stdout:6/871: dread d0/d1a/d28/d31/d4c/d80/f68 [0,4194304] 0 2024-04-24T08:25:11.166 INFO:tasks.workunit.client.0.smithi090.stdout:2/813: dread d0/d7/d67/f72 [0,4194304] 0 2024-04-24T08:25:11.166 INFO:tasks.workunit.client.0.smithi090.stdout:3/707: mkdir d2/d15/d36/d6f/d9d/ded 0 2024-04-24T08:25:11.170 INFO:tasks.workunit.client.1.smithi149.stdout:8/935: write d1/de/d18/fb7 [1205851,58752] 0 2024-04-24T08:25:11.175 INFO:tasks.workunit.client.1.smithi149.stdout:8/936: symlink d1/d1f/d5e/d92/l143 0 2024-04-24T08:25:11.177 INFO:tasks.workunit.client.1.smithi149.stdout:6/872: dread d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f8c [0,4194304] 0 2024-04-24T08:25:11.179 INFO:tasks.workunit.client.1.smithi149.stdout:4/959: dwrite d2/d18/f1b [0,4194304] 0 2024-04-24T08:25:11.182 INFO:tasks.workunit.client.1.smithi149.stdout:8/937: symlink d1/de/d18/d91/l144 0 2024-04-24T08:25:11.182 INFO:tasks.workunit.client.1.smithi149.stdout:6/873: readlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/l10e 0 2024-04-24T08:25:11.184 INFO:tasks.workunit.client.1.smithi149.stdout:6/874: stat d0/d1a/f2b 0 2024-04-24T08:25:11.188 INFO:tasks.workunit.client.1.smithi149.stdout:4/960: rmdir d2/d18/d1e 39 2024-04-24T08:25:11.189 INFO:tasks.workunit.client.1.smithi149.stdout:0/950: dwrite d2/d3c/dc7/f11a [0,4194304] 0 2024-04-24T08:25:11.204 INFO:tasks.workunit.client.1.smithi149.stdout:4/961: mkdir d2/d10/d1a/dac/ddd/d131 0 2024-04-24T08:25:11.205 INFO:tasks.workunit.client.1.smithi149.stdout:0/951: truncate d2/d12f/d41/fb3 427349 0 2024-04-24T08:25:11.205 INFO:tasks.workunit.client.1.smithi149.stdout:6/875: unlink d0/l22 0 2024-04-24T08:25:11.209 INFO:tasks.workunit.client.1.smithi149.stdout:2/873: dwrite d0/d6/db/d39/d3d/f51 [0,4194304] 0 2024-04-24T08:25:11.210 INFO:tasks.workunit.client.0.smithi090.stdout:0/735: write d1a/f3f [1246289,129823] 0 2024-04-24T08:25:11.210 INFO:tasks.workunit.client.0.smithi090.stdout:2/814: fdatasync d0/d7/d28/daa/ff5 0 2024-04-24T08:25:11.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/708: chown d2/df/d1a/d37/d59 719 1 2024-04-24T08:25:11.210 INFO:tasks.workunit.client.1.smithi149.stdout:6/876: fdatasync d0/d1a/d28/d4a/d11f/fe5 0 2024-04-24T08:25:11.212 INFO:tasks.workunit.client.0.smithi090.stdout:4/690: write d3/d8e/ddf/f70 [281514,105047] 0 2024-04-24T08:25:11.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/736: chown d1a/d4b/f4d 1 1 2024-04-24T08:25:11.215 INFO:tasks.workunit.client.0.smithi090.stdout:9/746: write d0/dc/d24/d7c/f82 [100466,27047] 0 2024-04-24T08:25:11.222 INFO:tasks.workunit.client.0.smithi090.stdout:3/709: truncate d2/d90/fe4 9259620 0 2024-04-24T08:25:11.227 INFO:tasks.workunit.client.0.smithi090.stdout:8/727: dwrite d2/da/f87 [4194304,4194304] 0 2024-04-24T08:25:11.228 INFO:tasks.workunit.client.1.smithi149.stdout:6/877: creat d0/d1a/d28/d31/d38/d106/f129 x:0 0 0 2024-04-24T08:25:11.230 INFO:tasks.workunit.client.1.smithi149.stdout:8/938: dwrite d1/d1f/f28 [0,4194304] 0 2024-04-24T08:25:11.231 INFO:tasks.workunit.client.0.smithi090.stdout:2/815: read - d0/d2e/f93 zero size 2024-04-24T08:25:11.233 INFO:tasks.workunit.client.1.smithi149.stdout:6/878: chown d0/d1a/d28/d4a/fb2 1049963 1 2024-04-24T08:25:11.233 INFO:tasks.workunit.client.0.smithi090.stdout:5/808: dwrite d8/da/d3a/d96/dcb/d21/f41 [4194304,4194304] 0 2024-04-24T08:25:11.236 INFO:tasks.workunit.client.1.smithi149.stdout:4/962: write d2/d10/f81 [5107217,98498] 0 2024-04-24T08:25:11.237 INFO:tasks.workunit.client.0.smithi090.stdout:8/728: creat d2/da/db0/ff5 x:0 0 0 2024-04-24T08:25:11.239 INFO:tasks.workunit.client.0.smithi090.stdout:9/747: stat d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96/fa1 0 2024-04-24T08:25:11.242 INFO:tasks.workunit.client.1.smithi149.stdout:8/939: rename d1/d1f/d22/c94 to d1/dce/de9/c145 0 2024-04-24T08:25:11.242 INFO:tasks.workunit.client.0.smithi090.stdout:3/710: fdatasync d2/df/d21/f30 0 2024-04-24T08:25:11.244 INFO:tasks.workunit.client.1.smithi149.stdout:6/879: write d0/d1a/d28/d31/d33/fa8 [434819,5138] 0 2024-04-24T08:25:11.250 INFO:tasks.workunit.client.0.smithi090.stdout:5/809: rename d8/d25/fc8 to d8/d59/dc7/d7a/da1/f108 0 2024-04-24T08:25:11.250 INFO:tasks.workunit.client.0.smithi090.stdout:9/748: truncate d0/dc/d24/d2d/d49/fb6 1407738 0 2024-04-24T08:25:11.255 INFO:tasks.workunit.client.0.smithi090.stdout:4/691: dwrite d3/d24/d92/fc2 [0,4194304] 0 2024-04-24T08:25:11.255 INFO:tasks.workunit.client.1.smithi149.stdout:4/963: dread - d2/d18/d1e/d3c/df0/f109 zero size 2024-04-24T08:25:11.256 INFO:tasks.workunit.client.1.smithi149.stdout:0/952: write d2/d9/df/d2f/d6c/d72/f99 [799337,72577] 0 2024-04-24T08:25:11.256 INFO:tasks.workunit.client.0.smithi090.stdout:4/692: dread - d3/d8e/ddf/d1d/fa1 zero size 2024-04-24T08:25:11.258 INFO:tasks.workunit.client.0.smithi090.stdout:3/711: rmdir d2/df/d1a/d37/de3 39 2024-04-24T08:25:11.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/816: rename d0/d7/d28/daa/def/d50/f5c to d0/d7/d28/daa/def/d50/d96/f111 0 2024-04-24T08:25:11.263 INFO:tasks.workunit.client.1.smithi149.stdout:2/874: truncate d0/d6/db/f72 6095221 0 2024-04-24T08:25:11.267 INFO:tasks.workunit.client.1.smithi149.stdout:8/940: creat d1/de/d18/d32/d65/f146 x:0 0 0 2024-04-24T08:25:11.267 INFO:tasks.workunit.client.1.smithi149.stdout:4/964: rmdir d2/d18/dd6 39 2024-04-24T08:25:11.268 INFO:tasks.workunit.client.0.smithi090.stdout:8/729: creat d2/da/d13/d64/d24/dad/ff6 x:0 0 0 2024-04-24T08:25:11.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/953: mknod d2/d9/df/d48/dba/d12c/c132 0 2024-04-24T08:25:11.271 INFO:tasks.workunit.client.0.smithi090.stdout:5/810: unlink d8/da/d3a/d96/dcb/d21/d4c/cb5 0 2024-04-24T08:25:11.271 INFO:tasks.workunit.client.1.smithi149.stdout:8/941: chown d1/d1f/d22/f5f 243704508 1 2024-04-24T08:25:11.272 INFO:tasks.workunit.client.0.smithi090.stdout:0/737: dread d1a/d4b/d53/f7f [0,4194304] 0 2024-04-24T08:25:11.273 INFO:tasks.workunit.client.1.smithi149.stdout:0/954: dread - d2/d9/df/d48/f126 zero size 2024-04-24T08:25:11.281 INFO:tasks.workunit.client.0.smithi090.stdout:4/693: fdatasync d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f35 0 2024-04-24T08:25:11.283 INFO:tasks.workunit.client.1.smithi149.stdout:2/875: rename d0/d6/db/d39/f90 to d0/d6/db/d39/d78/d7c/f10b 0 2024-04-24T08:25:11.284 INFO:tasks.workunit.client.1.smithi149.stdout:8/942: mknod d1/de/d18/d32/d65/d8b/dc9/c147 0 2024-04-24T08:25:11.285 INFO:tasks.workunit.client.0.smithi090.stdout:9/749: creat d0/da4/fe7 x:0 0 0 2024-04-24T08:25:11.286 INFO:tasks.workunit.client.1.smithi149.stdout:2/876: write d0/d6/db/d39/d3d/d44/d75/fff [956799,66651] 0 2024-04-24T08:25:11.286 INFO:tasks.workunit.client.0.smithi090.stdout:9/750: readlink d0/dc/d24/d7c/d4d/l44 0 2024-04-24T08:25:11.288 INFO:tasks.workunit.client.1.smithi149.stdout:0/955: symlink d2/d9/df/d56/dca/l133 0 2024-04-24T08:25:11.288 INFO:tasks.workunit.client.0.smithi090.stdout:8/730: chown d2/d62/d6b/c77 3244 1 2024-04-24T08:25:11.290 INFO:tasks.workunit.client.1.smithi149.stdout:0/956: chown d2/d9/df/d2f/d6c/dc1/cd3 343396210 1 2024-04-24T08:25:11.291 INFO:tasks.workunit.client.0.smithi090.stdout:8/731: read d2/da/d13/f52 [78825,117401] 0 2024-04-24T08:25:11.292 INFO:tasks.workunit.client.1.smithi149.stdout:0/957: fdatasync d2/d3c/dc7/f11a 0 2024-04-24T08:25:11.293 INFO:tasks.workunit.client.0.smithi090.stdout:8/732: dread - d2/da/d13/d60/fe6 zero size 2024-04-24T08:25:11.294 INFO:tasks.workunit.client.1.smithi149.stdout:8/943: creat d1/de/d18/d32/d63/d76/d12f/f148 x:0 0 0 2024-04-24T08:25:11.295 INFO:tasks.workunit.client.0.smithi090.stdout:2/817: symlink d0/d7/d67/d7f/d31/ddb/l112 0 2024-04-24T08:25:11.297 INFO:tasks.workunit.client.1.smithi149.stdout:8/944: dread - d1/de/d18/d32/d65/dad/f126 zero size 2024-04-24T08:25:11.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/818: chown d0/d7/d28/daa/db1/lf9 1 1 2024-04-24T08:25:11.303 INFO:tasks.workunit.client.1.smithi149.stdout:4/965: write d2/d10/d1a/fba [2501542,11784] 0 2024-04-24T08:25:11.305 INFO:tasks.workunit.client.1.smithi149.stdout:0/958: rmdir d2/d9/de6 39 2024-04-24T08:25:11.308 INFO:tasks.workunit.client.0.smithi090.stdout:3/712: dwrite d2/fa7 [4194304,4194304] 0 2024-04-24T08:25:11.310 INFO:tasks.workunit.client.0.smithi090.stdout:3/713: fdatasync d2/df/d1a/d37/d53/fe1 0 2024-04-24T08:25:11.314 INFO:tasks.workunit.client.0.smithi090.stdout:3/714: readlink d2/df/d1a/d37/d53/d60/ld3 0 2024-04-24T08:25:11.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/738: write d1a/f1b [4539991,29732] 0 2024-04-24T08:25:11.315 INFO:tasks.workunit.client.1.smithi149.stdout:8/945: rename d1/d1f/d22/f101 to d1/de/d18/d32/d63/f149 0 2024-04-24T08:25:11.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/819: rmdir d0/d14/d17 39 2024-04-24T08:25:11.320 INFO:tasks.workunit.client.1.smithi149.stdout:0/959: fsync d2/d12f/f24 0 2024-04-24T08:25:11.326 INFO:tasks.workunit.client.1.smithi149.stdout:2/877: creat d0/d6/db/d27/d64/f10c x:0 0 0 2024-04-24T08:25:11.326 INFO:tasks.workunit.client.0.smithi090.stdout:3/715: unlink d2/df/d21/l58 0 2024-04-24T08:25:11.330 INFO:tasks.workunit.client.1.smithi149.stdout:8/946: creat d1/de/d18/d32/d65/d8b/d7d/ddc/f14a x:0 0 0 2024-04-24T08:25:11.331 INFO:tasks.workunit.client.1.smithi149.stdout:6/880: dwrite d0/d1a/d28/d31/f82 [0,4194304] 0 2024-04-24T08:25:11.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/820: creat d0/d5e/de2/f113 x:0 0 0 2024-04-24T08:25:11.335 INFO:tasks.workunit.client.0.smithi090.stdout:8/733: mknod d2/d97/def/cf7 0 2024-04-24T08:25:11.336 INFO:tasks.workunit.client.0.smithi090.stdout:4/694: dwrite d3/d8e/ddf/d1d/f1f [0,4194304] 0 2024-04-24T08:25:11.336 INFO:tasks.workunit.client.1.smithi149.stdout:8/947: chown d1/de/d18/d32/d65/d8b/d7d/ddc/ff0 778796540 1 2024-04-24T08:25:11.337 INFO:tasks.workunit.client.1.smithi149.stdout:8/948: stat d1/de/d18/fae 0 2024-04-24T08:25:11.337 INFO:tasks.workunit.client.0.smithi090.stdout:3/716: creat d2/df/d21/fee x:0 0 0 2024-04-24T08:25:11.339 INFO:tasks.workunit.client.0.smithi090.stdout:9/751: link d0/dc/d24/d2d/cbd d0/d2a/ce8 0 2024-04-24T08:25:11.341 INFO:tasks.workunit.client.1.smithi149.stdout:4/966: rmdir d2/d10/d91/df7/d116/d12c 0 2024-04-24T08:25:11.358 INFO:tasks.workunit.client.1.smithi149.stdout:4/967: rmdir d2/d18/d1e/d3c/df0 39 2024-04-24T08:25:11.358 INFO:tasks.workunit.client.1.smithi149.stdout:4/968: readlink d2/d24/da7/lbc 0 2024-04-24T08:25:11.358 INFO:tasks.workunit.client.1.smithi149.stdout:4/969: chown d2/d24/d2e/d3f/d94/dfd 61953 1 2024-04-24T08:25:11.359 INFO:tasks.workunit.client.1.smithi149.stdout:0/960: rename d2/d9/df/d48/f130 to d2/d9/df/d2f/d6c/dc1/d4e/f134 0 2024-04-24T08:25:11.361 INFO:tasks.workunit.client.1.smithi149.stdout:8/949: unlink d1/de/d18/d32/d65/cb1 0 2024-04-24T08:25:11.363 INFO:tasks.workunit.client.1.smithi149.stdout:6/881: creat d0/d1a/d28/d31/d38/d54/d124/f12a x:0 0 0 2024-04-24T08:25:11.364 INFO:tasks.workunit.client.1.smithi149.stdout:8/950: dread - d1/d1f/d5e/fa0 zero size 2024-04-24T08:25:11.366 INFO:tasks.workunit.client.1.smithi149.stdout:2/878: write d0/d6/db/d27/f47 [592108,7728] 0 2024-04-24T08:25:11.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/961: unlink d2/d9/d1a/d36/d85/cff 0 2024-04-24T08:25:11.368 INFO:tasks.workunit.client.1.smithi149.stdout:4/970: read d2/d24/d2e/f98 [4773076,58259] 0 2024-04-24T08:25:11.368 INFO:tasks.workunit.client.1.smithi149.stdout:2/879: write d0/f2d [1366880,117600] 0 2024-04-24T08:25:11.375 INFO:tasks.workunit.client.1.smithi149.stdout:6/882: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/d10c/f12b x:0 0 0 2024-04-24T08:25:11.375 INFO:tasks.workunit.client.0.smithi090.stdout:6/774: sync 2024-04-24T08:25:11.376 INFO:tasks.workunit.client.1.smithi149.stdout:0/962: truncate d2/d9/fd8 3863931 0 2024-04-24T08:25:11.376 INFO:tasks.workunit.client.1.smithi149.stdout:4/971: symlink d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/l132 0 2024-04-24T08:25:11.377 INFO:tasks.workunit.client.0.smithi090.stdout:4/695: write d3/fc4 [141742,31951] 0 2024-04-24T08:25:11.377 INFO:tasks.workunit.client.0.smithi090.stdout:0/739: mknod d1a/d4b/d53/d79/d35/dea/cf8 0 2024-04-24T08:25:11.379 INFO:tasks.workunit.client.1.smithi149.stdout:2/880: dread d0/d6/db/d39/d3d/fd6 [0,4194304] 0 2024-04-24T08:25:11.382 INFO:tasks.workunit.client.1.smithi149.stdout:2/881: chown d0/d6/db/d27/d81/d9f 279745 1 2024-04-24T08:25:11.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/963: creat d2/d9/df/d2f/d12a/f135 x:0 0 0 2024-04-24T08:25:11.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/972: fdatasync d2/d18/db9/dd8/fe0 0 2024-04-24T08:25:11.385 INFO:tasks.workunit.client.1.smithi149.stdout:2/882: chown d0/d6/db/l32 1044042 1 2024-04-24T08:25:11.386 INFO:tasks.workunit.client.0.smithi090.stdout:0/740: readlink d1a/d4b/d53/d79/d35/d68/l77 0 2024-04-24T08:25:11.386 INFO:tasks.workunit.client.1.smithi149.stdout:6/883: fdatasync d0/d1a/d28/d31/d4c/d76/fe1 0 2024-04-24T08:25:11.386 INFO:tasks.workunit.client.0.smithi090.stdout:8/734: creat d2/da/d13/d60/d91/ff8 x:0 0 0 2024-04-24T08:25:11.387 INFO:tasks.workunit.client.1.smithi149.stdout:4/973: chown d2/d18/db9/dd8/fa0 4012462 1 2024-04-24T08:25:11.388 INFO:tasks.workunit.client.0.smithi090.stdout:0/741: write d1a/f3f [276473,76145] 0 2024-04-24T08:25:11.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/742: chown d1a/d4b/d53/d79/d35/d62 13986 1 2024-04-24T08:25:11.398 INFO:tasks.workunit.client.1.smithi149.stdout:2/883: rename d0/d6/db to d0/d6/db/d27/d64/d10d 22 2024-04-24T08:25:11.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/951: dread d1/de/d18/f45 [0,4194304] 0 2024-04-24T08:25:11.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/821: dwrite d0/d7/de1/f43 [0,4194304] 0 2024-04-24T08:25:11.403 INFO:tasks.workunit.client.1.smithi149.stdout:2/884: chown d0/d6/db/d28/d7f/d9b/ff6 19269430 1 2024-04-24T08:25:11.406 INFO:tasks.workunit.client.1.smithi149.stdout:2/885: chown d0/d6/db/cf 244111 1 2024-04-24T08:25:11.411 INFO:tasks.workunit.client.1.smithi149.stdout:8/952: mkdir d1/de/d18/d32/d65/d8b/d14b 0 2024-04-24T08:25:11.416 INFO:tasks.workunit.client.1.smithi149.stdout:6/884: creat d0/d1a/d28/f12c x:0 0 0 2024-04-24T08:25:11.416 INFO:tasks.workunit.client.1.smithi149.stdout:0/964: dread d2/d9/d1a/d36/d85/ffa [0,4194304] 0 2024-04-24T08:25:11.419 INFO:tasks.workunit.client.1.smithi149.stdout:2/886: chown d0/d6/db/d27/d81/db4/cfd 120 1 2024-04-24T08:25:11.423 INFO:tasks.workunit.client.1.smithi149.stdout:2/887: readlink d0/d6/db/d28/l5b 0 2024-04-24T08:25:11.430 INFO:tasks.workunit.client.1.smithi149.stdout:2/888: creat d0/d6/db/d39/d3d/d44/d75/f10e x:0 0 0 2024-04-24T08:25:11.432 INFO:tasks.workunit.client.1.smithi149.stdout:4/974: dread d2/d18/d1e/f33 [0,4194304] 0 2024-04-24T08:25:11.439 INFO:tasks.workunit.client.1.smithi149.stdout:8/953: dwrite d1/de/d18/d32/d63/d76/d12f/f132 [0,4194304] 0 2024-04-24T08:25:11.441 INFO:tasks.workunit.client.1.smithi149.stdout:8/954: chown d1/de/d18/d43/fb6 50731778 1 2024-04-24T08:25:11.446 INFO:tasks.workunit.client.1.smithi149.stdout:0/965: truncate d2/d12f/fd1 32488 0 2024-04-24T08:25:11.453 INFO:tasks.workunit.client.0.smithi090.stdout:6/775: mkdir d8/d1d/d29/df0 0 2024-04-24T08:25:11.456 INFO:tasks.workunit.client.0.smithi090.stdout:3/717: mknod d2/df/d1a/d37/d59/d98/cef 0 2024-04-24T08:25:11.458 INFO:tasks.workunit.client.0.smithi090.stdout:4/696: rename d3/d8e/ddf/d5c/l52 to d3/d24/d92/le0 0 2024-04-24T08:25:11.462 INFO:tasks.workunit.client.0.smithi090.stdout:8/735: fdatasync d2/da/d13/d64/d24/f79 0 2024-04-24T08:25:11.463 INFO:tasks.workunit.client.0.smithi090.stdout:9/752: mkdir d0/dc/d24/d7c/d4d/d32/d9f/d72/de9 0 2024-04-24T08:25:11.466 INFO:tasks.workunit.client.0.smithi090.stdout:3/718: dread - d2/d15/d36/d6f/d92/fae zero size 2024-04-24T08:25:11.467 INFO:tasks.workunit.client.0.smithi090.stdout:0/743: rename d1a/d4b/d53/d57/faf to d1a/d4b/d53/d79/d69/d78/d9b/ff9 0 2024-04-24T08:25:11.468 INFO:tasks.workunit.client.0.smithi090.stdout:1/742: sync 2024-04-24T08:25:11.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/776: mknod d8/d1d/d1f/d3e/d4f/d7e/daa/de4/cf1 0 2024-04-24T08:25:11.480 INFO:tasks.workunit.client.0.smithi090.stdout:9/753: creat d0/dc/d24/d2d/d49/d81/fea x:0 0 0 2024-04-24T08:25:11.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/822: creat d0/d7/de1/f114 x:0 0 0 2024-04-24T08:25:11.497 INFO:tasks.workunit.client.0.smithi090.stdout:9/754: symlink d0/dc/d24/d7c/leb 0 2024-04-24T08:25:11.500 INFO:tasks.workunit.client.0.smithi090.stdout:3/719: truncate d2/d15/d36/f5f 421152 0 2024-04-24T08:25:11.503 INFO:tasks.workunit.client.0.smithi090.stdout:1/743: mknod d3/c104 0 2024-04-24T08:25:11.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/697: creat d3/d8e/ddf/d5c/fe1 x:0 0 0 2024-04-24T08:25:11.506 INFO:tasks.workunit.client.1.smithi149.stdout:2/889: mknod d0/d6/db/d39/d3d/d44/df0/df8/c10f 0 2024-04-24T08:25:11.507 INFO:tasks.workunit.client.0.smithi090.stdout:4/698: dread - d3/d8e/fdd zero size 2024-04-24T08:25:11.507 INFO:tasks.workunit.client.0.smithi090.stdout:8/736: link d2/d62/d71/c80 d2/da/d13/d64/db2/db6/dd0/cf9 0 2024-04-24T08:25:11.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/975: mkdir d2/d18/d1e/d3c/d48/d69/d7d/d8d/dae/dbd/d133 0 2024-04-24T08:25:11.510 INFO:tasks.workunit.client.0.smithi090.stdout:2/823: rmdir d0/d7/d28/daa/def/d50/d96 39 2024-04-24T08:25:11.512 INFO:tasks.workunit.client.1.smithi149.stdout:6/885: symlink d0/d1a/d28/d4a/d11f/l12d 0 2024-04-24T08:25:11.514 INFO:tasks.workunit.client.0.smithi090.stdout:6/777: truncate d8/d1d/d1f/fb6 4262497 0 2024-04-24T08:25:11.517 INFO:tasks.workunit.client.0.smithi090.stdout:9/755: mkdir d0/dc/d24/d2d/d49/d62/dcb/dec 0 2024-04-24T08:25:11.517 INFO:tasks.workunit.client.1.smithi149.stdout:4/976: dread d2/d10/d1a/fda [0,4194304] 0 2024-04-24T08:25:11.520 INFO:tasks.workunit.client.0.smithi090.stdout:3/720: dread - d2/df/d1a/d37/d53/d60/f7f zero size 2024-04-24T08:25:11.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/756: stat d0/dc/d24/d7c/d4d/d32 0 2024-04-24T08:25:11.520 INFO:tasks.workunit.client.0.smithi090.stdout:0/744: dread d1a/f1e [0,4194304] 0 2024-04-24T08:25:11.522 INFO:tasks.workunit.client.0.smithi090.stdout:8/737: unlink d2/da/f87 0 2024-04-24T08:25:11.526 INFO:tasks.workunit.client.0.smithi090.stdout:3/721: dread d2/d15/d36/d6f/f9a [0,4194304] 0 2024-04-24T08:25:11.531 INFO:tasks.workunit.client.0.smithi090.stdout:9/757: fdatasync d0/dc/d24/d2d/f47 0 2024-04-24T08:25:11.534 INFO:tasks.workunit.client.1.smithi149.stdout:8/955: write d1/de/d18/d32/d65/d8b/d7d/ff8 [744758,32249] 0 2024-04-24T08:25:11.534 INFO:tasks.workunit.client.0.smithi090.stdout:5/811: sync 2024-04-24T08:25:11.534 INFO:tasks.workunit.client.0.smithi090.stdout:7/814: sync 2024-04-24T08:25:11.537 INFO:tasks.workunit.client.1.smithi149.stdout:2/890: creat d0/d6/db/d39/d3d/f110 x:0 0 0 2024-04-24T08:25:11.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/758: fdatasync d0/dc/d24/d2d/fde 0 2024-04-24T08:25:11.540 INFO:tasks.workunit.client.0.smithi090.stdout:4/699: dwrite d3/d8e/ddf/d1d/fa1 [0,4194304] 0 2024-04-24T08:25:11.540 INFO:tasks.workunit.client.0.smithi090.stdout:5/812: chown d8/da/d3a/d2d/d37/cb7 7 1 2024-04-24T08:25:11.543 INFO:tasks.workunit.client.1.smithi149.stdout:6/886: rename d0/d1a/d28/d31/d38 to d0/d1a/d28/d31/d4c/d4e/d12e 0 2024-04-24T08:25:11.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/778: creat d8/d1d/d29/df0/ff2 x:0 0 0 2024-04-24T08:25:11.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/779: readlink d8/dc/l78 0 2024-04-24T08:25:11.545 INFO:tasks.workunit.client.1.smithi149.stdout:0/966: dwrite d2/d9/df/d48/f66 [0,4194304] 0 2024-04-24T08:25:11.549 INFO:tasks.workunit.client.1.smithi149.stdout:8/956: stat d1/f9 0 2024-04-24T08:25:11.550 INFO:tasks.workunit.client.0.smithi090.stdout:3/722: creat d2/d15/d36/d6f/d9d/ff0 x:0 0 0 2024-04-24T08:25:11.553 INFO:tasks.workunit.client.1.smithi149.stdout:6/887: rename d0/d1a/d28/d31/d4c/d80/d6f to d0/d1a/d28/d31/d4c/d80/d6f/d97/d12f 22 2024-04-24T08:25:11.554 INFO:tasks.workunit.client.0.smithi090.stdout:1/744: dread d3/d5/d58/f60 [0,4194304] 0 2024-04-24T08:25:11.557 INFO:tasks.workunit.client.1.smithi149.stdout:2/891: creat d0/d6/db/d27/d81/db4/f111 x:0 0 0 2024-04-24T08:25:11.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/824: write d0/d7/d67/d7f/d31/d46/fb7 [1017080,102217] 0 2024-04-24T08:25:11.564 INFO:tasks.workunit.client.0.smithi090.stdout:2/825: chown d0/fc8 0 1 2024-04-24T08:25:11.566 INFO:tasks.workunit.client.0.smithi090.stdout:4/700: symlink d3/d8e/ddf/d5c/db5/d77/le2 0 2024-04-24T08:25:11.568 INFO:tasks.workunit.client.0.smithi090.stdout:4/701: fsync d3/d8e/ddf/fbc 0 2024-04-24T08:25:11.569 INFO:tasks.workunit.client.1.smithi149.stdout:4/977: write d2/d18/d1e/d3c/df0/f109 [831737,217] 0 2024-04-24T08:25:11.571 INFO:tasks.workunit.client.1.smithi149.stdout:2/892: symlink d0/l112 0 2024-04-24T08:25:11.572 INFO:tasks.workunit.client.1.smithi149.stdout:4/978: chown d2/d10/d1a/dac/ddd 73968 1 2024-04-24T08:25:11.574 INFO:tasks.workunit.client.1.smithi149.stdout:4/979: read - d2/d18/da4/f11e zero size 2024-04-24T08:25:11.575 INFO:tasks.workunit.client.1.smithi149.stdout:8/957: creat d1/d1f/f14c x:0 0 0 2024-04-24T08:25:11.576 INFO:tasks.workunit.client.1.smithi149.stdout:6/888: rename d0/d1a/d28/d31/d33/l42 to d0/d1a/d28/d31/d4c/d80/d6f/l130 0 2024-04-24T08:25:11.577 INFO:tasks.workunit.client.1.smithi149.stdout:8/958: chown d1/d1f/f69 5243000 1 2024-04-24T08:25:11.584 INFO:tasks.workunit.client.0.smithi090.stdout:6/780: symlink d8/dc/d19/dd2/lf3 0 2024-04-24T08:25:11.586 INFO:tasks.workunit.client.1.smithi149.stdout:6/889: symlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/l131 0 2024-04-24T08:25:11.588 INFO:tasks.workunit.client.0.smithi090.stdout:5/813: write d8/da/fc [3402655,119610] 0 2024-04-24T08:25:11.592 INFO:tasks.workunit.client.1.smithi149.stdout:6/890: chown d0/d1a/d28/d31/d4c/d80/d6f/dc2 53161 1 2024-04-24T08:25:11.596 INFO:tasks.workunit.client.1.smithi149.stdout:6/891: chown d0/d1a/d28/d31/d4c/d80/d6f/d97 1 1 2024-04-24T08:25:11.603 INFO:tasks.workunit.client.0.smithi090.stdout:8/738: getdents d2/d62/d6b 0 2024-04-24T08:25:11.604 INFO:tasks.workunit.client.0.smithi090.stdout:9/759: dwrite d0/dc/d24/d7c/d4d/d32/f95 [0,4194304] 0 2024-04-24T08:25:11.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/723: creat d2/df/d1a/d37/ff1 x:0 0 0 2024-04-24T08:25:11.607 INFO:tasks.workunit.client.0.smithi090.stdout:0/745: link d1a/d4b/d53/d79/d35/d62/c9e d1a/d4b/d53/d79/d69/cfa 0 2024-04-24T08:25:11.610 INFO:tasks.workunit.client.0.smithi090.stdout:5/814: creat d8/da/d3a/d96/dcb/d21/d4c/f109 x:0 0 0 2024-04-24T08:25:11.611 INFO:tasks.workunit.client.0.smithi090.stdout:3/724: readlink d2/d25/d33/lb8 0 2024-04-24T08:25:11.615 INFO:tasks.workunit.client.0.smithi090.stdout:1/745: dwrite d3/d25/d30/d9d/fa8 [0,4194304] 0 2024-04-24T08:25:11.615 INFO:tasks.workunit.client.0.smithi090.stdout:4/702: truncate d3/d8e/ddf/d5c/db5/d69/d8b/db2/f60 8369257 0 2024-04-24T08:25:11.618 INFO:tasks.workunit.client.1.smithi149.stdout:4/980: dread d2/d10/d91/d9f/fc7 [0,4194304] 0 2024-04-24T08:25:11.623 INFO:tasks.workunit.client.0.smithi090.stdout:2/826: getdents d0/d7/d28 0 2024-04-24T08:25:11.624 INFO:tasks.workunit.client.1.smithi149.stdout:8/959: dread d1/d1f/d25/f34 [0,4194304] 0 2024-04-24T08:25:11.625 INFO:tasks.workunit.client.0.smithi090.stdout:6/781: dread d8/d1d/d2f/f88 [0,4194304] 0 2024-04-24T08:25:11.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/725: creat d2/df/d1a/d37/d53/ff2 x:0 0 0 2024-04-24T08:25:11.632 INFO:tasks.workunit.client.1.smithi149.stdout:6/892: dwrite d0/d1a/d28/d4a/f72 [0,4194304] 0 2024-04-24T08:25:11.632 INFO:tasks.workunit.client.1.smithi149.stdout:4/981: creat d2/d10/d1a/d34/f134 x:0 0 0 2024-04-24T08:25:11.633 INFO:tasks.workunit.client.1.smithi149.stdout:2/893: dread d0/d6/db/d28/f48 [0,4194304] 0 2024-04-24T08:25:11.636 INFO:tasks.workunit.client.1.smithi149.stdout:4/982: write d2/d10/d1a/fbe [404855,40755] 0 2024-04-24T08:25:11.638 INFO:tasks.workunit.client.1.smithi149.stdout:8/960: rename d1/d1f/d25/f34 to d1/de/d18/d91/d9f/f14d 0 2024-04-24T08:25:11.643 INFO:tasks.workunit.client.1.smithi149.stdout:2/894: dread - d0/d6/db/d28/db0/ff3 zero size 2024-04-24T08:25:11.648 INFO:tasks.workunit.client.0.smithi090.stdout:4/703: creat d3/d8e/ddf/d5c/db5/d69/d8b/fe3 x:0 0 0 2024-04-24T08:25:11.650 INFO:tasks.workunit.client.0.smithi090.stdout:4/704: write d3/d8e/ddf/d1d/f27 [1978533,46795] 0 2024-04-24T08:25:11.652 INFO:tasks.workunit.client.0.smithi090.stdout:4/705: chown d3/d24/d92 25 1 2024-04-24T08:25:11.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/815: sync 2024-04-24T08:25:11.656 INFO:tasks.workunit.client.1.smithi149.stdout:4/983: mkdir d2/d18/d1e/d3c/df0/d135 0 2024-04-24T08:25:11.657 INFO:tasks.workunit.client.1.smithi149.stdout:8/961: creat d1/d1f/d25/d111/f14e x:0 0 0 2024-04-24T08:25:11.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/746: symlink d1a/d4b/d53/d79/lfb 0 2024-04-24T08:25:11.659 INFO:tasks.workunit.client.0.smithi090.stdout:7/816: chown db/d17/d29/d38/d3e/fb8 96990426 1 2024-04-24T08:25:11.660 INFO:tasks.workunit.client.0.smithi090.stdout:6/782: symlink d8/dc/d19/d8e/d38/lf4 0 2024-04-24T08:25:11.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/739: dread d2/f12 [0,4194304] 0 2024-04-24T08:25:11.662 INFO:tasks.workunit.client.1.smithi149.stdout:4/984: stat d2/d18/d1e/c86 0 2024-04-24T08:25:11.663 INFO:tasks.workunit.client.0.smithi090.stdout:6/783: chown d8/d1d/d29/l69 19 1 2024-04-24T08:25:11.666 INFO:tasks.workunit.client.1.smithi149.stdout:8/962: mknod d1/de/d18/daf/c14f 0 2024-04-24T08:25:11.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/815: dread d8/d59/dc7/d7a/da1/ff5 [4194304,4194304] 0 2024-04-24T08:25:11.673 INFO:tasks.workunit.client.1.smithi149.stdout:2/895: getdents d0/d6/db/d27/d81/db4 0 2024-04-24T08:25:11.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/816: write d8/d59/dc7/d7a/d50/f79 [821634,32268] 0 2024-04-24T08:25:11.674 INFO:tasks.workunit.client.0.smithi090.stdout:4/706: rmdir d3/d8e/ddf/d5c/db5/d69/d8b/db2 39 2024-04-24T08:25:11.676 INFO:tasks.workunit.client.1.smithi149.stdout:6/893: write d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fe8 [480772,75107] 0 2024-04-24T08:25:11.677 INFO:tasks.workunit.client.0.smithi090.stdout:9/760: creat d0/dc/d24/d2d/d49/fed x:0 0 0 2024-04-24T08:25:11.679 INFO:tasks.workunit.client.1.smithi149.stdout:2/896: readlink d0/d6/db/d39/d3d/d44/l5d 0 2024-04-24T08:25:11.681 INFO:tasks.workunit.client.1.smithi149.stdout:6/894: write d0/d1a/d28/d4a/f72 [3497980,117783] 0 2024-04-24T08:25:11.684 INFO:tasks.workunit.client.0.smithi090.stdout:3/726: getdents d2/d15/dc4 0 2024-04-24T08:25:11.684 INFO:tasks.workunit.client.0.smithi090.stdout:3/727: readlink d2/d25/lb9 0 2024-04-24T08:25:11.685 INFO:tasks.workunit.client.0.smithi090.stdout:0/747: mknod d1a/d4b/d53/d79/d35/d8e/cfc 0 2024-04-24T08:25:11.685 INFO:tasks.workunit.client.0.smithi090.stdout:1/746: mkdir d3/d5/da/d97/d9c/dc4/dcc/d105 0 2024-04-24T08:25:11.686 INFO:tasks.workunit.client.0.smithi090.stdout:0/748: fsync d1a/d4b/d53/d57/fe0 0 2024-04-24T08:25:11.687 INFO:tasks.workunit.client.1.smithi149.stdout:2/897: mkdir d0/d6/db/d113 0 2024-04-24T08:25:11.688 INFO:tasks.workunit.client.0.smithi090.stdout:9/761: dread d0/dc/d24/d2d/f65 [4194304,4194304] 0 2024-04-24T08:25:11.688 INFO:tasks.workunit.client.1.smithi149.stdout:6/895: symlink d0/d1a/d28/d31/d4c/d4e/d12e/d54/l132 0 2024-04-24T08:25:11.689 INFO:tasks.workunit.client.0.smithi090.stdout:6/784: mknod d8/d1d/d1f/d3e/d4f/d7e/daa/cf5 0 2024-04-24T08:25:11.690 INFO:tasks.workunit.client.1.smithi149.stdout:4/985: dwrite d2/d10/d91/d9f/fc7 [0,4194304] 0 2024-04-24T08:25:11.692 INFO:tasks.workunit.client.1.smithi149.stdout:8/963: truncate d1/d1f/d22/f71 2989762 0 2024-04-24T08:25:11.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/827: creat d0/d7/d67/f115 x:0 0 0 2024-04-24T08:25:11.695 INFO:tasks.workunit.client.0.smithi090.stdout:7/817: symlink db/l118 0 2024-04-24T08:25:11.696 INFO:tasks.workunit.client.0.smithi090.stdout:2/828: chown d0/d14/d36/dce/dee/fd3 69750212 1 2024-04-24T08:25:11.696 INFO:tasks.workunit.client.0.smithi090.stdout:3/728: fdatasync d2/d15/fab 0 2024-04-24T08:25:11.696 INFO:tasks.workunit.client.1.smithi149.stdout:6/896: mkdir d0/d1a/d28/d31/d4c/d122/d133 0 2024-04-24T08:25:11.697 INFO:tasks.workunit.client.0.smithi090.stdout:7/818: read - db/dbd/f107 zero size 2024-04-24T08:25:11.698 INFO:tasks.workunit.client.1.smithi149.stdout:6/897: read d0/d1a/d28/d4a/f72 [1628136,73998] 0 2024-04-24T08:25:11.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/986: creat d2/d10/d91/d9f/f136 x:0 0 0 2024-04-24T08:25:11.699 INFO:tasks.workunit.client.1.smithi149.stdout:6/898: chown d0/l40 0 1 2024-04-24T08:25:11.700 INFO:tasks.workunit.client.0.smithi090.stdout:8/740: rename d2/da/d13/d64/db2/dbb/ldc to d2/da/d13/d64/db2/dbb/lfa 0 2024-04-24T08:25:11.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/729: write d2/d15/da6/fdc [258762,1469] 0 2024-04-24T08:25:11.702 INFO:tasks.workunit.client.0.smithi090.stdout:0/749: mknod d1a/d4b/d53/d79/d69/d8a/cfd 0 2024-04-24T08:25:11.702 INFO:tasks.workunit.client.0.smithi090.stdout:2/829: write d0/d14/f15 [3873358,63112] 0 2024-04-24T08:25:11.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/762: mknod d0/dc/d24/d7c/cee 0 2024-04-24T08:25:11.705 INFO:tasks.workunit.client.0.smithi090.stdout:1/747: dread d3/f8c [0,4194304] 0 2024-04-24T08:25:11.706 INFO:tasks.workunit.client.1.smithi149.stdout:4/987: rmdir d2/d24/d2e/d38 39 2024-04-24T08:25:11.708 INFO:tasks.workunit.client.0.smithi090.stdout:8/741: fsync d2/da/d13/d64/d50/f5d 0 2024-04-24T08:25:11.709 INFO:tasks.workunit.client.1.smithi149.stdout:2/898: getdents d0/d6/db/d28/d7f 0 2024-04-24T08:25:11.710 INFO:tasks.workunit.client.0.smithi090.stdout:9/763: write d0/dc/d24/d2d/f3f [4301873,90928] 0 2024-04-24T08:25:11.711 INFO:tasks.workunit.client.1.smithi149.stdout:6/899: fsync d0/f3 0 2024-04-24T08:25:11.712 INFO:tasks.workunit.client.0.smithi090.stdout:4/707: mknod d3/dd9/ce4 0 2024-04-24T08:25:11.713 INFO:tasks.workunit.client.1.smithi149.stdout:8/964: symlink d1/de/d18/d32/d65/d125/l150 0 2024-04-24T08:25:11.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/785: dwrite f6 [4194304,4194304] 0 2024-04-24T08:25:11.717 INFO:tasks.workunit.client.0.smithi090.stdout:9/764: read d0/f2 [1327453,49063] 0 2024-04-24T08:25:11.747 INFO:tasks.workunit.client.0.smithi090.stdout:0/750: unlink d1a/le8 0 2024-04-24T08:25:11.748 INFO:tasks.workunit.client.1.smithi149.stdout:8/965: dread d1/d1f/d22/f11a [4194304,4194304] 0 2024-04-24T08:25:11.756 INFO:tasks.workunit.client.0.smithi090.stdout:2/830: mkdir d0/d7/d57/d79/d116 0 2024-04-24T08:25:11.757 INFO:tasks.workunit.client.1.smithi149.stdout:6/900: chown d0/d1a/d28/d31/d4c/d4e/d62/d66/l70 6932 1 2024-04-24T08:25:11.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/748: fdatasync d3/d5/da/dc/d80/fbc 0 2024-04-24T08:25:11.768 INFO:tasks.workunit.client.0.smithi090.stdout:8/742: rmdir d2/da/d13/d60/d76 39 2024-04-24T08:25:11.770 INFO:tasks.workunit.client.0.smithi090.stdout:6/786: mkdir d8/dc/d19/d8e/d81/d96/df6 0 2024-04-24T08:25:11.774 INFO:tasks.workunit.client.0.smithi090.stdout:9/765: mkdir d0/dc/d24/d2d/d49/d62/def 0 2024-04-24T08:25:11.775 INFO:tasks.workunit.client.1.smithi149.stdout:6/901: mkdir d0/d1a/d28/d31/d4c/d4e/d12e/d134 0 2024-04-24T08:25:11.775 INFO:tasks.workunit.client.1.smithi149.stdout:2/899: dwrite d0/d6/db/d28/d7f/d8f/fcc [0,4194304] 0 2024-04-24T08:25:11.776 INFO:tasks.workunit.client.1.smithi149.stdout:4/988: write d2/d24/d2e/f4a [592740,75433] 0 2024-04-24T08:25:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:11 smithi149 ceph-mon[31282]: pgmap v28: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 163 MiB/s wr, 344 op/s 2024-04-24T08:25:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:11 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:25:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:11 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:25:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:11 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:25:11.778 INFO:tasks.workunit.client.1.smithi149.stdout:8/966: mkdir d1/d1f/d25/d151 0 2024-04-24T08:25:11.778 INFO:tasks.workunit.client.0.smithi090.stdout:0/751: rename d1a/d4b/d53/d79/d35/d68 to d1a/d4b/d53/d79/d35/dea/dfe 0 2024-04-24T08:25:11.780 INFO:tasks.workunit.client.1.smithi149.stdout:2/900: creat d0/d6/db/d27/f114 x:0 0 0 2024-04-24T08:25:11.781 INFO:tasks.workunit.client.0.smithi090.stdout:5/817: link d8/d25/f3d d8/d59/f10a 0 2024-04-24T08:25:11.781 INFO:tasks.workunit.client.0.smithi090.stdout:3/730: dread d2/d15/d36/d6f/d9d/fa4 [0,4194304] 0 2024-04-24T08:25:11.783 INFO:tasks.workunit.client.1.smithi149.stdout:4/989: fsync d2/d10/d1a/fda 0 2024-04-24T08:25:11.786 INFO:tasks.workunit.client.0.smithi090.stdout:1/749: creat d3/d25/f106 x:0 0 0 2024-04-24T08:25:11.790 INFO:tasks.workunit.client.0.smithi090.stdout:8/743: mkdir d2/da/d13/d60/d91/dfb 0 2024-04-24T08:25:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:11 smithi090 ceph-mon[25016]: pgmap v28: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 163 MiB/s wr, 344 op/s 2024-04-24T08:25:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:11 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:25:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:11 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:25:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:11 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:25:11.796 INFO:tasks.workunit.client.1.smithi149.stdout:4/990: write d2/d18/d1e/d3c/d48/d69/d7d/d8d/def/ffc [2691772,71348] 0 2024-04-24T08:25:11.801 INFO:tasks.workunit.client.0.smithi090.stdout:9/766: rename d0/dc/cc7 to d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96/cf0 0 2024-04-24T08:25:11.803 INFO:tasks.workunit.client.0.smithi090.stdout:5/818: creat d8/d76/dad/f10b x:0 0 0 2024-04-24T08:25:11.804 INFO:tasks.workunit.client.1.smithi149.stdout:8/967: dread d1/de/d18/f3e [0,4194304] 0 2024-04-24T08:25:11.805 INFO:tasks.workunit.client.1.smithi149.stdout:8/968: write d1/d1f/f28 [2695726,114710] 0 2024-04-24T08:25:11.807 INFO:tasks.workunit.client.0.smithi090.stdout:1/750: read - d3/d25/d30/d9d/da2/fad zero size 2024-04-24T08:25:11.810 INFO:tasks.workunit.client.0.smithi090.stdout:8/744: truncate d2/da/d13/d60/d76/fca 244422 0 2024-04-24T08:25:11.811 INFO:tasks.workunit.client.0.smithi090.stdout:2/831: creat d0/d14/d17/dfa/f117 x:0 0 0 2024-04-24T08:25:11.812 INFO:tasks.workunit.client.1.smithi149.stdout:6/902: write d0/d1a/d28/d31/d4c/d80/dbd/ff7 [1576452,83077] 0 2024-04-24T08:25:11.814 INFO:tasks.workunit.client.0.smithi090.stdout:0/752: symlink d1a/d4b/d53/d79/lff 0 2024-04-24T08:25:11.816 INFO:tasks.workunit.client.0.smithi090.stdout:3/731: rename d2/d15/d36/d6f/cbb to d2/d25/d2c/cf3 0 2024-04-24T08:25:11.816 INFO:tasks.workunit.client.0.smithi090.stdout:7/819: truncate db/d17/d29/d38/d3e/fb7 1477044 0 2024-04-24T08:25:11.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/903: mknod d0/d1a/d28/d31/d4c/d4e/d12e/d54/c135 0 2024-04-24T08:25:11.819 INFO:tasks.workunit.client.0.smithi090.stdout:5/819: creat d8/d56/f10c x:0 0 0 2024-04-24T08:25:11.820 INFO:tasks.workunit.client.0.smithi090.stdout:4/708: truncate d3/d8e/ddf/d1d/f40 2646977 0 2024-04-24T08:25:11.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/820: chown d8/da/d3a/d2d/d37/ce4 66636168 1 2024-04-24T08:25:11.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/787: dread d8/f46 [0,4194304] 0 2024-04-24T08:25:11.824 INFO:tasks.workunit.client.0.smithi090.stdout:8/745: truncate d2/da/d13/d64/d24/f79 5189778 0 2024-04-24T08:25:11.826 INFO:tasks.workunit.client.1.smithi149.stdout:2/901: dwrite d0/d6/db/d28/f46 [0,4194304] 0 2024-04-24T08:25:11.826 INFO:tasks.workunit.client.1.smithi149.stdout:6/904: creat d0/d1a/d28/d31/d4c/d4e/d12e/d54/f136 x:0 0 0 2024-04-24T08:25:11.827 INFO:tasks.workunit.client.0.smithi090.stdout:0/753: rmdir d1a/d4b/d53/d79/d69/d90/dcc 39 2024-04-24T08:25:11.828 INFO:tasks.workunit.client.1.smithi149.stdout:2/902: readlink d0/d6/db/d39/l63 0 2024-04-24T08:25:11.831 INFO:tasks.workunit.client.0.smithi090.stdout:9/767: creat d0/dc/d24/d2d/d49/d62/def/ff1 x:0 0 0 2024-04-24T08:25:11.832 INFO:tasks.workunit.client.0.smithi090.stdout:3/732: stat d2/d15/l1c 0 2024-04-24T08:25:11.833 INFO:tasks.workunit.client.0.smithi090.stdout:1/751: mknod d3/d5/da/dc/d80/deb/c107 0 2024-04-24T08:25:11.833 INFO:tasks.workunit.client.1.smithi149.stdout:6/905: rmdir d0/db7 39 2024-04-24T08:25:11.833 INFO:tasks.workunit.client.1.smithi149.stdout:8/969: write d1/de/d18/f48 [2633254,14816] 0 2024-04-24T08:25:11.836 INFO:tasks.workunit.client.0.smithi090.stdout:4/709: readlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/lc3 0 2024-04-24T08:25:11.841 INFO:tasks.workunit.client.0.smithi090.stdout:4/710: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f6d 1251 1 2024-04-24T08:25:11.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/746: mknod d2/da/d13/d64/db2/db6/cfc 0 2024-04-24T08:25:11.843 INFO:tasks.workunit.client.0.smithi090.stdout:6/788: dwrite d8/dc/d19/d8e/d81/d96/fbd [0,4194304] 0 2024-04-24T08:25:11.844 INFO:tasks.workunit.client.0.smithi090.stdout:8/747: fsync d2/da/d13/d64/d50/f5d 0 2024-04-24T08:25:11.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/789: readlink d8/dc/d19/d8e/d81/lee 0 2024-04-24T08:25:11.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/748: write d2/da/d13/d60/d76/fca [817775,15699] 0 2024-04-24T08:25:11.850 INFO:tasks.workunit.client.1.smithi149.stdout:6/906: creat d0/db7/d112/f137 x:0 0 0 2024-04-24T08:25:11.850 INFO:tasks.workunit.client.0.smithi090.stdout:7/820: unlink db/d17/c31 0 2024-04-24T08:25:11.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/733: creat d2/d25/d2c/ff4 x:0 0 0 2024-04-24T08:25:11.853 INFO:tasks.workunit.client.0.smithi090.stdout:7/821: read - db/d17/d29/d32/d5a/d75/f68 zero size 2024-04-24T08:25:11.854 INFO:tasks.workunit.client.0.smithi090.stdout:1/752: creat d3/d68/def/f108 x:0 0 0 2024-04-24T08:25:11.855 INFO:tasks.workunit.client.0.smithi090.stdout:5/821: mknod d8/d59/dc7/d7b/d106/c10d 0 2024-04-24T08:25:11.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/711: mknod d3/d8e/ce5 0 2024-04-24T08:25:11.862 INFO:tasks.workunit.client.0.smithi090.stdout:0/754: creat d1a/d4b/d53/d79/d58/de7/f100 x:0 0 0 2024-04-24T08:25:11.865 INFO:tasks.workunit.client.0.smithi090.stdout:2/832: getdents d0/d14/d36/d77 0 2024-04-24T08:25:11.867 INFO:tasks.workunit.client.1.smithi149.stdout:6/907: getdents d0/d1a/d28/d31/d4c/d4e/d62 0 2024-04-24T08:25:11.871 INFO:tasks.workunit.client.0.smithi090.stdout:1/753: mkdir d3/d1d/d34/d73/dd8/d109 0 2024-04-24T08:25:11.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/822: symlink d8/da/d3a/l10e 0 2024-04-24T08:25:11.874 INFO:tasks.workunit.client.1.smithi149.stdout:2/903: dread d0/d6/db/f10 [0,4194304] 0 2024-04-24T08:25:11.875 INFO:tasks.workunit.client.0.smithi090.stdout:0/755: mknod d1a/d4b/d53/d79/d69/d8a/c101 0 2024-04-24T08:25:11.878 INFO:tasks.workunit.client.0.smithi090.stdout:5/823: fsync d8/f10 0 2024-04-24T08:25:11.880 INFO:tasks.workunit.client.1.smithi149.stdout:2/904: chown d0/d6/db/f1f 310792568 1 2024-04-24T08:25:11.884 INFO:tasks.workunit.client.1.smithi149.stdout:2/905: mkdir d0/d6/db/d28/d115 0 2024-04-24T08:25:11.888 INFO:tasks.workunit.client.0.smithi090.stdout:6/790: dwrite d8/d1d/d1f/d3e/d4d/f99 [0,4194304] 0 2024-04-24T08:25:11.888 INFO:tasks.workunit.client.0.smithi090.stdout:5/824: chown d8/d59/dc7/d7a/da1/dae/fe6 943665 1 2024-04-24T08:25:11.889 INFO:tasks.workunit.client.0.smithi090.stdout:8/749: link d2/f9d d2/da/d13/d64/d24/db3/db4/dcd/de8/ffd 0 2024-04-24T08:25:11.892 INFO:tasks.workunit.client.1.smithi149.stdout:8/970: dread d1/d1f/d31/f67 [0,4194304] 0 2024-04-24T08:25:11.893 INFO:tasks.workunit.client.0.smithi090.stdout:6/791: unlink d8/f46 0 2024-04-24T08:25:11.894 INFO:tasks.workunit.client.0.smithi090.stdout:5/825: mkdir d8/d59/dc7/d7a/d50/d9f/d10f 0 2024-04-24T08:25:11.897 INFO:tasks.workunit.client.0.smithi090.stdout:7/822: dwrite db/dbd/f110 [0,4194304] 0 2024-04-24T08:25:11.897 INFO:tasks.workunit.client.0.smithi090.stdout:8/750: mkdir d2/da/d13/d64/db2/dfe 0 2024-04-24T08:25:11.899 INFO:tasks.workunit.client.0.smithi090.stdout:6/792: dread - d8/d1d/d1f/d3e/d44/fde zero size 2024-04-24T08:25:11.900 INFO:tasks.workunit.client.1.smithi149.stdout:8/971: mknod d1/de/d18/d32/d65/d125/c152 0 2024-04-24T08:25:11.906 INFO:tasks.workunit.client.0.smithi090.stdout:6/793: rmdir d8/d1d/d29/df0 39 2024-04-24T08:25:11.907 INFO:tasks.workunit.client.1.smithi149.stdout:8/972: creat d1/d1f/d25/d151/f153 x:0 0 0 2024-04-24T08:25:11.911 INFO:tasks.workunit.client.0.smithi090.stdout:7/823: creat db/d17/d29/d32/d5a/f119 x:0 0 0 2024-04-24T08:25:11.912 INFO:tasks.workunit.client.0.smithi090.stdout:4/712: dread d3/f32 [0,4194304] 0 2024-04-24T08:25:11.913 INFO:tasks.workunit.client.0.smithi090.stdout:6/794: creat d8/dc/d19/d8e/d81/d96/d97/ff7 x:0 0 0 2024-04-24T08:25:11.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/973: symlink d1/dce/d13e/l154 0 2024-04-24T08:25:11.916 INFO:tasks.workunit.client.0.smithi090.stdout:6/795: read - d8/d1d/d2f/d85/fbc zero size 2024-04-24T08:25:11.916 INFO:tasks.workunit.client.0.smithi090.stdout:7/824: mknod db/d17/d39/c11a 0 2024-04-24T08:25:11.919 INFO:tasks.workunit.client.0.smithi090.stdout:8/751: dread d2/da/d13/d64/db2/fd7 [0,4194304] 0 2024-04-24T08:25:11.920 INFO:tasks.workunit.client.0.smithi090.stdout:7/825: unlink db/d54/d78/lbc 0 2024-04-24T08:25:11.921 INFO:tasks.workunit.client.0.smithi090.stdout:7/826: stat db/d17/fe8 0 2024-04-24T08:25:11.926 INFO:tasks.workunit.client.0.smithi090.stdout:7/827: write db/d17/d39/d86/dae/fb0 [2361379,115535] 0 2024-04-24T08:25:11.928 INFO:tasks.workunit.client.0.smithi090.stdout:7/828: stat db/d17/d39/d59 0 2024-04-24T08:25:11.933 INFO:tasks.workunit.client.0.smithi090.stdout:6/796: dwrite d8/d1d/d1f/d3e/d4f/d7e/fbe [0,4194304] 0 2024-04-24T08:25:11.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/974: dwrite d1/d1f/d31/fc3 [0,4194304] 0 2024-04-24T08:25:11.936 INFO:tasks.workunit.client.0.smithi090.stdout:5/826: dread d8/da/f20 [0,4194304] 0 2024-04-24T08:25:11.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/797: chown d8/dc/c18 529077 1 2024-04-24T08:25:11.937 INFO:tasks.workunit.client.0.smithi090.stdout:8/752: dwrite d2/d62/d71/fc9 [0,4194304] 0 2024-04-24T08:25:11.937 INFO:tasks.workunit.client.0.smithi090.stdout:7/829: truncate db/d17/d29/f69 1305909 0 2024-04-24T08:25:11.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/830: creat db/d17/d39/d59/f11b x:0 0 0 2024-04-24T08:25:11.946 INFO:tasks.workunit.client.0.smithi090.stdout:6/798: creat d8/d1d/d1f/d3e/d8f/ff8 x:0 0 0 2024-04-24T08:25:11.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/753: symlink d2/da/d13/d64/d50/lff 0 2024-04-24T08:25:11.949 INFO:tasks.workunit.client.0.smithi090.stdout:7/831: creat db/d17/d39/d59/f11c x:0 0 0 2024-04-24T08:25:11.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/799: dread d8/dc/d19/d8e/f7b [0,4194304] 0 2024-04-24T08:25:11.968 INFO:tasks.workunit.client.0.smithi090.stdout:8/754: dwrite d2/da/d13/d64/db2/dbb/fc0 [0,4194304] 0 2024-04-24T08:25:11.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/832: dwrite db/dbd/f116 [0,4194304] 0 2024-04-24T08:25:11.972 INFO:tasks.workunit.client.0.smithi090.stdout:8/755: write d2/da/d13/d64/d24/f5b [995381,23221] 0 2024-04-24T08:25:11.975 INFO:tasks.workunit.client.0.smithi090.stdout:7/833: stat db/d17/d39/d59/ff0 0 2024-04-24T08:25:11.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/756: dread d1a/d4b/f85 [0,4194304] 0 2024-04-24T08:25:11.984 INFO:tasks.workunit.client.0.smithi090.stdout:7/834: mknod db/d17/d29/d32/dd4/c11d 0 2024-04-24T08:25:11.984 INFO:tasks.workunit.client.0.smithi090.stdout:0/757: rmdir d1a/d4b/d53/d79/d35/dea/dfe 39 2024-04-24T08:25:11.986 INFO:tasks.workunit.client.0.smithi090.stdout:8/756: getdents d2/da/d13/d64/dcc 0 2024-04-24T08:25:11.989 INFO:tasks.workunit.client.0.smithi090.stdout:0/758: symlink d1a/d4b/d53/d57/l102 0 2024-04-24T08:25:11.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/759: chown d1a/d4b/d53/d79/d69/d78/d9b/cd9 2 1 2024-04-24T08:25:11.996 INFO:tasks.workunit.client.0.smithi090.stdout:3/734: write d2/df/d1a/d37/f47 [788542,79850] 0 2024-04-24T08:25:11.999 INFO:tasks.workunit.client.0.smithi090.stdout:9/768: truncate d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf 2510899 0 2024-04-24T08:25:12.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/833: write d0/d7/d67/d7f/d31/d46/d60/f74 [1711069,29962] 0 2024-04-24T08:25:12.004 INFO:tasks.workunit.client.0.smithi090.stdout:9/769: mknod d0/dc/d24/d7c/d4d/d8c/de2/cf2 0 2024-04-24T08:25:12.013 INFO:tasks.workunit.client.0.smithi090.stdout:7/835: dread db/d17/d29/d38/d3e/fb7 [0,4194304] 0 2024-04-24T08:25:12.014 INFO:tasks.workunit.client.0.smithi090.stdout:1/754: dwrite d3/d25/d30/f79 [0,4194304] 0 2024-04-24T08:25:12.017 INFO:tasks.workunit.client.1.smithi149.stdout:8/975: write d1/de/d18/d32/d63/d9e/fd6 [1339765,121685] 0 2024-04-24T08:25:12.018 INFO:tasks.workunit.client.0.smithi090.stdout:0/760: dwrite d1a/d4b/d53/d79/d58/db4/fb5 [0,4194304] 0 2024-04-24T08:25:12.021 INFO:tasks.workunit.client.0.smithi090.stdout:5/827: write d8/da/d14/d2f/d81/da6/fcc [359582,90898] 0 2024-04-24T08:25:12.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/976: mkdir d1/de/d18/d32/d65/d155 0 2024-04-24T08:25:12.027 INFO:tasks.workunit.client.0.smithi090.stdout:1/755: mknod d3/d68/c10a 0 2024-04-24T08:25:12.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/836: fsync db/d54/f108 0 2024-04-24T08:25:12.035 INFO:tasks.workunit.client.0.smithi090.stdout:0/761: unlink d1a/d4b/d53/d79/c28 0 2024-04-24T08:25:12.037 INFO:tasks.workunit.client.0.smithi090.stdout:1/756: write d3/d1d/d34/d5a/fec [679012,37869] 0 2024-04-24T08:25:12.044 INFO:tasks.workunit.client.0.smithi090.stdout:4/713: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/f7e [0,4194304] 0 2024-04-24T08:25:12.044 INFO:tasks.workunit.client.0.smithi090.stdout:6/800: dwrite d8/d1d/d1f/d3e/d4f/d7e/fbe [4194304,4194304] 0 2024-04-24T08:25:12.044 INFO:tasks.workunit.client.0.smithi090.stdout:9/770: dwrite d0/d2a/fb8 [4194304,4194304] 0 2024-04-24T08:25:12.046 INFO:tasks.workunit.client.0.smithi090.stdout:7/837: chown db/d17/d29/d32/d5a/fe6 813925 1 2024-04-24T08:25:12.049 INFO:tasks.workunit.client.0.smithi090.stdout:0/762: chown d1a/d4b/d53/d79/d35/d8e/fec 576109631 1 2024-04-24T08:25:12.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/735: dwrite d2/f14 [0,4194304] 0 2024-04-24T08:25:12.055 INFO:tasks.workunit.client.0.smithi090.stdout:0/763: chown d1a/d4b/d53/d57 78305 1 2024-04-24T08:25:12.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/757: fsync d3/d1d/d34/d5a/f8b 0 2024-04-24T08:25:12.056 INFO:tasks.workunit.client.1.smithi149.stdout:8/977: write d1/de/d18/d32/d63/d76/f9a [1442535,6479] 0 2024-04-24T08:25:12.060 INFO:tasks.workunit.client.0.smithi090.stdout:0/764: dread - d1a/d4b/d53/d79/d58/fd3 zero size 2024-04-24T08:25:12.060 INFO:tasks.workunit.client.0.smithi090.stdout:6/801: mkdir d8/d1d/d1f/d3e/d44/df9 0 2024-04-24T08:25:12.060 INFO:tasks.workunit.client.0.smithi090.stdout:4/714: unlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f7d 0 2024-04-24T08:25:12.065 INFO:tasks.workunit.client.0.smithi090.stdout:0/765: chown d1a/d4b/d53/d57 12455316 1 2024-04-24T08:25:12.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/838: unlink db/d17/d29/d32/d5a/d75/d101/f10d 0 2024-04-24T08:25:12.066 INFO:tasks.workunit.client.0.smithi090.stdout:2/834: dread d0/d14/d17/f18 [0,4194304] 0 2024-04-24T08:25:12.067 INFO:tasks.workunit.client.0.smithi090.stdout:7/839: chown db/f15 12 1 2024-04-24T08:25:12.068 INFO:tasks.workunit.client.0.smithi090.stdout:3/736: symlink d2/d15/d36/d6f/d92/lf5 0 2024-04-24T08:25:12.068 INFO:tasks.workunit.client.1.smithi149.stdout:8/978: symlink d1/de/d18/d43/d10e/d119/l156 0 2024-04-24T08:25:12.069 INFO:tasks.workunit.client.0.smithi090.stdout:3/737: readlink d2/d15/l4d 0 2024-04-24T08:25:12.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/840: fdatasync db/d54/d83/f8b 0 2024-04-24T08:25:12.071 INFO:tasks.workunit.client.0.smithi090.stdout:1/758: mkdir d3/d5/da/d94/d10b 0 2024-04-24T08:25:12.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/828: write d8/d59/dc7/d7a/d50/f57 [2819688,114139] 0 2024-04-24T08:25:12.076 INFO:tasks.workunit.client.0.smithi090.stdout:0/766: creat d1a/d4b/d53/d79/d69/d90/dbc/f103 x:0 0 0 2024-04-24T08:25:12.079 INFO:tasks.workunit.client.0.smithi090.stdout:5/829: truncate d8/d56/f10c 399654 0 2024-04-24T08:25:12.079 INFO:tasks.workunit.client.1.smithi149.stdout:8/979: mkdir d1/de/d18/d43/d157 0 2024-04-24T08:25:12.081 INFO:tasks.workunit.client.1.smithi149.stdout:8/980: dread - d1/de/d18/d32/dde/f11b zero size 2024-04-24T08:25:12.083 INFO:tasks.workunit.client.0.smithi090.stdout:6/802: dwrite d8/dc/d19/d8e/d38/fd7 [0,4194304] 0 2024-04-24T08:25:12.089 INFO:tasks.workunit.client.0.smithi090.stdout:3/738: creat d2/df/d1a/d37/dc0/ff6 x:0 0 0 2024-04-24T08:25:12.089 INFO:tasks.workunit.client.0.smithi090.stdout:1/759: symlink d3/d5/da/dc/d80/l10c 0 2024-04-24T08:25:12.089 INFO:tasks.workunit.client.0.smithi090.stdout:2/835: mknod d0/d14/d17/dc2/dff/c118 0 2024-04-24T08:25:12.091 INFO:tasks.workunit.client.0.smithi090.stdout:1/760: write d3/d5/da/dc/fe3 [775860,62174] 0 2024-04-24T08:25:12.093 INFO:tasks.workunit.client.0.smithi090.stdout:9/771: link d0/fb d0/dc/d24/d7c/d4d/d32/ff3 0 2024-04-24T08:25:12.093 INFO:tasks.workunit.client.0.smithi090.stdout:1/761: chown d3/d1d/d34/d73/dd8 5 1 2024-04-24T08:25:12.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/762: chown d3/d5/d3e/l8e 8773 1 2024-04-24T08:25:12.096 INFO:tasks.workunit.client.0.smithi090.stdout:3/739: mknod d2/d90/cf7 0 2024-04-24T08:25:12.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/836: chown d0/d7/l2f 6970 1 2024-04-24T08:25:12.099 INFO:tasks.workunit.client.0.smithi090.stdout:7/841: rmdir db/d17/d29/d32/d5a/d75/d101/d111 0 2024-04-24T08:25:12.103 INFO:tasks.workunit.client.0.smithi090.stdout:3/740: rmdir d2/d15/da6 39 2024-04-24T08:25:12.104 INFO:tasks.workunit.client.0.smithi090.stdout:3/741: chown d2/ld 3607441 1 2024-04-24T08:25:12.106 INFO:tasks.workunit.client.0.smithi090.stdout:3/742: write d2/df/d1a/d37/d53/fe7 [831871,42773] 0 2024-04-24T08:25:12.110 INFO:tasks.workunit.client.0.smithi090.stdout:7/842: mknod db/d17/d39/d86/dae/c11e 0 2024-04-24T08:25:12.112 INFO:tasks.workunit.client.0.smithi090.stdout:0/767: link d1a/d4b/d53/d79/d69/d78/d9b/ff9 d1a/d4b/d53/d79/d58/d96/f104 0 2024-04-24T08:25:12.113 INFO:tasks.workunit.client.1.smithi149.stdout:8/981: dwrite d1/de/d18/d32/dde/f11b [0,4194304] 0 2024-04-24T08:25:12.114 INFO:tasks.workunit.client.0.smithi090.stdout:3/743: dread - d2/d84/fa3 zero size 2024-04-24T08:25:12.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/803: link d8/d1d/f7c d8/d1d/d2f/ffa 0 2024-04-24T08:25:12.119 INFO:tasks.workunit.client.0.smithi090.stdout:9/772: creat d0/dc/d24/d2d/d49/ff4 x:0 0 0 2024-04-24T08:25:12.120 INFO:tasks.workunit.client.0.smithi090.stdout:1/763: creat d3/d5/da/d94/f10d x:0 0 0 2024-04-24T08:25:12.123 INFO:tasks.workunit.client.0.smithi090.stdout:4/715: truncate d3/d8e/ddf/d5c/f7f 2554885 0 2024-04-24T08:25:12.124 INFO:tasks.workunit.client.0.smithi090.stdout:2/837: creat d0/d2e/f119 x:0 0 0 2024-04-24T08:25:12.124 INFO:tasks.workunit.client.0.smithi090.stdout:5/830: dread d8/d99/f9d [0,4194304] 0 2024-04-24T08:25:12.125 INFO:tasks.workunit.client.0.smithi090.stdout:4/716: readlink d3/d8e/ddf/d5c/db5/d77/ld3 0 2024-04-24T08:25:12.126 INFO:tasks.workunit.client.0.smithi090.stdout:2/838: readlink d0/d7/d28/daa/def/d50/lb2 0 2024-04-24T08:25:12.129 INFO:tasks.workunit.client.1.smithi149.stdout:8/982: dread d1/de/d18/d32/d63/d9e/dec/f121 [0,4194304] 0 2024-04-24T08:25:12.129 INFO:tasks.workunit.client.0.smithi090.stdout:2/839: fdatasync d0/d7/d67/d7f/d31/d46/d60/f74 0 2024-04-24T08:25:12.133 INFO:tasks.workunit.client.0.smithi090.stdout:7/843: rename db/d17/d39/d86 to db/d17/d29/d32/d5a/d75/d11f 0 2024-04-24T08:25:12.134 INFO:tasks.workunit.client.1.smithi149.stdout:8/983: rmdir d1/de/d18/d43 39 2024-04-24T08:25:12.138 INFO:tasks.workunit.client.1.smithi149.stdout:8/984: dread - d1/de/d18/d32/d65/dad/f126 zero size 2024-04-24T08:25:12.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/768: dwrite d1a/fef [0,4194304] 0 2024-04-24T08:25:12.141 INFO:tasks.workunit.client.1.smithi149.stdout:8/985: mknod d1/de/d18/d32/dde/c158 0 2024-04-24T08:25:12.142 INFO:tasks.workunit.client.0.smithi090.stdout:9/773: dread d0/dc/d24/d2d/d49/fb3 [0,4194304] 0 2024-04-24T08:25:12.142 INFO:tasks.workunit.client.0.smithi090.stdout:3/744: truncate d2/df/d1a/d37/d59/f61 887822 0 2024-04-24T08:25:12.145 INFO:tasks.workunit.client.0.smithi090.stdout:3/745: dread - d2/df/d1a/d37/d53/ff2 zero size 2024-04-24T08:25:12.145 INFO:tasks.workunit.client.0.smithi090.stdout:0/769: truncate d1a/f3b 592562 0 2024-04-24T08:25:12.154 INFO:tasks.workunit.client.0.smithi090.stdout:3/746: unlink d2/df/d5b/f8e 0 2024-04-24T08:25:12.155 INFO:tasks.workunit.client.0.smithi090.stdout:0/770: symlink d1a/d4b/d53/d79/d35/db8/l105 0 2024-04-24T08:25:12.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/774: link d0/dc/d24/c6d d0/dc/d24/d7c/d4d/d8c/cf5 0 2024-04-24T08:25:12.157 INFO:tasks.workunit.client.1.smithi149.stdout:8/986: dwrite d1/de/d18/d32/d63/d9e/fd6 [0,4194304] 0 2024-04-24T08:25:12.158 INFO:tasks.workunit.client.0.smithi090.stdout:5/831: dread d8/d76/dad/ff6 [0,4194304] 0 2024-04-24T08:25:12.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/717: dwrite d3/f9 [4194304,4194304] 0 2024-04-24T08:25:12.164 INFO:tasks.workunit.client.0.smithi090.stdout:3/747: fdatasync d2/df/d5b/f63 0 2024-04-24T08:25:12.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/771: symlink d1a/d4b/d53/d79/d35/dbd/l106 0 2024-04-24T08:25:12.166 INFO:tasks.workunit.client.1.smithi149.stdout:8/987: mkdir d1/de/d18/d32/d65/d159 0 2024-04-24T08:25:12.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/844: dwrite db/d17/d39/f10b [0,4194304] 0 2024-04-24T08:25:12.174 INFO:tasks.workunit.client.1.smithi149.stdout:8/988: rename d1/de/d18/d32/d63/d9e/ce6 to d1/de/d18/d32/d65/d8b/c15a 0 2024-04-24T08:25:12.178 INFO:tasks.workunit.client.0.smithi090.stdout:5/832: unlink d8/d59/dc7/d7b/d106/c10d 0 2024-04-24T08:25:12.179 INFO:tasks.workunit.client.0.smithi090.stdout:0/772: rmdir d1a/d4b/d53/d79/d69/d90/df2 39 2024-04-24T08:25:12.184 INFO:tasks.workunit.client.0.smithi090.stdout:4/718: symlink d3/d8e/ddf/d5c/db5/d7c/le6 0 2024-04-24T08:25:12.184 INFO:tasks.workunit.client.0.smithi090.stdout:2/840: dread d0/d7/d28/daa/def/db4/dcd/f110 [0,4194304] 0 2024-04-24T08:25:12.184 INFO:tasks.workunit.client.0.smithi090.stdout:9/775: write d0/d19/f28 [1669279,50187] 0 2024-04-24T08:25:12.185 INFO:tasks.workunit.client.0.smithi090.stdout:0/773: write d1a/d4b/d53/d79/d69/d8a/fd6 [314873,78121] 0 2024-04-24T08:25:12.187 INFO:tasks.workunit.client.0.smithi090.stdout:2/841: dread - d0/d14/d36/ff7 zero size 2024-04-24T08:25:12.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/719: truncate d3/d8e/fb7 941756 0 2024-04-24T08:25:12.189 INFO:tasks.workunit.client.0.smithi090.stdout:7/845: link db/d17/d29/d32/d5a/d75/f10c db/d17/d29/d38/d99/f120 0 2024-04-24T08:25:12.189 INFO:tasks.workunit.client.0.smithi090.stdout:5/833: rename d8/da/d14/f69 to d8/d7c/da9/f110 0 2024-04-24T08:25:12.193 INFO:tasks.workunit.client.0.smithi090.stdout:9/776: creat d0/dc/d24/d7c/d4d/ff6 x:0 0 0 2024-04-24T08:25:12.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/774: symlink d1a/d4b/d53/d79/d69/d90/dcc/l107 0 2024-04-24T08:25:12.197 INFO:tasks.workunit.client.0.smithi090.stdout:5/834: fdatasync d8/da/d3a/f55 0 2024-04-24T08:25:12.199 INFO:tasks.workunit.client.0.smithi090.stdout:9/777: mkdir d0/d19/df7 0 2024-04-24T08:25:12.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/748: dwrite d2/df/d1a/f2b [8388608,4194304] 0 2024-04-24T08:25:12.211 INFO:tasks.workunit.client.0.smithi090.stdout:4/720: rename d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f8c to d3/d8e/ddf/d5c/db5/d69/fe7 0 2024-04-24T08:25:12.214 INFO:tasks.workunit.client.0.smithi090.stdout:2/842: dwrite d0/d7/de1/f114 [0,4194304] 0 2024-04-24T08:25:12.215 INFO:tasks.workunit.client.0.smithi090.stdout:0/775: write d1a/f1e [3542507,122480] 0 2024-04-24T08:25:12.218 INFO:tasks.workunit.client.0.smithi090.stdout:9/778: mknod d0/dc/d24/d7c/d4d/d32/dab/cf8 0 2024-04-24T08:25:12.219 INFO:tasks.workunit.client.0.smithi090.stdout:3/749: mkdir d2/d15/df8 0 2024-04-24T08:25:12.225 INFO:tasks.workunit.client.0.smithi090.stdout:7/846: dread db/dab/fd8 [0,4194304] 0 2024-04-24T08:25:12.226 INFO:tasks.workunit.client.0.smithi090.stdout:4/721: link d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/fb4 d3/d8e/ddf/d5c/db5/db6/fe8 0 2024-04-24T08:25:12.227 INFO:tasks.workunit.client.0.smithi090.stdout:0/776: creat d1a/d4b/d53/d79/d35/dea/dfe/f108 x:0 0 0 2024-04-24T08:25:12.230 INFO:tasks.workunit.client.0.smithi090.stdout:9/779: symlink d0/dc/d24/d7c/d4d/d32/d9f/d72/de9/lf9 0 2024-04-24T08:25:12.232 INFO:tasks.workunit.client.0.smithi090.stdout:0/777: creat d1a/d4b/d53/d79/d58/db4/f109 x:0 0 0 2024-04-24T08:25:12.233 INFO:tasks.workunit.client.0.smithi090.stdout:4/722: symlink d3/d8e/ddf/d5c/db5/d7c/le9 0 2024-04-24T08:25:12.238 INFO:tasks.workunit.client.0.smithi090.stdout:9/780: mkdir d0/dc/d24/d2d/d49/dfa 0 2024-04-24T08:25:12.240 INFO:tasks.workunit.client.0.smithi090.stdout:4/723: rename d3/d8e/ddf/d5c/db5/d69/d8b/fac to d3/d8e/ddf/d1d/dcb/fea 0 2024-04-24T08:25:12.245 INFO:tasks.workunit.client.0.smithi090.stdout:9/781: creat d0/dc/d24/d2d/d49/d62/dcb/dcd/ffb x:0 0 0 2024-04-24T08:25:12.246 INFO:tasks.workunit.client.0.smithi090.stdout:5/835: dread d8/d59/fdf [0,4194304] 0 2024-04-24T08:25:12.258 INFO:tasks.workunit.client.0.smithi090.stdout:7/847: dwrite db/d17/d29/d32/d5a/d75/d11f/dae/fc7 [4194304,4194304] 0 2024-04-24T08:25:12.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/750: dwrite d2/df/d1a/d37/d59/f7b [0,4194304] 0 2024-04-24T08:25:12.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/724: dwrite d3/d8e/ddf/d5c/db5/f98 [0,4194304] 0 2024-04-24T08:25:12.266 INFO:tasks.workunit.client.0.smithi090.stdout:3/751: chown d2/d15/l88 1272 1 2024-04-24T08:25:12.269 INFO:tasks.workunit.client.0.smithi090.stdout:0/778: dwrite d1a/d4b/d53/d79/d3c/f54 [0,4194304] 0 2024-04-24T08:25:12.271 INFO:tasks.workunit.client.0.smithi090.stdout:3/752: creat d2/df/d1a/d37/d59/d98/ff9 x:0 0 0 2024-04-24T08:25:12.271 INFO:tasks.workunit.client.0.smithi090.stdout:4/725: mknod d3/d8e/ddf/ceb 0 2024-04-24T08:25:12.277 INFO:tasks.workunit.client.0.smithi090.stdout:3/753: mkdir d2/df/d1a/d37/dc0/de6/dfa 0 2024-04-24T08:25:12.278 INFO:tasks.workunit.client.0.smithi090.stdout:0/779: read d1a/d4b/f61 [2051185,61115] 0 2024-04-24T08:25:12.278 INFO:tasks.workunit.client.0.smithi090.stdout:8/757: sync 2024-04-24T08:25:12.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/836: dwrite d8/d59/dc7/d7a/d50/f8a [0,4194304] 0 2024-04-24T08:25:12.284 INFO:tasks.workunit.client.0.smithi090.stdout:4/726: read d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f62 [1170770,75019] 0 2024-04-24T08:25:12.286 INFO:tasks.workunit.client.0.smithi090.stdout:0/780: read d1a/d4b/d53/d79/d58/db4/fed [751077,82680] 0 2024-04-24T08:25:12.288 INFO:tasks.workunit.client.0.smithi090.stdout:7/848: dwrite db/d17/d29/d32/d5a/f8d [0,4194304] 0 2024-04-24T08:25:12.290 INFO:tasks.workunit.client.0.smithi090.stdout:3/754: creat d2/df/d1a/d37/d59/d87/ffb x:0 0 0 2024-04-24T08:25:12.291 INFO:tasks.workunit.client.0.smithi090.stdout:3/755: chown d2/d15/d36/d6f/d7a/f8f 48224563 1 2024-04-24T08:25:12.295 INFO:tasks.workunit.client.0.smithi090.stdout:4/727: mkdir d3/d8e/ddf/d5c/db5/d77/dec 0 2024-04-24T08:25:12.300 INFO:tasks.workunit.client.0.smithi090.stdout:8/758: creat d2/d62/d83/f100 x:0 0 0 2024-04-24T08:25:12.302 INFO:tasks.workunit.client.0.smithi090.stdout:8/759: fsync d2/da/d13/d60/fb1 0 2024-04-24T08:25:12.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/837: dread d8/da/d14/d2f/d44/f97 [0,4194304] 0 2024-04-24T08:25:12.312 INFO:tasks.workunit.client.0.smithi090.stdout:5/838: creat d8/da/d3a/d96/f111 x:0 0 0 2024-04-24T08:25:12.314 INFO:tasks.workunit.client.0.smithi090.stdout:5/839: stat d8/da/d3a/d96/dcb/c38 0 2024-04-24T08:25:12.317 INFO:tasks.workunit.client.0.smithi090.stdout:5/840: chown d8/d59/dc7/d7a/d50/f57 8 1 2024-04-24T08:25:12.319 INFO:tasks.workunit.client.0.smithi090.stdout:3/756: dread d2/df/db5/fd6 [0,4194304] 0 2024-04-24T08:25:12.322 INFO:tasks.workunit.client.0.smithi090.stdout:3/757: write d2/f24 [4492682,95145] 0 2024-04-24T08:25:12.324 INFO:tasks.workunit.client.0.smithi090.stdout:3/758: stat d2/df/d1a/d37/dc0/de6/dfa 0 2024-04-24T08:25:12.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/728: dread d3/d8e/ddf/d5c/f3d [0,4194304] 0 2024-04-24T08:25:12.327 INFO:tasks.workunit.client.0.smithi090.stdout:8/760: truncate d2/da/d13/d64/f3f 2433607 0 2024-04-24T08:25:12.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/759: rmdir d2/d25/d33 39 2024-04-24T08:25:12.337 INFO:tasks.workunit.client.0.smithi090.stdout:8/761: creat d2/da/d13/d64/db2/dfe/f101 x:0 0 0 2024-04-24T08:25:12.339 INFO:tasks.workunit.client.0.smithi090.stdout:0/781: dwrite d1a/d4b/d53/d79/fb6 [0,4194304] 0 2024-04-24T08:25:12.342 INFO:tasks.workunit.client.0.smithi090.stdout:7/849: dwrite db/f1a [0,4194304] 0 2024-04-24T08:25:12.342 INFO:tasks.workunit.client.0.smithi090.stdout:3/760: link d2/d15/fbf d2/df/d21/ffc 0 2024-04-24T08:25:12.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/841: dwrite d8/da/d3a/d96/dcb/d21/d4c/f109 [0,4194304] 0 2024-04-24T08:25:12.350 INFO:tasks.workunit.client.0.smithi090.stdout:0/782: dread - d1a/d4b/d53/d79/d58/fa5 zero size 2024-04-24T08:25:12.353 INFO:tasks.workunit.client.0.smithi090.stdout:0/783: stat d1a/d4b/d53/d79/d69/d8a/c94 0 2024-04-24T08:25:12.355 INFO:tasks.workunit.client.0.smithi090.stdout:3/761: mkdir d2/df/d1a/d37/d59/d98/dfd 0 2024-04-24T08:25:12.357 INFO:tasks.workunit.client.0.smithi090.stdout:4/729: dwrite d3/d8e/ddf/d5c/db5/d7c/fa9 [0,4194304] 0 2024-04-24T08:25:12.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/850: fdatasync db/d17/d29/d32/d5a/d75/f79 0 2024-04-24T08:25:12.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/730: chown d3/d24/l5f 863548 1 2024-04-24T08:25:12.361 INFO:tasks.workunit.client.0.smithi090.stdout:7/851: chown db/d17/d29/d38/d3e/d46/f103 2 1 2024-04-24T08:25:12.363 INFO:tasks.workunit.client.0.smithi090.stdout:3/762: creat d2/df/d1a/d37/dc0/de6/dfa/ffe x:0 0 0 2024-04-24T08:25:12.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/731: unlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/c9c 0 2024-04-24T08:25:12.372 INFO:tasks.workunit.client.0.smithi090.stdout:5/842: rename d8/da/d3a/d96 to d8/da/d14/d2f/d112 0 2024-04-24T08:25:12.374 INFO:tasks.workunit.client.0.smithi090.stdout:3/763: mknod d2/d84/cff 0 2024-04-24T08:25:12.376 INFO:tasks.workunit.client.0.smithi090.stdout:4/732: creat d3/d8e/fed x:0 0 0 2024-04-24T08:25:12.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/852: dwrite db/d17/d39/f10b [4194304,4194304] 0 2024-04-24T08:25:12.378 INFO:tasks.workunit.client.0.smithi090.stdout:3/764: write d2/df/fa5 [623828,124294] 0 2024-04-24T08:25:12.381 INFO:tasks.workunit.client.0.smithi090.stdout:7/853: chown db/f10 30 1 2024-04-24T08:25:12.393 INFO:tasks.workunit.client.0.smithi090.stdout:6/804: sync 2024-04-24T08:25:12.395 INFO:tasks.workunit.client.0.smithi090.stdout:6/805: fdatasync d8/d1d/d1f/d3e/d44/d90/da2/fd8 0 2024-04-24T08:25:12.399 INFO:tasks.workunit.client.0.smithi090.stdout:5/843: write d8/da/d14/d2f/d112/dcb/fa5 [1107804,22770] 0 2024-04-24T08:25:12.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/733: symlink d3/dd0/lee 0 2024-04-24T08:25:12.410 INFO:tasks.workunit.client.0.smithi090.stdout:8/762: unlink d2/da/d13/d64/db2/fd9 0 2024-04-24T08:25:12.411 INFO:tasks.workunit.client.0.smithi090.stdout:0/784: dread f18 [0,4194304] 0 2024-04-24T08:25:12.411 INFO:tasks.workunit.client.0.smithi090.stdout:7/854: creat db/d17/d29/d38/d3e/d46/f121 x:0 0 0 2024-04-24T08:25:12.412 INFO:tasks.workunit.client.0.smithi090.stdout:3/765: truncate d2/df/d1a/d37/f49 3746430 0 2024-04-24T08:25:12.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/844: rmdir d8/d76 39 2024-04-24T08:25:12.413 INFO:tasks.workunit.client.0.smithi090.stdout:6/806: dread - d8/d1d/d29/df0/ff2 zero size 2024-04-24T08:25:12.415 INFO:tasks.workunit.client.0.smithi090.stdout:3/766: chown d2/df/d1a/d37/d53/c62 276504 1 2024-04-24T08:25:12.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/734: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f62 30347495 1 2024-04-24T08:25:12.417 INFO:tasks.workunit.client.0.smithi090.stdout:7/855: creat db/d54/db1/f122 x:0 0 0 2024-04-24T08:25:12.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/735: read - d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f83 zero size 2024-04-24T08:25:12.424 INFO:tasks.workunit.client.0.smithi090.stdout:6/807: creat d8/d1d/d29/ffb x:0 0 0 2024-04-24T08:25:12.427 INFO:tasks.workunit.client.0.smithi090.stdout:4/736: creat d3/d8e/ddf/fef x:0 0 0 2024-04-24T08:25:12.428 INFO:tasks.workunit.client.0.smithi090.stdout:6/808: symlink d8/d1d/d1f/d3e/d4f/d7e/daa/de4/lfc 0 2024-04-24T08:25:12.430 INFO:tasks.workunit.client.0.smithi090.stdout:2/843: sync 2024-04-24T08:25:12.430 INFO:tasks.workunit.client.0.smithi090.stdout:1/764: sync 2024-04-24T08:25:12.430 INFO:tasks.workunit.client.0.smithi090.stdout:9/782: sync 2024-04-24T08:25:12.430 INFO:tasks.workunit.client.0.smithi090.stdout:6/809: stat d8/dc/d19/d8e/f62 0 2024-04-24T08:25:12.432 INFO:tasks.workunit.client.0.smithi090.stdout:4/737: read d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/fb4 [309857,49081] 0 2024-04-24T08:25:12.432 INFO:tasks.workunit.client.0.smithi090.stdout:2/844: chown d0/d7/d67/lfb 123874 1 2024-04-24T08:25:12.434 INFO:tasks.workunit.client.0.smithi090.stdout:9/783: mkdir d0/d19/dfc 0 2024-04-24T08:25:12.434 INFO:tasks.workunit.client.0.smithi090.stdout:6/810: rmdir d8/d1d/d1f/d3e/d44 39 2024-04-24T08:25:12.437 INFO:tasks.workunit.client.0.smithi090.stdout:1/765: mkdir d3/d5/d10e 0 2024-04-24T08:25:12.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/738: rmdir d3/dd0 39 2024-04-24T08:25:12.440 INFO:tasks.workunit.client.0.smithi090.stdout:6/811: fsync d8/d1d/d1f/d3e/d4f/f98 0 2024-04-24T08:25:12.441 INFO:tasks.workunit.client.0.smithi090.stdout:9/784: read d0/dc/d24/d7c/d4d/d32/ff3 [26188,46256] 0 2024-04-24T08:25:12.442 INFO:tasks.workunit.client.0.smithi090.stdout:7/856: dread db/d17/d29/d32/d5a/d75/d11f/dae/fb0 [0,4194304] 0 2024-04-24T08:25:12.447 INFO:tasks.workunit.client.0.smithi090.stdout:4/739: creat d3/d8e/ddf/d5c/db5/d77/ff0 x:0 0 0 2024-04-24T08:25:12.450 INFO:tasks.workunit.client.0.smithi090.stdout:0/785: write f8 [2421145,49047] 0 2024-04-24T08:25:12.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/845: write f6 [6028697,31250] 0 2024-04-24T08:25:12.451 INFO:tasks.workunit.client.0.smithi090.stdout:1/766: dwrite d3/d1d/f86 [4194304,4194304] 0 2024-04-24T08:25:12.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/767: write d2/df/d21/f30 [854694,44264] 0 2024-04-24T08:25:12.459 INFO:tasks.workunit.client.0.smithi090.stdout:6/812: mkdir d8/dc/d19/d8e/d38/dfd 0 2024-04-24T08:25:12.459 INFO:tasks.workunit.client.0.smithi090.stdout:2/845: creat d0/d7/d28/daa/f11a x:0 0 0 2024-04-24T08:25:12.462 INFO:tasks.workunit.client.0.smithi090.stdout:8/763: dwrite d2/da/d13/d64/d24/db3/db4/fdd [0,4194304] 0 2024-04-24T08:25:12.463 INFO:tasks.workunit.client.0.smithi090.stdout:7/857: stat db/f12 0 2024-04-24T08:25:12.471 INFO:tasks.workunit.client.0.smithi090.stdout:3/768: dread d2/df/d1a/d37/d53/f6b [0,4194304] 0 2024-04-24T08:25:12.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/740: dread d3/d87/f9f [0,4194304] 0 2024-04-24T08:25:12.476 INFO:tasks.workunit.client.0.smithi090.stdout:9/785: creat d0/d19/df7/ffd x:0 0 0 2024-04-24T08:25:12.478 INFO:tasks.workunit.client.0.smithi090.stdout:9/786: fdatasync d0/dc/d24/d7c/fb0 0 2024-04-24T08:25:12.480 INFO:tasks.workunit.client.0.smithi090.stdout:2/846: creat d0/d14/d17/dc2/dcb/f11b x:0 0 0 2024-04-24T08:25:12.481 INFO:tasks.workunit.client.0.smithi090.stdout:6/813: creat d8/dc/d19/d8e/d81/d96/d97/ffe x:0 0 0 2024-04-24T08:25:12.483 INFO:tasks.workunit.client.0.smithi090.stdout:8/764: read d2/da/d13/d64/f4e [1741183,128201] 0 2024-04-24T08:25:12.491 INFO:tasks.workunit.client.0.smithi090.stdout:5/846: creat d8/d59/dc7/d7a/d50/d9f/d10f/f113 x:0 0 0 2024-04-24T08:25:12.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/769: mknod d2/df/d21/c100 0 2024-04-24T08:25:12.494 INFO:tasks.workunit.client.0.smithi090.stdout:1/767: write d3/d5/da/d97/fd7 [64267,27877] 0 2024-04-24T08:25:12.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/741: rename d3/d8e/ddf/d5c/db5/d69/d8b/db2 to d3/d8e/ddf/d5c/db5/d69/d8b/db2/df1 22 2024-04-24T08:25:12.504 INFO:tasks.workunit.client.0.smithi090.stdout:8/765: write d2/f54 [2853660,127901] 0 2024-04-24T08:25:12.514 INFO:tasks.workunit.client.0.smithi090.stdout:2/847: symlink d0/d7/d67/l11c 0 2024-04-24T08:25:12.516 INFO:tasks.workunit.client.0.smithi090.stdout:4/742: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ff2 x:0 0 0 2024-04-24T08:25:12.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/814: dwrite d8/d1d/d1f/d3e/d44/d90/da2/fdd [0,4194304] 0 2024-04-24T08:25:12.519 INFO:tasks.workunit.client.0.smithi090.stdout:8/766: creat d2/da/d13/d64/d24/dad/f102 x:0 0 0 2024-04-24T08:25:12.520 INFO:tasks.workunit.client.0.smithi090.stdout:4/743: write d3/d8e/ddf/d5c/db5/d69/d8b/fe3 [281381,83976] 0 2024-04-24T08:25:12.531 INFO:tasks.workunit.client.0.smithi090.stdout:1/768: write d3/d5/da/dc/f2b [1794777,105141] 0 2024-04-24T08:25:12.531 INFO:tasks.workunit.client.0.smithi090.stdout:6/815: dwrite d8/d1d/d1f/d3e/d8f/ff8 [0,4194304] 0 2024-04-24T08:25:12.533 INFO:tasks.workunit.client.0.smithi090.stdout:1/769: chown d3/d68/def 1 1 2024-04-24T08:25:12.535 INFO:tasks.workunit.client.0.smithi090.stdout:7/858: dwrite db/d17/d39/f65 [0,4194304] 0 2024-04-24T08:25:12.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/786: link d1a/c31 d1a/d4b/d53/d79/d3c/c10a 0 2024-04-24T08:25:12.541 INFO:tasks.workunit.client.0.smithi090.stdout:3/770: creat d2/df/f101 x:0 0 0 2024-04-24T08:25:12.542 INFO:tasks.workunit.client.0.smithi090.stdout:3/771: stat d2/df/d1a/d37 0 2024-04-24T08:25:12.544 INFO:tasks.workunit.client.0.smithi090.stdout:8/767: symlink d2/da/d13/d64/d24/db3/db4/dcd/l103 0 2024-04-24T08:25:12.546 INFO:tasks.workunit.client.0.smithi090.stdout:9/787: link d0/dc/d24/d7c/d4d/l2b d0/da4/lfe 0 2024-04-24T08:25:12.547 INFO:tasks.workunit.client.0.smithi090.stdout:1/770: fsync d3/f4 0 2024-04-24T08:25:12.549 INFO:tasks.workunit.client.0.smithi090.stdout:6/816: symlink d8/d1d/d1f/d3e/d4f/d7e/lff 0 2024-04-24T08:25:12.551 INFO:tasks.workunit.client.0.smithi090.stdout:0/787: mknod d1a/d4b/d53/d79/d58/c10b 0 2024-04-24T08:25:12.553 INFO:tasks.workunit.client.0.smithi090.stdout:8/768: fsync d2/d62/d83/faf 0 2024-04-24T08:25:12.556 INFO:tasks.workunit.client.0.smithi090.stdout:6/817: symlink d8/d1d/d1f/d3e/d44/d90/da2/l100 0 2024-04-24T08:25:12.557 INFO:tasks.workunit.client.0.smithi090.stdout:1/771: truncate d3/d5/da/dc/d80/fba 191228 0 2024-04-24T08:25:12.559 INFO:tasks.workunit.client.0.smithi090.stdout:6/818: read - d8/dc/d19/d8e/d81/d96/fe6 zero size 2024-04-24T08:25:12.561 INFO:tasks.workunit.client.0.smithi090.stdout:3/772: dwrite d2/df/d21/fee [0,4194304] 0 2024-04-24T08:25:12.561 INFO:tasks.workunit.client.0.smithi090.stdout:6/819: dread - d8/feb zero size 2024-04-24T08:25:12.565 INFO:tasks.workunit.client.0.smithi090.stdout:0/788: fsync d1a/d4b/d53/d79/d35/dea/dfe/fd4 0 2024-04-24T08:25:12.572 INFO:tasks.workunit.client.0.smithi090.stdout:6/820: mknod d8/d1d/d1f/d3e/d44/d90/da2/c101 0 2024-04-24T08:25:12.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/773: dwrite d2/d15/d36/d6f/d7a/fc9 [4194304,4194304] 0 2024-04-24T08:25:12.576 INFO:tasks.workunit.client.0.smithi090.stdout:0/789: creat d1a/d4b/d53/d79/d35/db8/f10c x:0 0 0 2024-04-24T08:25:12.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/790: stat f18 0 2024-04-24T08:25:12.578 INFO:tasks.workunit.client.0.smithi090.stdout:1/772: link d3/d25/c66 d3/d5/da/dc/d80/deb/c10f 0 2024-04-24T08:25:12.582 INFO:tasks.workunit.client.0.smithi090.stdout:6/821: mkdir d8/d1d/d29/d102 0 2024-04-24T08:25:12.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/773: mknod d3/d25/d30/dc5/c110 0 2024-04-24T08:25:12.583 INFO:tasks.workunit.client.0.smithi090.stdout:2/848: write d0/d14/d36/dce/dee/fcc [2148157,129915] 0 2024-04-24T08:25:12.585 INFO:tasks.workunit.client.0.smithi090.stdout:0/791: truncate d1a/d4b/d53/f7f 1541712 0 2024-04-24T08:25:12.587 INFO:tasks.workunit.client.0.smithi090.stdout:3/774: symlink d2/df/d1a/d37/de3/l102 0 2024-04-24T08:25:12.590 INFO:tasks.workunit.client.0.smithi090.stdout:6/822: rmdir d8/d1d/d1f/d3e/d4f/d7e/daa 39 2024-04-24T08:25:12.591 INFO:tasks.workunit.client.0.smithi090.stdout:7/859: dread db/d17/d29/f30 [0,4194304] 0 2024-04-24T08:25:12.593 INFO:tasks.workunit.client.0.smithi090.stdout:0/792: creat d1a/d4b/d53/d79/d58/d96/f10d x:0 0 0 2024-04-24T08:25:12.594 INFO:tasks.workunit.client.0.smithi090.stdout:3/775: mknod d2/d15/d36/d6f/d7a/c103 0 2024-04-24T08:25:12.595 INFO:tasks.workunit.client.0.smithi090.stdout:4/744: write d3/d8e/ddf/d1d/dcb/fea [1036350,109796] 0 2024-04-24T08:25:12.596 INFO:tasks.workunit.client.0.smithi090.stdout:6/823: rmdir d8/dc/d19/d8e/d81 39 2024-04-24T08:25:12.597 INFO:tasks.workunit.client.0.smithi090.stdout:2/849: link d0/d7/d6f/laf d0/d14/d17/dc2/l11d 0 2024-04-24T08:25:12.608 INFO:tasks.workunit.client.0.smithi090.stdout:6/824: fdatasync d8/d1d/d2f/ffa 0 2024-04-24T08:25:12.613 INFO:tasks.workunit.client.0.smithi090.stdout:9/788: dwrite d0/dc/d24/d7c/f36 [0,4194304] 0 2024-04-24T08:25:12.615 INFO:tasks.workunit.client.0.smithi090.stdout:3/776: write d2/df/d1a/f6d [169625,12416] 0 2024-04-24T08:25:12.617 INFO:tasks.workunit.client.0.smithi090.stdout:4/745: truncate d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f35 1243911 0 2024-04-24T08:25:12.624 INFO:tasks.workunit.client.0.smithi090.stdout:8/769: dwrite d2/da/d13/d64/d24/f79 [0,4194304] 0 2024-04-24T08:25:12.626 INFO:tasks.workunit.client.0.smithi090.stdout:3/777: readlink d2/df/d1a/d37/d4b/l9e 0 2024-04-24T08:25:12.628 INFO:tasks.workunit.client.0.smithi090.stdout:2/850: dwrite d0/d14/f1d [0,4194304] 0 2024-04-24T08:25:12.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/906: sync 2024-04-24T08:25:12.630 INFO:tasks.workunit.client.1.smithi149.stdout:8/989: sync 2024-04-24T08:25:12.630 INFO:tasks.workunit.client.1.smithi149.stdout:4/991: sync 2024-04-24T08:25:12.631 INFO:tasks.workunit.client.1.smithi149.stdout:0/967: sync 2024-04-24T08:25:12.631 INFO:tasks.workunit.client.1.smithi149.stdout:6/908: sync 2024-04-24T08:25:12.635 INFO:tasks.workunit.client.0.smithi090.stdout:9/789: dread d0/d19/f28 [8388608,4194304] 0 2024-04-24T08:25:12.636 INFO:tasks.workunit.client.0.smithi090.stdout:1/774: dwrite d3/f41 [0,4194304] 0 2024-04-24T08:25:12.637 INFO:tasks.workunit.client.0.smithi090.stdout:3/778: mknod d2/d84/c104 0 2024-04-24T08:25:12.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/992: rmdir d2/d10/d1a/dac/ddd 39 2024-04-24T08:25:12.640 INFO:tasks.workunit.client.0.smithi090.stdout:7/860: dwrite db/dab/fb3 [0,4194304] 0 2024-04-24T08:25:12.642 INFO:tasks.workunit.client.0.smithi090.stdout:8/770: truncate d2/da/d13/f52 4886037 0 2024-04-24T08:25:12.646 INFO:tasks.workunit.client.1.smithi149.stdout:6/909: unlink d0/d1a/d28/d31/d33/f9e 0 2024-04-24T08:25:12.647 INFO:tasks.workunit.client.0.smithi090.stdout:2/851: creat d0/d7/d67/d7f/d31/ddb/f11e x:0 0 0 2024-04-24T08:25:12.650 INFO:tasks.workunit.client.0.smithi090.stdout:4/746: dwrite d3/d8e/ddf/d1d/dcb/fea [0,4194304] 0 2024-04-24T08:25:12.650 INFO:tasks.workunit.client.0.smithi090.stdout:3/779: unlink d2/d15/d36/f83 0 2024-04-24T08:25:12.654 INFO:tasks.workunit.client.0.smithi090.stdout:1/775: mkdir d3/d5/da/dc/d80/deb/d111 0 2024-04-24T08:25:12.658 INFO:tasks.workunit.client.1.smithi149.stdout:4/993: chown d2/d10/d1a/c11d 121910 1 2024-04-24T08:25:12.661 INFO:tasks.workunit.client.0.smithi090.stdout:7/861: rename f4 to db/d17/d29/d38/d99/f123 0 2024-04-24T08:25:12.662 INFO:tasks.workunit.client.0.smithi090.stdout:6/825: truncate d8/d1d/d1f/d3e/d4f/d7e/f80 1458948 0 2024-04-24T08:25:12.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/771: mknod d2/da/d13/d64/d50/c104 0 2024-04-24T08:25:12.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/990: creat d1/de/f15b x:0 0 0 2024-04-24T08:25:12.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/847: sync 2024-04-24T08:25:12.669 INFO:tasks.workunit.client.1.smithi149.stdout:6/910: mknod d0/d1a/d28/d31/d4c/d80/d6f/dc2/c138 0 2024-04-24T08:25:12.670 INFO:tasks.workunit.client.0.smithi090.stdout:3/780: mkdir d2/df/d5b/d105 0 2024-04-24T08:25:12.673 INFO:tasks.workunit.client.1.smithi149.stdout:4/994: unlink d2/d24/f39 0 2024-04-24T08:25:12.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/776: mknod d3/d25/d30/dc5/c112 0 2024-04-24T08:25:12.675 INFO:tasks.workunit.client.1.smithi149.stdout:4/995: readlink d2/d18/d1e/d3c/d48/le4 0 2024-04-24T08:25:12.677 INFO:tasks.workunit.client.1.smithi149.stdout:8/991: unlink d1/de/d18/f48 0 2024-04-24T08:25:12.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/996: write d2/d24/d2e/f4a [1572542,58517] 0 2024-04-24T08:25:12.680 INFO:tasks.workunit.client.1.smithi149.stdout:6/911: creat d0/d1a/d28/d31/d4c/df8/f139 x:0 0 0 2024-04-24T08:25:12.681 INFO:tasks.workunit.client.1.smithi149.stdout:4/997: chown d2/d18/d1e/d3c/d48/d69/d7d/d8d/def/l105 0 1 2024-04-24T08:25:12.681 INFO:tasks.workunit.client.0.smithi090.stdout:6/826: dwrite d8/d1d/d1f/f9c [0,4194304] 0 2024-04-24T08:25:12.682 INFO:tasks.workunit.client.1.smithi149.stdout:2/907: getdents d0/d6/db/d39/d3d/d44/d57 0 2024-04-24T08:25:12.684 INFO:tasks.workunit.client.0.smithi090.stdout:5/848: unlink d8/da/d14/d2f/d112/dcb/cdc 0 2024-04-24T08:25:12.689 INFO:tasks.workunit.client.0.smithi090.stdout:5/849: fsync d8/da/d14/d2f/d81/fb8 0 2024-04-24T08:25:12.690 INFO:tasks.workunit.client.1.smithi149.stdout:0/968: getdents d2/d3c/d57 0 2024-04-24T08:25:12.691 INFO:tasks.workunit.client.0.smithi090.stdout:5/850: write d8/f9e [4652754,63021] 0 2024-04-24T08:25:12.692 INFO:tasks.workunit.client.0.smithi090.stdout:9/790: dwrite d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/fb2 [0,4194304] 0 2024-04-24T08:25:12.696 INFO:tasks.workunit.client.1.smithi149.stdout:8/992: unlink d1/d1f/d31/fc0 0 2024-04-24T08:25:12.696 INFO:tasks.workunit.client.0.smithi090.stdout:9/791: fdatasync d0/dc/d24/d2d/d49/d81/fea 0 2024-04-24T08:25:12.700 INFO:tasks.workunit.client.0.smithi090.stdout:7/862: mknod db/c124 0 2024-04-24T08:25:12.705 INFO:tasks.workunit.client.1.smithi149.stdout:2/908: truncate d0/d6/db/d39/d3d/d44/f53 720736 0 2024-04-24T08:25:12.705 INFO:tasks.workunit.client.0.smithi090.stdout:4/747: write d3/d8e/ddf/d5c/db5/f30 [1411849,70728] 0 2024-04-24T08:25:12.718 INFO:tasks.workunit.client.1.smithi149.stdout:6/912: mkdir d0/d1a/d28/d31/df4/d13a 0 2024-04-24T08:25:12.721 INFO:tasks.workunit.client.1.smithi149.stdout:4/998: dwrite d2/d24/d2e/d38/fec [8388608,4194304] 0 2024-04-24T08:25:12.723 INFO:tasks.workunit.client.1.smithi149.stdout:8/993: fsync d1/de/d18/d43/d10e/f12c 0 2024-04-24T08:25:12.725 INFO:tasks.workunit.client.0.smithi090.stdout:5/851: mknod d8/d59/dc7/d7a/dbc/c114 0 2024-04-24T08:25:12.727 INFO:tasks.workunit.client.1.smithi149.stdout:4/999: chown d2/d24/d2e/d3f/fcc 41923 1 2024-04-24T08:25:12.733 INFO:tasks.workunit.client.0.smithi090.stdout:9/792: creat d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d92/fff x:0 0 0 2024-04-24T08:25:12.734 INFO:tasks.workunit.client.0.smithi090.stdout:6/827: write d8/dc/fa1 [4039797,48794] 0 2024-04-24T08:25:12.735 INFO:tasks.workunit.client.0.smithi090.stdout:1/777: dwrite d3/d1d/d34/d5a/f5e [8388608,4194304] 0 2024-04-24T08:25:12.739 INFO:tasks.workunit.client.0.smithi090.stdout:4/748: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/ff3 x:0 0 0 2024-04-24T08:25:12.742 INFO:tasks.workunit.client.0.smithi090.stdout:3/781: creat d2/df/d1a/f106 x:0 0 0 2024-04-24T08:25:12.742 INFO:tasks.workunit.client.0.smithi090.stdout:8/772: dwrite d2/da/d13/d64/d24/db3/fc2 [0,4194304] 0 2024-04-24T08:25:12.742 INFO:tasks.workunit.client.1.smithi149.stdout:8/994: symlink d1/de/d18/daf/d129/l15c 0 2024-04-24T08:25:12.746 INFO:tasks.workunit.client.1.smithi149.stdout:0/969: dwrite d2/d9/d1a/d36/d85/fb2 [4194304,4194304] 0 2024-04-24T08:25:12.754 INFO:tasks.workunit.client.0.smithi090.stdout:6/828: rmdir d8/dc/d19/d8e 39 2024-04-24T08:25:12.759 INFO:tasks.workunit.client.1.smithi149.stdout:6/913: write d0/d1a/d28/d31/d4c/d80/d6f/dd7/dd8/fc4 [943984,13824] 0 2024-04-24T08:25:12.762 INFO:tasks.workunit.client.1.smithi149.stdout:2/909: link d0/d6/db/d39/d3d/d44/d8b/lc1 d0/d6/db/d39/d78/d8a/dc5/l116 0 2024-04-24T08:25:12.764 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:12 smithi149 ceph-mon[31282]: Upgrade: Updating mgr.smithi149.vavftl 2024-04-24T08:25:12.764 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:12 smithi149 ceph-mon[31282]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:25:12.765 INFO:tasks.workunit.client.0.smithi090.stdout:4/749: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/fbb [0,4194304] 0 2024-04-24T08:25:12.765 INFO:tasks.workunit.client.1.smithi149.stdout:8/995: symlink d1/d1f/l15d 0 2024-04-24T08:25:12.765 INFO:tasks.workunit.client.1.smithi149.stdout:2/910: readlink d0/d6/db/d27/l67 0 2024-04-24T08:25:12.767 INFO:tasks.workunit.client.0.smithi090.stdout:8/773: dwrite d2/f54 [0,4194304] 0 2024-04-24T08:25:12.769 INFO:tasks.workunit.client.0.smithi090.stdout:4/750: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f 4 1 2024-04-24T08:25:12.773 INFO:tasks.workunit.client.0.smithi090.stdout:4/751: write d3/d8e/ddf/d5c/db5/d7c/fcd [422569,47966] 0 2024-04-24T08:25:12.776 INFO:tasks.workunit.client.0.smithi090.stdout:7/863: creat db/d17/d29/f125 x:0 0 0 2024-04-24T08:25:12.777 INFO:tasks.workunit.client.1.smithi149.stdout:8/996: chown d1/l5 1843064 1 2024-04-24T08:25:12.778 INFO:tasks.workunit.client.0.smithi090.stdout:6/829: read d8/dc/f5f [1235630,39081] 0 2024-04-24T08:25:12.782 INFO:tasks.workunit.client.0.smithi090.stdout:5/852: truncate d8/da/d14/d2f/d112/dcb/f1f 1067719 0 2024-04-24T08:25:12.785 INFO:tasks.workunit.client.0.smithi090.stdout:8/774: mkdir d2/d62/d6b/d9c/dd8/d105 0 2024-04-24T08:25:12.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/775: chown d2/da/d13/d64/d24/c89 13096 1 2024-04-24T08:25:12.787 INFO:tasks.workunit.client.0.smithi090.stdout:7/864: dread - db/dbd/fea zero size 2024-04-24T08:25:12.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/782: write d2/d15/d36/d6f/d9d/fa4 [5213183,33017] 0 2024-04-24T08:25:12.790 INFO:tasks.workunit.client.1.smithi149.stdout:6/914: write d0/f12 [4126267,89623] 0 2024-04-24T08:25:12.790 INFO:tasks.workunit.client.0.smithi090.stdout:6/830: fdatasync d8/d1d/d1f/f89 0 2024-04-24T08:25:12.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:12 smithi090 ceph-mon[25016]: Upgrade: Updating mgr.smithi149.vavftl 2024-04-24T08:25:12.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:12 smithi090 ceph-mon[25016]: Deploying daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:25:12.795 INFO:tasks.workunit.client.0.smithi090.stdout:6/831: chown d8/d1d/d2f/f84 64465860 1 2024-04-24T08:25:12.796 INFO:tasks.workunit.client.0.smithi090.stdout:5/853: unlink d8/d59/dc7/d7a/d50/d9f/d10f/f113 0 2024-04-24T08:25:12.798 INFO:tasks.workunit.client.1.smithi149.stdout:0/970: dwrite d2/d12f/d41/d68/ddc/fdd [0,4194304] 0 2024-04-24T08:25:12.800 INFO:tasks.workunit.client.0.smithi090.stdout:5/854: write d8/d59/dc7/d7a/d50/f57 [5063689,74452] 0 2024-04-24T08:25:12.801 INFO:tasks.workunit.client.0.smithi090.stdout:0/793: sync 2024-04-24T08:25:12.801 INFO:tasks.workunit.client.0.smithi090.stdout:2/852: sync 2024-04-24T08:25:12.802 INFO:tasks.workunit.client.1.smithi149.stdout:6/915: mknod d0/d1a/d28/d31/c13b 0 2024-04-24T08:25:12.803 INFO:tasks.workunit.client.0.smithi090.stdout:5/855: write f6 [4561439,94313] 0 2024-04-24T08:25:12.805 INFO:tasks.workunit.client.0.smithi090.stdout:1/778: link d3/d5/da/dc/f47 d3/d25/d30/d9d/f113 0 2024-04-24T08:25:12.805 INFO:tasks.workunit.client.0.smithi090.stdout:0/794: write d1a/d4b/d53/d79/fb6 [2249794,128291] 0 2024-04-24T08:25:12.808 INFO:tasks.workunit.client.1.smithi149.stdout:2/911: dwrite d0/d6/db/f10 [8388608,4194304] 0 2024-04-24T08:25:12.809 INFO:tasks.workunit.client.0.smithi090.stdout:9/793: rename d0/dc/d24/d7c/d4d/d32/d9f/d78/c7a to d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/c100 0 2024-04-24T08:25:12.812 INFO:tasks.workunit.client.0.smithi090.stdout:6/832: symlink d8/d1d/d1f/d3e/d44/d90/l103 0 2024-04-24T08:25:12.814 INFO:tasks.workunit.client.0.smithi090.stdout:6/833: chown d8/l35 43191930 1 2024-04-24T08:25:12.815 INFO:tasks.workunit.client.0.smithi090.stdout:7/865: dwrite db/f21 [0,4194304] 0 2024-04-24T08:25:12.815 INFO:tasks.workunit.client.0.smithi090.stdout:3/783: dread d2/fa [0,4194304] 0 2024-04-24T08:25:12.821 INFO:tasks.workunit.client.1.smithi149.stdout:2/912: unlink d0/d6/db/d39/d78/cb7 0 2024-04-24T08:25:12.822 INFO:tasks.workunit.client.1.smithi149.stdout:6/916: rename d0/d1a/d28/d31/d33/cd6 to d0/d1a/c13c 0 2024-04-24T08:25:12.824 INFO:tasks.workunit.client.0.smithi090.stdout:2/853: mkdir d0/d14/d17/d11f 0 2024-04-24T08:25:12.827 INFO:tasks.workunit.client.1.smithi149.stdout:2/913: readlink d0/d6/db/d39/d78/d7c/l92 0 2024-04-24T08:25:12.827 INFO:tasks.workunit.client.0.smithi090.stdout:4/752: dwrite d3/d8e/fa2 [0,4194304] 0 2024-04-24T08:25:12.832 INFO:tasks.workunit.client.1.smithi149.stdout:0/971: dwrite d2/d9/df/d48/dba/f110 [0,4194304] 0 2024-04-24T08:25:12.834 INFO:tasks.workunit.client.1.smithi149.stdout:6/917: creat d0/d1a/d28/d31/d4c/d76/f13d x:0 0 0 2024-04-24T08:25:12.834 INFO:tasks.workunit.client.1.smithi149.stdout:2/914: stat d0/d6/db/d28/d7f/d8f/db9/f107 0 2024-04-24T08:25:12.835 INFO:tasks.workunit.client.0.smithi090.stdout:7/866: read - db/d17/d39/d59/fc2 zero size 2024-04-24T08:25:12.835 INFO:tasks.workunit.client.0.smithi090.stdout:6/834: write d8/d1d/fcf [1566046,52532] 0 2024-04-24T08:25:12.839 INFO:tasks.workunit.client.0.smithi090.stdout:8/776: rmdir d2/d62/d6b/d9c/dd8/d105 0 2024-04-24T08:25:12.840 INFO:tasks.workunit.client.0.smithi090.stdout:9/794: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f5c [0,4194304] 0 2024-04-24T08:25:12.842 INFO:tasks.workunit.client.1.smithi149.stdout:8/997: dread d1/d1f/d22/f30 [0,4194304] 0 2024-04-24T08:25:12.844 INFO:tasks.workunit.client.0.smithi090.stdout:6/835: creat d8/dc/d19/dd2/f104 x:0 0 0 2024-04-24T08:25:12.844 INFO:tasks.workunit.client.0.smithi090.stdout:5/856: rmdir d8/d59/dc7/d7b/d106 0 2024-04-24T08:25:12.850 INFO:tasks.workunit.client.1.smithi149.stdout:6/918: unlink d0/f26 0 2024-04-24T08:25:12.851 INFO:tasks.workunit.client.0.smithi090.stdout:0/795: dwrite d1a/d4b/d53/d79/d3c/f5b [0,4194304] 0 2024-04-24T08:25:12.857 INFO:tasks.workunit.client.0.smithi090.stdout:7/867: rename db/d17/d39/f10b to db/d17/d29/d32/f126 0 2024-04-24T08:25:12.859 INFO:tasks.workunit.client.0.smithi090.stdout:7/868: fdatasync db/d54/d78/f10e 0 2024-04-24T08:25:12.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/779: dread d3/d5/f32 [0,4194304] 0 2024-04-24T08:25:12.866 INFO:tasks.workunit.client.1.smithi149.stdout:8/998: mkdir d1/dce/d13e/d15e 0 2024-04-24T08:25:12.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/795: dwrite d0/dc/d24/d2d/d49/d62/fad [0,4194304] 0 2024-04-24T08:25:12.871 INFO:tasks.workunit.client.0.smithi090.stdout:9/796: write d0/f1b [5197805,57479] 0 2024-04-24T08:25:12.881 INFO:tasks.workunit.client.0.smithi090.stdout:6/836: write d8/d1d/d2f/f88 [2935200,57149] 0 2024-04-24T08:25:12.883 INFO:tasks.workunit.client.1.smithi149.stdout:8/999: creat d1/de/d18/d43/d10e/d119/f15f x:0 0 0 2024-04-24T08:25:12.889 INFO:tasks.workunit.client.0.smithi090.stdout:5/857: truncate d8/d59/dc7/d7a/d50/d9f/fba 341805 0 2024-04-24T08:25:12.891 INFO:tasks.workunit.client.0.smithi090.stdout:5/858: chown d8/d59/dc7/d7a/d50/f79 8808485 1 2024-04-24T08:25:12.891 INFO:tasks.workunit.client.1.smithi149.stdout:0/972: link d2/d9/df/d40/lc8 d2/d9/d1a/l136 0 2024-04-24T08:25:12.894 INFO:tasks.workunit.client.1.smithi149.stdout:0/973: chown d2/d9/df/d48/c8b 6976 1 2024-04-24T08:25:12.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/854: dread d0/d14/d36/dce/dee/fd3 [0,4194304] 0 2024-04-24T08:25:12.901 INFO:tasks.workunit.client.0.smithi090.stdout:2/855: write d0/d5e/ffc [1875419,2876] 0 2024-04-24T08:25:12.901 INFO:tasks.workunit.client.0.smithi090.stdout:3/784: dwrite d2/df/d1a/d37/f55 [0,4194304] 0 2024-04-24T08:25:12.903 INFO:tasks.workunit.client.0.smithi090.stdout:4/753: creat d3/d8e/ddf/d5c/db5/d69/d8b/ff4 x:0 0 0 2024-04-24T08:25:12.904 INFO:tasks.workunit.client.1.smithi149.stdout:6/919: getdents d0/d1a/d28/d31/d33 0 2024-04-24T08:25:12.907 INFO:tasks.workunit.client.0.smithi090.stdout:7/869: creat db/dbd/f127 x:0 0 0 2024-04-24T08:25:12.909 INFO:tasks.workunit.client.0.smithi090.stdout:7/870: read - db/d17/d29/d32/dd4/ff4 zero size 2024-04-24T08:25:12.910 INFO:tasks.workunit.client.0.smithi090.stdout:7/871: stat db/d54/d83/l98 0 2024-04-24T08:25:12.912 INFO:tasks.workunit.client.1.smithi149.stdout:0/974: mknod d2/d9/d1a/d36/d85/db5/def/c137 0 2024-04-24T08:25:12.913 INFO:tasks.workunit.client.0.smithi090.stdout:1/780: symlink d3/d1d/d54/l114 0 2024-04-24T08:25:12.914 INFO:tasks.workunit.client.0.smithi090.stdout:7/872: truncate db/d17/d29/d32/d5a/ff8 886880 0 2024-04-24T08:25:12.915 INFO:tasks.workunit.client.1.smithi149.stdout:6/920: truncate d0/d1a/d28/d4a/f77 1999114 0 2024-04-24T08:25:12.916 INFO:tasks.workunit.client.1.smithi149.stdout:0/975: readlink d2/d9/df/d2f/d6c/dc1/d4e/d6f/l10f 0 2024-04-24T08:25:12.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/976: chown d2/d3c/fd4 101 1 2024-04-24T08:25:12.922 INFO:tasks.workunit.client.0.smithi090.stdout:8/777: write d2/f59 [3747561,45684] 0 2024-04-24T08:25:12.923 INFO:tasks.workunit.client.0.smithi090.stdout:6/837: fdatasync d8/d1d/d2f/f3c 0 2024-04-24T08:25:12.924 INFO:tasks.workunit.client.0.smithi090.stdout:5/859: symlink d8/da/d3a/d2d/l115 0 2024-04-24T08:25:12.927 INFO:tasks.workunit.client.1.smithi149.stdout:6/921: creat d0/d1a/d28/d31/d4c/d80/d6f/f13e x:0 0 0 2024-04-24T08:25:12.928 INFO:tasks.workunit.client.0.smithi090.stdout:0/796: mkdir d1a/d4b/d53/d79/d69/d90/d10e 0 2024-04-24T08:25:12.931 INFO:tasks.workunit.client.1.smithi149.stdout:6/922: readlink d0/d1a/d28/d31/d4c/d4e/d12e/lb3 0 2024-04-24T08:25:12.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/873: creat db/d17/d29/d38/d3e/f128 x:0 0 0 2024-04-24T08:25:12.933 INFO:tasks.workunit.client.0.smithi090.stdout:7/874: chown db/d17/d29/f55 7962 1 2024-04-24T08:25:12.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/797: creat d0/dc/d24/d2d/d49/dfa/f101 x:0 0 0 2024-04-24T08:25:12.937 INFO:tasks.workunit.client.0.smithi090.stdout:7/875: truncate db/d17/d29/d38/d3e/ff2 360047 0 2024-04-24T08:25:12.938 INFO:tasks.workunit.client.1.smithi149.stdout:6/923: mkdir d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/d13f 0 2024-04-24T08:25:12.940 INFO:tasks.workunit.client.0.smithi090.stdout:8/778: truncate d2/d62/f93 849920 0 2024-04-24T08:25:12.940 INFO:tasks.workunit.client.1.smithi149.stdout:6/924: chown d0/d1a/d28/d4a/l85 3720067 1 2024-04-24T08:25:12.941 INFO:tasks.workunit.client.0.smithi090.stdout:8/779: chown d2/da/d13/d64/f8f 27 1 2024-04-24T08:25:12.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/780: fdatasync d2/da/d13/d64/d50/f57 0 2024-04-24T08:25:12.944 INFO:tasks.workunit.client.1.smithi149.stdout:0/977: dwrite d2/d9/fd8 [0,4194304] 0 2024-04-24T08:25:12.945 INFO:tasks.workunit.client.0.smithi090.stdout:6/838: creat d8/d1d/d1f/d3e/d7a/f105 x:0 0 0 2024-04-24T08:25:12.951 INFO:tasks.workunit.client.0.smithi090.stdout:1/781: rename d3/d25 to d3/d5/da/d94/da6/d115 0 2024-04-24T08:25:12.952 INFO:tasks.workunit.client.1.smithi149.stdout:0/978: symlink d2/d9/df/d48/l138 0 2024-04-24T08:25:12.955 INFO:tasks.workunit.client.0.smithi090.stdout:9/798: creat d0/dc/d24/d2d/dda/de1/f102 x:0 0 0 2024-04-24T08:25:12.959 INFO:tasks.workunit.client.1.smithi149.stdout:0/979: creat d2/d9/df/d56/dca/f139 x:0 0 0 2024-04-24T08:25:12.962 INFO:tasks.workunit.client.0.smithi090.stdout:6/839: unlink d8/l9d 0 2024-04-24T08:25:12.965 INFO:tasks.workunit.client.0.smithi090.stdout:1/782: creat d3/d5/da/d97/d9c/dc4/dcc/f116 x:0 0 0 2024-04-24T08:25:12.966 INFO:tasks.workunit.client.0.smithi090.stdout:5/860: dwrite d8/da/d3a/d2d/f4d [0,4194304] 0 2024-04-24T08:25:12.967 INFO:tasks.workunit.client.1.smithi149.stdout:6/925: write d0/d1a/d28/d31/d4c/d76/fe1 [530927,47095] 0 2024-04-24T08:25:12.970 INFO:tasks.workunit.client.0.smithi090.stdout:9/799: mkdir d0/dc/d24/d2d/dda/de1/d103 0 2024-04-24T08:25:12.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/856: getdents d0/d14/d17 0 2024-04-24T08:25:12.975 INFO:tasks.workunit.client.0.smithi090.stdout:9/800: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f9d [981862,14173] 0 2024-04-24T08:25:12.977 INFO:tasks.workunit.client.1.smithi149.stdout:6/926: dread - d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fbf zero size 2024-04-24T08:25:12.977 INFO:tasks.workunit.client.0.smithi090.stdout:0/797: creat d1a/d4b/f10f x:0 0 0 2024-04-24T08:25:12.981 INFO:tasks.workunit.client.0.smithi090.stdout:6/840: unlink d8/d1d/d1f/d3e/d44/d90/l103 0 2024-04-24T08:25:12.981 INFO:tasks.workunit.client.0.smithi090.stdout:6/841: fsync d8/dc/fa1 0 2024-04-24T08:25:12.982 INFO:tasks.workunit.client.0.smithi090.stdout:6/842: fsync d8/d1d/d1f/d8c/fd4 0 2024-04-24T08:25:12.983 INFO:tasks.workunit.client.0.smithi090.stdout:5/861: truncate d8/d9c/fc5 5239997 0 2024-04-24T08:25:12.984 INFO:tasks.workunit.client.1.smithi149.stdout:6/927: fdatasync d0/d1a/d28/d31/d33/f9b 0 2024-04-24T08:25:12.989 INFO:tasks.workunit.client.1.smithi149.stdout:6/928: write d0/d1a/d28/d31/d4c/d80/d6f/dd7/ff6 [2051425,101900] 0 2024-04-24T08:25:12.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/929: stat d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d 0 2024-04-24T08:25:12.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/798: creat d1a/d4b/d53/d79/d58/db4/f110 x:0 0 0 2024-04-24T08:25:12.995 INFO:tasks.workunit.client.0.smithi090.stdout:5/862: creat d8/da/d3a/d2d/f116 x:0 0 0 2024-04-24T08:25:12.995 INFO:tasks.workunit.client.0.smithi090.stdout:8/781: write d2/da/d13/d64/d50/f9f [788147,109531] 0 2024-04-24T08:25:12.995 INFO:tasks.workunit.client.1.smithi149.stdout:6/930: chown d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec 581591024 1 2024-04-24T08:25:12.996 INFO:tasks.workunit.client.0.smithi090.stdout:9/801: truncate d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96/fa1 676078 0 2024-04-24T08:25:12.996 INFO:tasks.workunit.client.0.smithi090.stdout:2/857: dwrite d0/d14/d17/dc2/dcb/f11b [0,4194304] 0 2024-04-24T08:25:12.999 INFO:tasks.workunit.client.0.smithi090.stdout:9/802: chown d0/dc/d24/d7c/d4d/l74 44 1 2024-04-24T08:25:13.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/799: dread - d1a/d4b/d53/d79/d69/d8a/fbe zero size 2024-04-24T08:25:13.001 INFO:tasks.workunit.client.0.smithi090.stdout:7/876: truncate db/d17/d29/d38/d99/fad 489684 0 2024-04-24T08:25:13.002 INFO:tasks.workunit.client.0.smithi090.stdout:2/858: write d0/d7/d28/daa/ddc/f109 [576635,116450] 0 2024-04-24T08:25:13.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/980: dread d2/d9/df/d48/f4f [0,4194304] 0 2024-04-24T08:25:13.010 INFO:tasks.workunit.client.0.smithi090.stdout:6/843: getdents d8/d1d/d1f/d3e/d4f 0 2024-04-24T08:25:13.010 INFO:tasks.workunit.client.0.smithi090.stdout:7/877: chown db/d17/d29/d32/d5a/fe6 95228 1 2024-04-24T08:25:13.013 INFO:tasks.workunit.client.0.smithi090.stdout:9/803: creat d0/dc/d24/d2d/d49/d62/dcb/f104 x:0 0 0 2024-04-24T08:25:13.014 INFO:tasks.workunit.client.0.smithi090.stdout:0/800: mknod d1a/d4b/d53/d79/d35/db8/c111 0 2024-04-24T08:25:13.015 INFO:tasks.workunit.client.1.smithi149.stdout:6/931: link d0/d1a/f4d d0/d1a/d28/d31/d4c/d80/d6f/f140 0 2024-04-24T08:25:13.016 INFO:tasks.workunit.client.0.smithi090.stdout:2/859: mknod d0/d14/d17/dc2/dcb/c120 0 2024-04-24T08:25:13.017 INFO:tasks.workunit.client.0.smithi090.stdout:1/783: dwrite d3/d5/da/f82 [0,4194304] 0 2024-04-24T08:25:13.019 INFO:tasks.workunit.client.0.smithi090.stdout:9/804: truncate d0/d19/df7/ffd 324311 0 2024-04-24T08:25:13.022 INFO:tasks.workunit.client.1.smithi149.stdout:6/932: creat d0/d1a/d28/d31/d4c/d4e/d62/d66/f141 x:0 0 0 2024-04-24T08:25:13.025 INFO:tasks.workunit.client.0.smithi090.stdout:0/801: rmdir d1a/d4b/d53/d79/d69 39 2024-04-24T08:25:13.027 INFO:tasks.workunit.client.1.smithi149.stdout:6/933: dread - d0/d1a/d28/d31/d4c/d4e/d12e/f63 zero size 2024-04-24T08:25:13.028 INFO:tasks.workunit.client.0.smithi090.stdout:7/878: mkdir db/dab/d109/d129 0 2024-04-24T08:25:13.028 INFO:tasks.workunit.client.0.smithi090.stdout:5/863: write d8/da/d14/d2f/d81/f98 [772716,105158] 0 2024-04-24T08:25:13.030 INFO:tasks.workunit.client.0.smithi090.stdout:2/860: creat d0/d7/d28/daa/def/db4/dcd/f121 x:0 0 0 2024-04-24T08:25:13.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/805: rename d0/dc/d24/d7c/d4d/d32/dab/cf8 to d0/dc/d24/d2d/d49/d62/dcb/dec/c105 0 2024-04-24T08:25:13.034 INFO:tasks.workunit.client.0.smithi090.stdout:4/754: sync 2024-04-24T08:25:13.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/785: sync 2024-04-24T08:25:13.034 INFO:tasks.workunit.client.0.smithi090.stdout:7/879: write db/dbd/fd0 [1099206,77923] 0 2024-04-24T08:25:13.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/934: symlink d0/d1a/d28/d31/d4c/df8/df9/l142 0 2024-04-24T08:25:13.041 INFO:tasks.workunit.client.0.smithi090.stdout:9/806: write d0/dc/d24/d7c/d4d/d8c/fdd [395125,3171] 0 2024-04-24T08:25:13.041 INFO:tasks.workunit.client.0.smithi090.stdout:0/802: mknod d1a/d4b/d53/c112 0 2024-04-24T08:25:13.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/782: dwrite d2/d62/f7a [0,4194304] 0 2024-04-24T08:25:13.045 INFO:tasks.workunit.client.0.smithi090.stdout:3/786: write d2/df/d57/d4f/f86 [1708896,32897] 0 2024-04-24T08:25:13.047 INFO:tasks.workunit.client.0.smithi090.stdout:4/755: truncate d3/d8e/ddf/d1d/f40 2368760 0 2024-04-24T08:25:13.047 INFO:tasks.workunit.client.0.smithi090.stdout:1/784: dwrite d3/d5/da/d97/d9c/dc4/dcc/f116 [0,4194304] 0 2024-04-24T08:25:13.051 INFO:tasks.workunit.client.1.smithi149.stdout:0/981: dread d2/d9/df/d56/f80 [0,4194304] 0 2024-04-24T08:25:13.052 INFO:tasks.workunit.client.0.smithi090.stdout:4/756: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f51 [3845302,9111] 0 2024-04-24T08:25:13.053 INFO:tasks.workunit.client.0.smithi090.stdout:7/880: creat db/dbd/d112/f12a x:0 0 0 2024-04-24T08:25:13.053 INFO:tasks.workunit.client.0.smithi090.stdout:5/864: mkdir d8/da/d14/d2f/d112/dcb/de3/d117 0 2024-04-24T08:25:13.053 INFO:tasks.workunit.client.0.smithi090.stdout:4/757: readlink d3/d24/l5f 0 2024-04-24T08:25:13.055 INFO:tasks.workunit.client.0.smithi090.stdout:4/758: chown d3/f32 7 1 2024-04-24T08:25:13.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/807: mkdir d0/d2a/d106 0 2024-04-24T08:25:13.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/881: write db/d17/d29/d32/dd4/ff4 [438686,16697] 0 2024-04-24T08:25:13.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/882: read db/d17/d29/d32/d5a/f8d [3743585,34653] 0 2024-04-24T08:25:13.061 INFO:tasks.workunit.client.0.smithi090.stdout:2/861: creat d0/d7/de1/f122 x:0 0 0 2024-04-24T08:25:13.063 INFO:tasks.workunit.client.0.smithi090.stdout:9/808: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/fc4 [866264,126992] 0 2024-04-24T08:25:13.063 INFO:tasks.workunit.client.0.smithi090.stdout:8/783: chown d2/da/d13/d64/d24/db3/db4/cbf 434 1 2024-04-24T08:25:13.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/883: write db/d17/d29/d32/d5a/d75/f81 [1865180,10958] 0 2024-04-24T08:25:13.067 INFO:tasks.workunit.client.0.smithi090.stdout:3/787: creat d2/df/d5b/f107 x:0 0 0 2024-04-24T08:25:13.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/785: rename d3/d5/da/f4d to d3/d5/d3e/f117 0 2024-04-24T08:25:13.069 INFO:tasks.workunit.client.0.smithi090.stdout:0/803: rmdir d1a 39 2024-04-24T08:25:13.070 INFO:tasks.workunit.client.1.smithi149.stdout:0/982: mkdir d2/d9/df/d2f/d6c/dc1/d4e/d6f/db6/df0/d13a 0 2024-04-24T08:25:13.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/784: write d2/da/d13/d64/d24/db3/db4/fdd [4160413,76744] 0 2024-04-24T08:25:13.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/862: symlink d0/d5e/de2/l123 0 2024-04-24T08:25:13.080 INFO:tasks.workunit.client.1.smithi149.stdout:6/935: dwrite d0/d1a/d28/d4a/f77 [0,4194304] 0 2024-04-24T08:25:13.080 INFO:tasks.workunit.client.0.smithi090.stdout:3/788: fsync d2/df/d57/d4f/f86 0 2024-04-24T08:25:13.083 INFO:tasks.workunit.client.0.smithi090.stdout:6/844: dwrite d8/dc/d19/d8e/d81/d96/d97/f9b [0,4194304] 0 2024-04-24T08:25:13.086 INFO:tasks.workunit.client.0.smithi090.stdout:4/759: dwrite d3/d8e/fdd [0,4194304] 0 2024-04-24T08:25:13.086 INFO:tasks.workunit.client.0.smithi090.stdout:7/884: rmdir db/dab/d109 39 2024-04-24T08:25:13.087 INFO:tasks.workunit.client.0.smithi090.stdout:4/760: stat d3/d8e/ddf/d5c/f74 0 2024-04-24T08:25:13.093 INFO:tasks.workunit.client.1.smithi149.stdout:6/936: mknod d0/d1a/d28/d31/d4c/df8/c143 0 2024-04-24T08:25:13.098 INFO:tasks.workunit.client.0.smithi090.stdout:9/809: creat d0/dc/d24/d7c/d4d/d32/d9f/d72/de9/f107 x:0 0 0 2024-04-24T08:25:13.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/865: rename d8/da/d14/d2f/d81 to d8/d7c/da9/d118 0 2024-04-24T08:25:13.113 INFO:tasks.workunit.client.0.smithi090.stdout:7/885: chown db/fd 287743621 1 2024-04-24T08:25:13.114 INFO:tasks.workunit.client.1.smithi149.stdout:0/983: write d2/d9/df/d48/dba/d12c/fed [214824,55245] 0 2024-04-24T08:25:13.114 INFO:tasks.workunit.client.0.smithi090.stdout:3/789: dwrite d2/df/d57/d4f/f86 [0,4194304] 0 2024-04-24T08:25:13.115 INFO:tasks.workunit.client.0.smithi090.stdout:6/845: dwrite d8/d1d/d1f/d3e/d4d/f99 [0,4194304] 0 2024-04-24T08:25:13.118 INFO:tasks.workunit.client.0.smithi090.stdout:2/863: rename d0/d7/d28/daa/def/db4 to d0/d7/d28/d124 0 2024-04-24T08:25:13.119 INFO:tasks.workunit.client.0.smithi090.stdout:1/786: fdatasync d3/d5/da/d94/da6/d115/d30/d9d/f113 0 2024-04-24T08:25:13.119 INFO:tasks.workunit.client.0.smithi090.stdout:4/761: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/fe3 [0,4194304] 0 2024-04-24T08:25:13.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/864: fsync d0/d7/de1/ff8 0 2024-04-24T08:25:13.121 INFO:tasks.workunit.client.0.smithi090.stdout:7/886: truncate db/d17/d29/d38/d3e/fe7 775782 0 2024-04-24T08:25:13.123 INFO:tasks.workunit.client.0.smithi090.stdout:8/785: mknod d2/da/d13/d64/d24/db3/db4/dcd/de8/c106 0 2024-04-24T08:25:13.126 INFO:tasks.workunit.client.0.smithi090.stdout:1/787: unlink d3/d5/da/d97/d9c/dc4/dcc/f116 0 2024-04-24T08:25:13.127 INFO:tasks.workunit.client.0.smithi090.stdout:0/804: creat d1a/d4b/f113 x:0 0 0 2024-04-24T08:25:13.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/790: getdents d2/d25/d33/dc8 0 2024-04-24T08:25:13.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/805: fdatasync d1a/d4b/d53/f6a 0 2024-04-24T08:25:13.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/786: mkdir d2/da/d13/d64/db2/d107 0 2024-04-24T08:25:13.135 INFO:tasks.workunit.client.0.smithi090.stdout:3/791: chown d2/d15/df8 5188 1 2024-04-24T08:25:13.136 INFO:tasks.workunit.client.0.smithi090.stdout:7/887: dread db/f21 [0,4194304] 0 2024-04-24T08:25:13.138 INFO:tasks.workunit.client.0.smithi090.stdout:2/865: rename d0/d14/d36/d77/l83 to d0/d7/d67/l125 0 2024-04-24T08:25:13.142 INFO:tasks.workunit.client.1.smithi149.stdout:6/937: write d0/f10 [1773660,111377] 0 2024-04-24T08:25:13.143 INFO:tasks.workunit.client.0.smithi090.stdout:5/866: dwrite d8/d76/dad/fdb [0,4194304] 0 2024-04-24T08:25:13.146 INFO:tasks.workunit.client.1.smithi149.stdout:6/938: chown d0/d1a/d7e/la5 4 1 2024-04-24T08:25:13.147 INFO:tasks.workunit.client.0.smithi090.stdout:1/788: rename d3/d5/da/dc/l4b to d3/d5/da/d94/da6/d115/d30/dc5/d102/l118 0 2024-04-24T08:25:13.152 INFO:tasks.workunit.client.0.smithi090.stdout:7/888: write db/d17/d29/d32/d5a/d75/d11f/dae/fb0 [1697098,117305] 0 2024-04-24T08:25:13.158 INFO:tasks.workunit.client.0.smithi090.stdout:7/889: fsync db/d17/d29/d32/d5a/ff8 0 2024-04-24T08:25:13.160 INFO:tasks.workunit.client.0.smithi090.stdout:8/787: dwrite d2/da/d13/d60/d76/fca [0,4194304] 0 2024-04-24T08:25:13.166 INFO:tasks.workunit.client.0.smithi090.stdout:7/890: creat db/dbd/d112/f12b x:0 0 0 2024-04-24T08:25:13.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/866: getdents d0/d7/d28/daa/def 0 2024-04-24T08:25:13.173 INFO:tasks.workunit.client.0.smithi090.stdout:5/867: dwrite d8/d59/dc7/d7a/da1/dae/fe6 [0,4194304] 0 2024-04-24T08:25:13.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/789: dwrite d3/d5/d3e/da3/ffc [0,4194304] 0 2024-04-24T08:25:13.180 INFO:tasks.workunit.client.0.smithi090.stdout:4/762: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f4c [0,4194304] 0 2024-04-24T08:25:13.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/790: stat d3/d1d/fc9 0 2024-04-24T08:25:13.183 INFO:tasks.workunit.client.0.smithi090.stdout:5/868: mkdir d8/d76/d119 0 2024-04-24T08:25:13.185 INFO:tasks.workunit.client.0.smithi090.stdout:4/763: write d3/d8e/ddf/d5c/db5/d7c/fcd [1405489,59815] 0 2024-04-24T08:25:13.190 INFO:tasks.workunit.client.0.smithi090.stdout:2/867: symlink d0/d7/d67/d7f/d31/d46/l126 0 2024-04-24T08:25:13.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/939: write d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/f94 [444627,88621] 0 2024-04-24T08:25:13.195 INFO:tasks.workunit.client.0.smithi090.stdout:3/792: dread d2/d15/d36/d6f/d7a/fc9 [0,4194304] 0 2024-04-24T08:25:13.195 INFO:tasks.workunit.client.0.smithi090.stdout:4/764: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/fbb 68925 1 2024-04-24T08:25:13.197 INFO:tasks.workunit.client.0.smithi090.stdout:5/869: unlink d8/d59/dc7/d7a/d50/f79 0 2024-04-24T08:25:13.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/891: link db/d17/d29/d32/d5a/d75/d11f/ff9 db/d17/d29/d38/d99/dfb/f12c 0 2024-04-24T08:25:13.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/846: dread d8/dc/d19/d8e/f4b [0,4194304] 0 2024-04-24T08:25:13.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/870: stat d8/d76/dad/lcd 0 2024-04-24T08:25:13.201 INFO:tasks.workunit.client.0.smithi090.stdout:5/871: stat d8/d59/f61 0 2024-04-24T08:25:13.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/868: mknod d0/d7/d28/d124/dcd/c127 0 2024-04-24T08:25:13.208 INFO:tasks.workunit.client.0.smithi090.stdout:7/892: mknod db/d17/d29/d38/d99/dfb/c12d 0 2024-04-24T08:25:13.208 INFO:tasks.workunit.client.0.smithi090.stdout:2/869: fsync d0/fad 0 2024-04-24T08:25:13.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/793: creat d2/d25/f108 x:0 0 0 2024-04-24T08:25:13.212 INFO:tasks.workunit.client.0.smithi090.stdout:6/847: rename d8/dc/d19/d8e to d8/d1d/d1f/d3e/d44/d90/d106 0 2024-04-24T08:25:13.213 INFO:tasks.workunit.client.0.smithi090.stdout:3/794: write d2/df/d1a/d37/dc0/ff6 [60940,57987] 0 2024-04-24T08:25:13.213 INFO:tasks.workunit.client.0.smithi090.stdout:6/848: stat d8/dc/d19 0 2024-04-24T08:25:13.216 INFO:tasks.workunit.client.0.smithi090.stdout:7/893: mknod db/d17/d29/c12e 0 2024-04-24T08:25:13.216 INFO:tasks.workunit.client.0.smithi090.stdout:6/849: write d8/d1d/d1f/d3e/d44/d90/da2/fdd [2477255,100565] 0 2024-04-24T08:25:13.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/795: mkdir d2/d25/d109 0 2024-04-24T08:25:13.218 INFO:tasks.workunit.client.0.smithi090.stdout:7/894: chown db/dbd/fd0 123637 1 2024-04-24T08:25:13.221 INFO:tasks.workunit.client.0.smithi090.stdout:6/850: stat d8/d1d/d1f/d3e/d44/d90/fa9 0 2024-04-24T08:25:13.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/895: fsync db/d17/d29/d38/d3e/d46/f103 0 2024-04-24T08:25:13.223 INFO:tasks.workunit.client.0.smithi090.stdout:3/796: creat d2/d25/d2c/f10a x:0 0 0 2024-04-24T08:25:13.225 INFO:tasks.workunit.client.0.smithi090.stdout:2/870: dwrite d0/d7/d28/ff3 [0,4194304] 0 2024-04-24T08:25:13.225 INFO:tasks.workunit.client.0.smithi090.stdout:3/797: chown d2/df/d1a/lc5 2435635 1 2024-04-24T08:25:13.228 INFO:tasks.workunit.client.0.smithi090.stdout:2/871: fsync d0/d14/f34 0 2024-04-24T08:25:13.238 INFO:tasks.workunit.client.1.smithi149.stdout:6/940: write d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f81 [3610496,82088] 0 2024-04-24T08:25:13.243 INFO:tasks.workunit.client.1.smithi149.stdout:6/941: unlink d0/d1a/d28/d31/d4c/lb1 0 2024-04-24T08:25:13.246 INFO:tasks.workunit.client.1.smithi149.stdout:6/942: creat d0/d1a/d28/d31/d4c/d4e/d12e/d106/f144 x:0 0 0 2024-04-24T08:25:13.253 INFO:tasks.workunit.client.1.smithi149.stdout:6/943: link d0/d1a/d28/d31/d4c/d4e/d12e/lfb d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/l145 0 2024-04-24T08:25:13.255 INFO:tasks.workunit.client.1.smithi149.stdout:6/944: chown d0/d1a/d28/d31/d4c/d4e/db8/dea 0 1 2024-04-24T08:25:13.258 INFO:tasks.workunit.client.1.smithi149.stdout:6/945: fsync d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/fa0 0 2024-04-24T08:25:13.273 INFO:tasks.workunit.client.1.smithi149.stdout:6/946: dwrite d0/f11 [8388608,4194304] 0 2024-04-24T08:25:13.274 INFO:tasks.workunit.client.0.smithi090.stdout:9/810: sync 2024-04-24T08:25:13.275 INFO:tasks.workunit.client.1.smithi149.stdout:6/947: stat d0/d1a/d28/d31/d4c/d4e/d62/d66/cd1 0 2024-04-24T08:25:13.277 INFO:tasks.workunit.client.0.smithi090.stdout:3/798: creat d2/d15/dc4/f10b x:0 0 0 2024-04-24T08:25:13.277 INFO:tasks.workunit.client.0.smithi090.stdout:6/851: truncate d8/d1d/d1f/d3e/d7a/f9a 4082425 0 2024-04-24T08:25:13.278 INFO:tasks.workunit.client.1.smithi149.stdout:6/948: chown d0/d1a/c6d 6 1 2024-04-24T08:25:13.280 INFO:tasks.workunit.client.0.smithi090.stdout:2/872: dread - d0/d7/d67/d7f/d31/d90/f108 zero size 2024-04-24T08:25:13.281 INFO:tasks.workunit.client.0.smithi090.stdout:5/872: write d8/d7c/da9/f110 [973912,36850] 0 2024-04-24T08:25:13.284 INFO:tasks.workunit.client.0.smithi090.stdout:9/811: read - d0/dc/d24/d7c/d4d/d32/fc8 zero size 2024-04-24T08:25:13.285 INFO:tasks.workunit.client.1.smithi149.stdout:6/949: unlink d0/d1a/d28/d31/d4c/d80/f68 0 2024-04-24T08:25:13.286 INFO:tasks.workunit.client.0.smithi090.stdout:6/852: mknod d8/d1d/d1f/d3e/d4d/c107 0 2024-04-24T08:25:13.288 INFO:tasks.workunit.client.0.smithi090.stdout:2/873: fdatasync d0/d7/d28/ff3 0 2024-04-24T08:25:13.290 INFO:tasks.workunit.client.0.smithi090.stdout:6/853: fdatasync f6 0 2024-04-24T08:25:13.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/873: fsync d8/da/d3a/d2d/d37/dec/dfe/f101 0 2024-04-24T08:25:13.295 INFO:tasks.workunit.client.0.smithi090.stdout:1/791: dwrite d3/d5/da/d94/da6/fb9 [0,4194304] 0 2024-04-24T08:25:13.298 INFO:tasks.workunit.client.0.smithi090.stdout:4/765: dread d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f4f [0,4194304] 0 2024-04-24T08:25:13.303 INFO:tasks.workunit.client.0.smithi090.stdout:7/896: dwrite db/d17/d29/d38/d3e/fc8 [0,4194304] 0 2024-04-24T08:25:13.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/874: rename d8/d99/f9d to d8/da/d3a/f11a 0 2024-04-24T08:25:13.309 INFO:tasks.workunit.client.0.smithi090.stdout:6/854: rename d8/cc1 to d8/c108 0 2024-04-24T08:25:13.312 INFO:tasks.workunit.client.0.smithi090.stdout:5/875: creat d8/d76/f11b x:0 0 0 2024-04-24T08:25:13.312 INFO:tasks.workunit.client.0.smithi090.stdout:7/897: truncate db/d17/d39/f97 3154950 0 2024-04-24T08:25:13.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/855: fdatasync d8/d1d/d1f/d3e/d7a/fef 0 2024-04-24T08:25:13.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/792: fsync d3/d5/da/d94/da6/d115/d30/dc5/fe8 0 2024-04-24T08:25:13.321 INFO:tasks.workunit.client.0.smithi090.stdout:9/812: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/fc4 [0,4194304] 0 2024-04-24T08:25:13.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/874: dwrite d0/d2e/f37 [4194304,4194304] 0 2024-04-24T08:25:13.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/898: creat db/d17/d29/d38/d99/dfb/f12f x:0 0 0 2024-04-24T08:25:13.323 INFO:tasks.workunit.client.0.smithi090.stdout:5/876: creat d8/da/d14/f11c x:0 0 0 2024-04-24T08:25:13.324 INFO:tasks.workunit.client.0.smithi090.stdout:3/799: dread d2/df/d1a/d37/d4b/fe5 [0,4194304] 0 2024-04-24T08:25:13.325 INFO:tasks.workunit.client.1.smithi149.stdout:6/950: dread d0/f12 [0,4194304] 0 2024-04-24T08:25:13.328 INFO:tasks.workunit.client.0.smithi090.stdout:2/875: write d0/d7/d67/d7f/f10e [92720,55520] 0 2024-04-24T08:25:13.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/800: chown d2/d15/d36 357 1 2024-04-24T08:25:13.329 INFO:tasks.workunit.client.0.smithi090.stdout:7/899: fdatasync db/d17/d29/d38/d99/f105 0 2024-04-24T08:25:13.330 INFO:tasks.workunit.client.1.smithi149.stdout:6/951: readlink d0/d1a/d28/d4a/l56 0 2024-04-24T08:25:13.331 INFO:tasks.workunit.client.0.smithi090.stdout:6/856: mknod d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/c109 0 2024-04-24T08:25:13.341 INFO:tasks.workunit.client.0.smithi090.stdout:0/806: sync 2024-04-24T08:25:13.344 INFO:tasks.workunit.client.0.smithi090.stdout:4/766: truncate d3/d24/f2b 3696463 0 2024-04-24T08:25:13.344 INFO:tasks.workunit.client.0.smithi090.stdout:2/876: write d0/d7/d28/daa/db1/ff0 [5018297,115891] 0 2024-04-24T08:25:13.348 INFO:tasks.workunit.client.0.smithi090.stdout:2/877: rmdir d0/d7/dda 39 2024-04-24T08:25:13.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/767: mknod d3/dd9/cf5 0 2024-04-24T08:25:13.352 INFO:tasks.workunit.client.0.smithi090.stdout:3/801: dwrite d2/f2f [0,4194304] 0 2024-04-24T08:25:13.355 INFO:tasks.workunit.client.0.smithi090.stdout:6/857: dread d8/d1d/d1f/d3e/d44/d90/d106/f4b [0,4194304] 0 2024-04-24T08:25:13.356 INFO:tasks.workunit.client.0.smithi090.stdout:2/878: creat d0/d7/d28/d124/dcd/f128 x:0 0 0 2024-04-24T08:25:13.363 INFO:tasks.workunit.client.0.smithi090.stdout:1/793: dwrite d3/d5/da/d97/fbf [0,4194304] 0 2024-04-24T08:25:13.364 INFO:tasks.workunit.client.0.smithi090.stdout:2/879: write d0/d7/d67/d7f/d31/d90/f108 [310633,116119] 0 2024-04-24T08:25:13.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/768: link d3/d8e/ddf/d5c/fe1 d3/ff6 0 2024-04-24T08:25:13.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/802: creat d2/df/d1a/d37/dc0/f10c x:0 0 0 2024-04-24T08:25:13.373 INFO:tasks.workunit.client.0.smithi090.stdout:2/880: fsync d0/d5e/ffc 0 2024-04-24T08:25:13.374 INFO:tasks.workunit.client.1.smithi149.stdout:6/952: dwrite d0/d1a/d28/d31/f37 [0,4194304] 0 2024-04-24T08:25:13.377 INFO:tasks.workunit.client.0.smithi090.stdout:3/803: chown d2/d15/cc1 13403 1 2024-04-24T08:25:13.377 INFO:tasks.workunit.client.0.smithi090.stdout:6/858: getdents d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/df6 0 2024-04-24T08:25:13.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/769: fsync d3/d24/f2b 0 2024-04-24T08:25:13.378 INFO:tasks.workunit.client.0.smithi090.stdout:1/794: truncate d3/d1d/ff1 745445 0 2024-04-24T08:25:13.381 INFO:tasks.workunit.client.1.smithi149.stdout:6/953: truncate d0/d1a/d28/d31/d4c/d4e/d62/fbc 1061282 0 2024-04-24T08:25:13.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/877: dwrite d8/da/d14/d2f/d112/dcb/d21/f48 [0,4194304] 0 2024-04-24T08:25:13.384 INFO:tasks.workunit.client.0.smithi090.stdout:0/807: dwrite d1a/d4b/d53/d79/d69/d90/dcc/ff7 [0,4194304] 0 2024-04-24T08:25:13.399 INFO:tasks.workunit.client.0.smithi090.stdout:0/808: write d1a/fcd [3938225,50253] 0 2024-04-24T08:25:13.401 INFO:tasks.workunit.client.0.smithi090.stdout:5/878: fsync d8/da/d3a/d2d/d37/dec/dfe/f101 0 2024-04-24T08:25:13.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/813: dwrite d0/dc/d24/d2d/f93 [0,4194304] 0 2024-04-24T08:25:13.407 INFO:tasks.workunit.client.0.smithi090.stdout:2/881: dread d0/d7/d67/d7f/d31/d90/f108 [0,4194304] 0 2024-04-24T08:25:13.409 INFO:tasks.workunit.client.0.smithi090.stdout:8/788: sync 2024-04-24T08:25:13.410 INFO:tasks.workunit.client.1.smithi149.stdout:6/954: dwrite d0/d1a/d28/f12c [0,4194304] 0 2024-04-24T08:25:13.414 INFO:tasks.workunit.client.0.smithi090.stdout:6/859: dwrite d8/d1d/d1f/d3e/f6a [0,4194304] 0 2024-04-24T08:25:13.455 INFO:tasks.workunit.client.0.smithi090.stdout:5/879: mkdir d8/d7c/da9/d118/da6/d11d 0 2024-04-24T08:25:13.456 INFO:tasks.workunit.client.0.smithi090.stdout:4/770: dread d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f93 [0,4194304] 0 2024-04-24T08:25:13.459 INFO:tasks.workunit.client.1.smithi149.stdout:6/955: dread d0/d1a/d28/d31/d4c/d4e/d12e/f3a [4194304,4194304] 0 2024-04-24T08:25:13.460 INFO:tasks.workunit.client.0.smithi090.stdout:8/789: creat d2/da/d13/d64/dcc/f108 x:0 0 0 2024-04-24T08:25:13.461 INFO:tasks.workunit.client.1.smithi149.stdout:6/956: dread - d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f88 zero size 2024-04-24T08:25:13.463 INFO:tasks.workunit.client.0.smithi090.stdout:2/882: symlink d0/da7/l129 0 2024-04-24T08:25:13.464 INFO:tasks.workunit.client.1.smithi149.stdout:6/957: write d0/d1a/d28/d31/d33/fa8 [1459050,122114] 0 2024-04-24T08:25:13.464 INFO:tasks.workunit.client.0.smithi090.stdout:5/880: creat d8/d99/f11e x:0 0 0 2024-04-24T08:25:13.466 INFO:tasks.workunit.client.0.smithi090.stdout:7/900: sync 2024-04-24T08:25:13.469 INFO:tasks.workunit.client.0.smithi090.stdout:7/901: write db/d17/d39/d59/f11b [116149,43040] 0 2024-04-24T08:25:13.469 INFO:tasks.workunit.client.0.smithi090.stdout:3/804: rename d2/df/d1a/d37/f47 to d2/df/f10d 0 2024-04-24T08:25:13.472 INFO:tasks.workunit.client.0.smithi090.stdout:2/883: fsync d0/d7/f33 0 2024-04-24T08:25:13.473 INFO:tasks.workunit.client.0.smithi090.stdout:0/809: dread d1a/d4b/d53/d79/d58/d96/fba [0,4194304] 0 2024-04-24T08:25:13.475 INFO:tasks.workunit.client.0.smithi090.stdout:4/771: getdents d3/d8e/ddf/d5c/db5/d77/dec 0 2024-04-24T08:25:13.478 INFO:tasks.workunit.client.0.smithi090.stdout:8/790: mkdir d2/da/d13/d64/db2/d109 0 2024-04-24T08:25:13.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/795: rename d3/d5/da/l22 to d3/d5/da/d94/da6/d115/d30/d9d/da2/l119 0 2024-04-24T08:25:13.482 INFO:tasks.workunit.client.0.smithi090.stdout:7/902: stat db/d17/d29/cc6 0 2024-04-24T08:25:13.485 INFO:tasks.workunit.client.0.smithi090.stdout:5/881: dwrite d8/da/d14/f17 [0,4194304] 0 2024-04-24T08:25:13.486 INFO:tasks.workunit.client.1.smithi149.stdout:6/958: mkdir d0/d1a/d28/d31/d4c/df8/d146 0 2024-04-24T08:25:13.487 INFO:tasks.workunit.client.0.smithi090.stdout:0/810: chown d1a/d4b/fa0 818 1 2024-04-24T08:25:13.488 INFO:tasks.workunit.client.0.smithi090.stdout:2/884: mkdir d0/d7/d28/daa/ddc/d12a 0 2024-04-24T08:25:13.494 INFO:tasks.workunit.client.1.smithi149.stdout:6/959: mknod d0/d1a/d28/d31/d4c/d80/d6f/d97/c147 0 2024-04-24T08:25:13.500 INFO:tasks.workunit.client.0.smithi090.stdout:1/796: dread - d3/d5/da/d94/da6/d115/d30/d9d/da2/fd6 zero size 2024-04-24T08:25:13.501 INFO:tasks.workunit.client.0.smithi090.stdout:1/797: dread - d3/d5/da/d94/f10d zero size 2024-04-24T08:25:13.502 INFO:tasks.workunit.client.1.smithi149.stdout:0/984: sync 2024-04-24T08:25:13.502 INFO:tasks.workunit.client.1.smithi149.stdout:2/915: sync 2024-04-24T08:25:13.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/882: fdatasync d8/d59/dc7/d7b/ff9 0 2024-04-24T08:25:13.506 INFO:tasks.workunit.client.0.smithi090.stdout:4/772: creat d3/d8e/ddf/d5c/db5/db6/ff7 x:0 0 0 2024-04-24T08:25:13.507 INFO:tasks.workunit.client.0.smithi090.stdout:5/883: dread - d8/d59/dc7/d7a/f100 zero size 2024-04-24T08:25:13.507 INFO:tasks.workunit.client.0.smithi090.stdout:4/773: readlink d3/d8e/ddf/d5c/l9d 0 2024-04-24T08:25:13.509 INFO:tasks.workunit.client.0.smithi090.stdout:4/774: fdatasync d3/d8e/ddf/d5c/db5/f9a 0 2024-04-24T08:25:13.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/775: chown d3/d24/f2b 557856 1 2024-04-24T08:25:13.512 INFO:tasks.workunit.client.0.smithi090.stdout:1/798: fdatasync d3/d5/da/dc/d80/fbc 0 2024-04-24T08:25:13.515 INFO:tasks.workunit.client.1.smithi149.stdout:0/985: read d2/d9/df/d2f/d6c/dc1/fa9 [1776676,119727] 0 2024-04-24T08:25:13.516 INFO:tasks.workunit.client.0.smithi090.stdout:6/860: rename d8/dc to d8/d1d/d1f/d3e/d10a 0 2024-04-24T08:25:13.517 INFO:tasks.workunit.client.0.smithi090.stdout:3/805: getdents d2/df/d1a/d37/d4b 0 2024-04-24T08:25:13.520 INFO:tasks.workunit.client.1.smithi149.stdout:0/986: dread - d2/d9/d1a/d36/d85/dcd/dda/fe0 zero size 2024-04-24T08:25:13.521 INFO:tasks.workunit.client.0.smithi090.stdout:0/811: dwrite f18 [4194304,4194304] 0 2024-04-24T08:25:13.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/812: readlink d1a/d4b/d53/d79/d35/dbd/lc6 0 2024-04-24T08:25:13.527 INFO:tasks.workunit.client.1.smithi149.stdout:0/987: write d2/d9/d1a/d36/f74 [4397141,48674] 0 2024-04-24T08:25:13.529 INFO:tasks.workunit.client.0.smithi090.stdout:2/885: rename d0/d7/d67/d7f/l71 to d0/d7/d67/d7f/d31/d46/d60/l12b 0 2024-04-24T08:25:13.531 INFO:tasks.workunit.client.0.smithi090.stdout:6/861: mknod d8/c10b 0 2024-04-24T08:25:13.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/813: mknod d1a/d4b/d53/d79/d69/d90/d10e/c114 0 2024-04-24T08:25:13.540 INFO:tasks.workunit.client.0.smithi090.stdout:3/806: dwrite d2/df/d1a/d37/ff1 [0,4194304] 0 2024-04-24T08:25:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:13 smithi090 ceph-mon[25016]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 97 MiB/s rd, 241 MiB/s wr, 524 op/s 2024-04-24T08:25:13.546 INFO:tasks.workunit.client.0.smithi090.stdout:9/814: truncate d0/dc/d24/d2d/f65 7406611 0 2024-04-24T08:25:13.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/807: rename d2/df/l6c to d2/df/d1a/d37/d59/dcb/l10e 0 2024-04-24T08:25:13.549 INFO:tasks.workunit.client.0.smithi090.stdout:2/886: dwrite d0/d7/d28/ff3 [0,4194304] 0 2024-04-24T08:25:13.550 INFO:tasks.workunit.client.0.smithi090.stdout:9/815: mkdir d0/dc/d24/d7c/d4d/d8c/d108 0 2024-04-24T08:25:13.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/887: chown d0/d7/d67/d7f/d31/l82 1 1 2024-04-24T08:25:13.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/816: mkdir d0/dc/d24/d2d/d49/d62/def/d109 0 2024-04-24T08:25:13.562 INFO:tasks.workunit.client.0.smithi090.stdout:3/808: rename d2/df/d1a/d37/d4b/c67 to d2/df/d1a/c10f 0 2024-04-24T08:25:13.567 INFO:tasks.workunit.client.0.smithi090.stdout:3/809: fdatasync d2/df/f10 0 2024-04-24T08:25:13.567 INFO:tasks.workunit.client.0.smithi090.stdout:8/791: write d2/d62/d83/da3/fbd [900904,95897] 0 2024-04-24T08:25:13.571 INFO:tasks.workunit.client.1.smithi149.stdout:2/916: write d0/d6/db/d28/d7f/d8f/fbe [313738,61966] 0 2024-04-24T08:25:13.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/903: write db/d17/d29/f42 [1959469,13012] 0 2024-04-24T08:25:13.572 INFO:tasks.workunit.client.0.smithi090.stdout:5/884: write d8/f10 [241842,94984] 0 2024-04-24T08:25:13.574 INFO:tasks.workunit.client.0.smithi090.stdout:8/792: read - d2/da/d13/d64/d24/f92 zero size 2024-04-24T08:25:13.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/793: stat d2/d62/d6b/d9c/dd8/df2 0 2024-04-24T08:25:13.581 INFO:tasks.workunit.client.0.smithi090.stdout:7/904: mknod db/d54/d78/c130 0 2024-04-24T08:25:13.582 INFO:tasks.workunit.client.1.smithi149.stdout:6/960: dwrite d0/d1a/d28/d31/d4c/df8/ffd [0,4194304] 0 2024-04-24T08:25:13.584 INFO:tasks.workunit.client.0.smithi090.stdout:5/885: rmdir d8/da/d14/d2f/d44 39 2024-04-24T08:25:13.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/988: write d2/d9/df/d2f/d6c/dc1/fa9 [3115935,18499] 0 2024-04-24T08:25:13.585 INFO:tasks.workunit.client.0.smithi090.stdout:4/776: dwrite d3/d24/d92/fb0 [0,4194304] 0 2024-04-24T08:25:13.587 INFO:tasks.workunit.client.1.smithi149.stdout:2/917: fsync d0/d6/db/d39/d3d/fb5 0 2024-04-24T08:25:13.587 INFO:tasks.workunit.client.0.smithi090.stdout:3/810: link d2/df/l4a d2/d15/dc4/l110 0 2024-04-24T08:25:13.602 INFO:tasks.workunit.client.0.smithi090.stdout:5/886: fdatasync d8/d25/f3d 0 2024-04-24T08:25:13.602 INFO:tasks.workunit.client.0.smithi090.stdout:6/862: dwrite d8/d1d/d1f/d3e/d10a/d19/f3d [0,4194304] 0 2024-04-24T08:25:13.604 INFO:tasks.workunit.client.0.smithi090.stdout:0/814: dwrite d1a/d4b/d53/d79/d35/f42 [0,4194304] 0 2024-04-24T08:25:13.604 INFO:tasks.workunit.client.1.smithi149.stdout:2/918: creat d0/d6/db/d39/d3d/d44/df0/f117 x:0 0 0 2024-04-24T08:25:13.605 INFO:tasks.workunit.client.0.smithi090.stdout:6/863: write d8/d1d/d29/ffb [164367,64049] 0 2024-04-24T08:25:13.610 INFO:tasks.workunit.client.1.smithi149.stdout:0/989: dwrite d2/d9/df/d2f/d6c/d72/f117 [0,4194304] 0 2024-04-24T08:25:13.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/794: write d2/da/d13/f4b [331822,8461] 0 2024-04-24T08:25:13.628 INFO:tasks.workunit.client.1.smithi149.stdout:2/919: rename d0/d6/db/d27/d64/l84 to d0/d6/db/d28/d7f/d9b/l118 0 2024-04-24T08:25:13.628 INFO:tasks.workunit.client.0.smithi090.stdout:6/864: rmdir d8 39 2024-04-24T08:25:13.632 INFO:tasks.workunit.client.1.smithi149.stdout:0/990: dread d2/d9/df/d40/f70 [0,4194304] 0 2024-04-24T08:25:13.639 INFO:tasks.workunit.client.1.smithi149.stdout:0/991: mknod d2/d9/df/d56/c13b 0 2024-04-24T08:25:13.642 INFO:tasks.workunit.client.1.smithi149.stdout:0/992: chown d2/d9/df/d40/c45 12888211 1 2024-04-24T08:25:13.642 INFO:tasks.workunit.client.0.smithi090.stdout:7/905: getdents db/d17/d29 0 2024-04-24T08:25:13.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/865: mkdir d8/d1d/d1f/d3e/d44/d90/da2/d10c 0 2024-04-24T08:25:13.643 INFO:tasks.workunit.client.1.smithi149.stdout:6/961: dread d0/d1a/f29 [0,4194304] 0 2024-04-24T08:25:13.645 INFO:tasks.workunit.client.0.smithi090.stdout:1/799: sync 2024-04-24T08:25:13.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/993: symlink d2/d9/d1a/d36/d85/db5/def/l13c 0 2024-04-24T08:25:13.649 INFO:tasks.workunit.client.1.smithi149.stdout:0/994: chown d2/d9/cd5 5742653 1 2024-04-24T08:25:13.649 INFO:tasks.workunit.client.0.smithi090.stdout:4/777: getdents d3/d8e/ddf/d5c/db5/d77 0 2024-04-24T08:25:13.650 INFO:tasks.workunit.client.0.smithi090.stdout:7/906: stat db/d17/d29/d38/d99/fef 0 2024-04-24T08:25:13.652 INFO:tasks.workunit.client.1.smithi149.stdout:6/962: dread d0/d1a/d28/d31/f6c [0,4194304] 0 2024-04-24T08:25:13.652 INFO:tasks.workunit.client.1.smithi149.stdout:0/995: fdatasync d2/d9/df/d2f/d6c/dc1/fcf 0 2024-04-24T08:25:13.655 INFO:tasks.workunit.client.1.smithi149.stdout:2/920: dread d0/d6/db/d27/f10a [4194304,4194304] 0 2024-04-24T08:25:13.655 INFO:tasks.workunit.client.0.smithi090.stdout:6/866: chown d8/d1d/d1f/d3e/d10a/fa1 2 1 2024-04-24T08:25:13.656 INFO:tasks.workunit.client.0.smithi090.stdout:4/778: dread - d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/fde zero size 2024-04-24T08:25:13.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/996: symlink d2/l13d 0 2024-04-24T08:25:13.661 INFO:tasks.workunit.client.1.smithi149.stdout:6/963: mkdir d0/d1a/d28/d31/d4c/d4e/d12e/d134/d148 0 2024-04-24T08:25:13.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/964: dread - d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/f88 zero size 2024-04-24T08:25:13.664 INFO:tasks.workunit.client.1.smithi149.stdout:0/997: truncate d2/d9/d1a/d36/d85/dcd/fee 547926 0 2024-04-24T08:25:13.666 INFO:tasks.workunit.client.1.smithi149.stdout:6/965: write d0/d1a/f2b [6001226,33450] 0 2024-04-24T08:25:13.671 INFO:tasks.workunit.client.1.smithi149.stdout:2/921: write d0/d6/db/d28/db0/fc0 [230953,29558] 0 2024-04-24T08:25:13.677 INFO:tasks.workunit.client.1.smithi149.stdout:2/922: symlink d0/d6/db/d39/d3d/l119 0 2024-04-24T08:25:13.684 INFO:tasks.workunit.client.1.smithi149.stdout:2/923: creat d0/d6/db/d27/d64/d66/db1/ddb/f11a x:0 0 0 2024-04-24T08:25:13.686 INFO:tasks.workunit.client.1.smithi149.stdout:0/998: dwrite d2/d9/d1a/d36/f38 [0,4194304] 0 2024-04-24T08:25:13.689 INFO:tasks.workunit.client.1.smithi149.stdout:6/966: dwrite d0/d1a/d28/d31/d4c/d80/d6f/dd7/ff6 [0,4194304] 0 2024-04-24T08:25:13.695 INFO:tasks.workunit.client.1.smithi149.stdout:0/999: rename d2/l3 to d2/d9/d1a/d36/d85/db5/def/l13e 0 2024-04-24T08:25:13.700 INFO:tasks.workunit.client.1.smithi149.stdout:6/967: chown d0/d1a/d28/d31/d4c/d80/d6f/f140 58424 1 2024-04-24T08:25:13.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/968: truncate d0/d1a/d28/d31/f4b 1253441 0 2024-04-24T08:25:13.714 INFO:tasks.workunit.client.1.smithi149.stdout:2/924: dwrite d0/d6/db/d28/f73 [0,4194304] 0 2024-04-24T08:25:13.723 INFO:tasks.workunit.client.1.smithi149.stdout:6/969: dwrite d0/d1a/d28/d31/d4c/d80/d6f/dc2/fd3 [0,4194304] 0 2024-04-24T08:25:13.727 INFO:tasks.workunit.client.1.smithi149.stdout:2/925: link d0/d6/db/d28/d7f/d9b/f106 d0/d6/db/d28/d7f/d8f/db9/ddf/de3/f11b 0 2024-04-24T08:25:13.729 INFO:tasks.workunit.client.1.smithi149.stdout:6/970: creat d0/d1a/d28/d31/d4c/d4e/d62/f149 x:0 0 0 2024-04-24T08:25:13.730 INFO:tasks.workunit.client.1.smithi149.stdout:6/971: fdatasync d0/d1a/f27 0 2024-04-24T08:25:13.733 INFO:tasks.workunit.client.1.smithi149.stdout:2/926: creat d0/d6/db/d39/d3d/d44/d57/f11c x:0 0 0 2024-04-24T08:25:13.742 INFO:tasks.workunit.client.0.smithi090.stdout:0/815: write d1a/d4b/d53/f6a [1307870,106405] 0 2024-04-24T08:25:13.743 INFO:tasks.workunit.client.0.smithi090.stdout:7/907: chown db/d17/d29/d32/d5a/d75/d11f/dae/fb0 766 1 2024-04-24T08:25:13.743 INFO:tasks.workunit.client.1.smithi149.stdout:6/972: rename d0/d1a/d28/d31/d4c/d76/la3 to d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/l14a 0 2024-04-24T08:25:13.749 INFO:tasks.workunit.client.0.smithi090.stdout:3/811: dwrite d2/d15/fab [0,4194304] 0 2024-04-24T08:25:13.749 INFO:tasks.workunit.client.1.smithi149.stdout:2/927: write d0/d6/db/d39/d78/d8a/fee [1158515,66353] 0 2024-04-24T08:25:13.750 INFO:tasks.workunit.client.0.smithi090.stdout:7/908: stat db/d17/d39/ld9 0 2024-04-24T08:25:13.756 INFO:tasks.workunit.client.1.smithi149.stdout:2/928: rmdir d0/d6/db/d39/d3d/d44/d57 39 2024-04-24T08:25:13.761 INFO:tasks.workunit.client.0.smithi090.stdout:1/800: dwrite d3/d5/da/dc/d80/fe4 [0,4194304] 0 2024-04-24T08:25:13.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/812: creat d2/df/d1a/d37/dc0/de6/f111 x:0 0 0 2024-04-24T08:25:13.769 INFO:tasks.workunit.client.1.smithi149.stdout:2/929: creat d0/d6/db/d39/f11d x:0 0 0 2024-04-24T08:25:13.772 INFO:tasks.workunit.client.1.smithi149.stdout:6/973: truncate d0/d1a/d28/d31/f82 1146356 0 2024-04-24T08:25:13.772 INFO:tasks.workunit.client.0.smithi090.stdout:0/816: dwrite d1a/d4b/d53/d79/d58/d96/f10d [0,4194304] 0 2024-04-24T08:25:13.775 INFO:tasks.workunit.client.0.smithi090.stdout:3/813: unlink d2/d25/d2c/l3d 0 2024-04-24T08:25:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:13 smithi149 ceph-mon[31282]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 97 MiB/s rd, 241 MiB/s wr, 524 op/s 2024-04-24T08:25:13.777 INFO:tasks.workunit.client.1.smithi149.stdout:2/930: mkdir d0/d6/db/d28/d7f/d11e 0 2024-04-24T08:25:13.778 INFO:tasks.workunit.client.0.smithi090.stdout:6/867: write d8/d1d/d1f/d3e/d44/d90/d106/d38/f49 [766371,86302] 0 2024-04-24T08:25:13.780 INFO:tasks.workunit.client.0.smithi090.stdout:1/801: creat d3/d5/da/d94/d10b/f11a x:0 0 0 2024-04-24T08:25:13.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/974: read d0/d1a/f25 [2031899,45799] 0 2024-04-24T08:25:13.782 INFO:tasks.workunit.client.0.smithi090.stdout:0/817: unlink d1a/d4b/d53/d79/d35/d8e/fec 0 2024-04-24T08:25:13.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/868: stat d8/d1d/d1f/d3e/d8f/fd0 0 2024-04-24T08:25:13.785 INFO:tasks.workunit.client.0.smithi090.stdout:4/779: dwrite d3/d8e/ddf/d5c/f7f [0,4194304] 0 2024-04-24T08:25:13.785 INFO:tasks.workunit.client.0.smithi090.stdout:3/814: creat d2/df/d1a/d37/d4b/f112 x:0 0 0 2024-04-24T08:25:13.786 INFO:tasks.workunit.client.0.smithi090.stdout:0/818: fsync d1a/d4b/d53/f6a 0 2024-04-24T08:25:13.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/802: chown d3/d5/da/d94/da6/lb0 30159799 1 2024-04-24T08:25:13.789 INFO:tasks.workunit.client.0.smithi090.stdout:7/909: dread db/d54/d83/f8f [0,4194304] 0 2024-04-24T08:25:13.792 INFO:tasks.workunit.client.0.smithi090.stdout:3/815: creat d2/d15/d36/d6f/f113 x:0 0 0 2024-04-24T08:25:13.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/803: fdatasync d3/d5/da/d94/da6/d115/d30/d9d/fd3 0 2024-04-24T08:25:13.798 INFO:tasks.workunit.client.0.smithi090.stdout:1/804: chown d3/d5/da/dc/d80/deb/c107 24 1 2024-04-24T08:25:13.800 INFO:tasks.workunit.client.0.smithi090.stdout:0/819: dread d1a/d4b/d53/d57/fa4 [0,4194304] 0 2024-04-24T08:25:13.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/780: unlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f72 0 2024-04-24T08:25:13.805 INFO:tasks.workunit.client.0.smithi090.stdout:1/805: mkdir d3/d68/d7f/d11b 0 2024-04-24T08:25:13.807 INFO:tasks.workunit.client.0.smithi090.stdout:2/888: sync 2024-04-24T08:25:13.808 INFO:tasks.workunit.client.0.smithi090.stdout:4/781: creat d3/d8e/ddf/d1d/dcb/ff8 x:0 0 0 2024-04-24T08:25:13.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/782: chown d3/d8e/ddf/d5c/db5/c1c 83 1 2024-04-24T08:25:13.811 INFO:tasks.workunit.client.0.smithi090.stdout:2/889: dread d0/d7/d67/d7f/f10e [0,4194304] 0 2024-04-24T08:25:13.812 INFO:tasks.workunit.client.0.smithi090.stdout:2/890: write d0/d7/de1/f43 [5057964,5355] 0 2024-04-24T08:25:13.814 INFO:tasks.workunit.client.0.smithi090.stdout:0/820: symlink d1a/d4b/d53/d79/d35/dea/l115 0 2024-04-24T08:25:13.815 INFO:tasks.workunit.client.1.smithi149.stdout:6/975: write d0/d1a/d28/d31/f4b [798635,103003] 0 2024-04-24T08:25:13.815 INFO:tasks.workunit.client.0.smithi090.stdout:6/869: link d8/d1d/d1f/d3e/d44/d90/d106/f62 d8/d1d/d1f/d3e/d44/d90/da2/d10c/f10d 0 2024-04-24T08:25:13.817 INFO:tasks.workunit.client.0.smithi090.stdout:7/910: creat db/d17/d29/d32/d5a/f131 x:0 0 0 2024-04-24T08:25:13.819 INFO:tasks.workunit.client.0.smithi090.stdout:3/816: rmdir d2/df/d1a/d37/d53/d60/d93 0 2024-04-24T08:25:13.821 INFO:tasks.workunit.client.1.smithi149.stdout:2/931: dwrite d0/d6/db/f1c [0,4194304] 0 2024-04-24T08:25:13.825 INFO:tasks.workunit.client.1.smithi149.stdout:6/976: unlink d0/d1a/d28/d31/d4c/d4e/d62/d66/d7d/fe8 0 2024-04-24T08:25:13.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/806: write d3/d5/da/dc/d80/fbc [221736,1460] 0 2024-04-24T08:25:13.830 INFO:tasks.workunit.client.1.smithi149.stdout:2/932: creat d0/d6/db/d27/d81/db4/f11f x:0 0 0 2024-04-24T08:25:13.832 INFO:tasks.workunit.client.1.smithi149.stdout:6/977: mkdir d0/d1a/d28/d31/df4/d14b 0 2024-04-24T08:25:13.833 INFO:tasks.workunit.client.1.smithi149.stdout:2/933: read - d0/d6/db/d27/d81/db4/f111 zero size 2024-04-24T08:25:13.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/807: dread d3/d5/da/dc/d80/fbc [0,4194304] 0 2024-04-24T08:25:13.835 INFO:tasks.workunit.client.0.smithi090.stdout:1/808: chown d3/d68 40 1 2024-04-24T08:25:13.837 INFO:tasks.workunit.client.0.smithi090.stdout:6/870: mknod d8/d1d/d1f/d3e/d10a/d19/dd2/c10e 0 2024-04-24T08:25:13.838 INFO:tasks.workunit.client.1.smithi149.stdout:6/978: mknod d0/d1a/d28/d31/d4c/d4e/d12e/d106/c14c 0 2024-04-24T08:25:13.838 INFO:tasks.workunit.client.0.smithi090.stdout:3/817: stat d2/d25/d33/lbd 0 2024-04-24T08:25:13.839 INFO:tasks.workunit.client.0.smithi090.stdout:6/871: dread - d8/d1d/d1f/d3e/d10a/d19/dd2/f104 zero size 2024-04-24T08:25:13.842 INFO:tasks.workunit.client.0.smithi090.stdout:6/872: chown d8/d1d/d2f/l43 3 1 2024-04-24T08:25:13.842 INFO:tasks.workunit.client.0.smithi090.stdout:7/911: dread db/d17/d29/d32/d5a/d75/f81 [0,4194304] 0 2024-04-24T08:25:13.842 INFO:tasks.workunit.client.0.smithi090.stdout:3/818: mkdir d2/df/d57/d114 0 2024-04-24T08:25:13.843 INFO:tasks.workunit.client.1.smithi149.stdout:2/934: symlink d0/d6/db/d27/d81/d9f/l120 0 2024-04-24T08:25:13.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/809: mknod d3/d1d/d34/c11c 0 2024-04-24T08:25:13.848 INFO:tasks.workunit.client.1.smithi149.stdout:2/935: dread d0/d6/db/d28/d7f/d8f/fbe [0,4194304] 0 2024-04-24T08:25:13.850 INFO:tasks.workunit.client.0.smithi090.stdout:7/912: creat db/d54/d83/f132 x:0 0 0 2024-04-24T08:25:13.850 INFO:tasks.workunit.client.0.smithi090.stdout:1/810: write d3/d5/da/d94/da6/d115/d30/d9d/fa8 [3511609,87074] 0 2024-04-24T08:25:13.852 INFO:tasks.workunit.client.0.smithi090.stdout:3/819: creat d2/d25/d33/f115 x:0 0 0 2024-04-24T08:25:13.852 INFO:tasks.workunit.client.1.smithi149.stdout:6/979: link d0/d1a/f4d d0/d1a/d28/d31/d4c/d4e/f14d 0 2024-04-24T08:25:13.853 INFO:tasks.workunit.client.0.smithi090.stdout:4/783: write d3/d8e/ddf/d5c/db5/d69/fe7 [104550,61467] 0 2024-04-24T08:25:13.854 INFO:tasks.workunit.client.0.smithi090.stdout:7/913: chown db/d17/d29/d32/d5a/f119 4694943 1 2024-04-24T08:25:13.855 INFO:tasks.workunit.client.0.smithi090.stdout:2/891: link d0/d7/d67/d7f/d31/d46/d60/fa4 d0/d7/dda/f12c 0 2024-04-24T08:25:13.859 INFO:tasks.workunit.client.0.smithi090.stdout:0/821: dwrite d1a/d4b/f61 [0,4194304] 0 2024-04-24T08:25:13.861 INFO:tasks.workunit.client.0.smithi090.stdout:6/873: rename d8/d1d/d29/f8d to d8/d1d/d1f/d3e/d4f/d7e/daa/de4/f10f 0 2024-04-24T08:25:13.862 INFO:tasks.workunit.client.0.smithi090.stdout:0/822: dread - d1a/d4b/d53/d79/d58/db4/f110 zero size 2024-04-24T08:25:13.862 INFO:tasks.workunit.client.1.smithi149.stdout:2/936: truncate d0/d6/db/d39/d78/d7c/df1/f100 838820 0 2024-04-24T08:25:13.865 INFO:tasks.workunit.client.0.smithi090.stdout:2/892: unlink d0/d7/d67/l11c 0 2024-04-24T08:25:13.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/784: mknod d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/cf9 0 2024-04-24T08:25:13.870 INFO:tasks.workunit.client.0.smithi090.stdout:3/820: rename d2/d15/l1c to d2/df/d1a/d37/dc0/de6/dfa/l116 0 2024-04-24T08:25:13.873 INFO:tasks.workunit.client.0.smithi090.stdout:2/893: mkdir d0/d7/d57/d79/d12d 0 2024-04-24T08:25:13.874 INFO:tasks.workunit.client.0.smithi090.stdout:1/811: dwrite d3/d5/f8d [0,4194304] 0 2024-04-24T08:25:13.876 INFO:tasks.workunit.client.0.smithi090.stdout:2/894: stat d0/d7/d67/d7f/d31/d90/f108 0 2024-04-24T08:25:13.877 INFO:tasks.workunit.client.0.smithi090.stdout:0/823: unlink ld 0 2024-04-24T08:25:13.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/785: creat d3/d8e/ddf/d5c/db5/db6/ffa x:0 0 0 2024-04-24T08:25:13.880 INFO:tasks.workunit.client.0.smithi090.stdout:7/914: dread db/f10 [0,4194304] 0 2024-04-24T08:25:13.881 INFO:tasks.workunit.client.1.smithi149.stdout:6/980: truncate d0/d1a/d28/f12c 470330 0 2024-04-24T08:25:13.882 INFO:tasks.workunit.client.1.smithi149.stdout:2/937: link d0/d6/db/d27/d64/fa4 d0/d6/db/d27/d81/db4/f121 0 2024-04-24T08:25:13.882 INFO:tasks.workunit.client.0.smithi090.stdout:8/795: sync 2024-04-24T08:25:13.882 INFO:tasks.workunit.client.0.smithi090.stdout:9/817: sync 2024-04-24T08:25:13.882 INFO:tasks.workunit.client.0.smithi090.stdout:5/887: sync 2024-04-24T08:25:13.883 INFO:tasks.workunit.client.0.smithi090.stdout:1/812: rmdir d3/d5/da/d94/da6/d115/d30/d9d/da2 39 2024-04-24T08:25:13.883 INFO:tasks.workunit.client.1.smithi149.stdout:2/938: readlink d0/d6/db/l32 0 2024-04-24T08:25:13.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/888: chown d8/da/d14/d2f/d112/dcb/d21/f48 12839 1 2024-04-24T08:25:13.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/895: rename d0/d14/d17/lc3 to d0/d7/d28/daa/ddc/l12e 0 2024-04-24T08:25:13.885 INFO:tasks.workunit.client.0.smithi090.stdout:5/889: chown d8/d25/l102 1787105370 1 2024-04-24T08:25:13.886 INFO:tasks.workunit.client.1.smithi149.stdout:2/939: write d0/d6/db/f10 [10568686,9493] 0 2024-04-24T08:25:13.886 INFO:tasks.workunit.client.0.smithi090.stdout:0/824: mkdir d1a/d4b/d53/d79/d82/d9d/d116 0 2024-04-24T08:25:13.888 INFO:tasks.workunit.client.0.smithi090.stdout:4/786: unlink d3/f9 0 2024-04-24T08:25:13.894 INFO:tasks.workunit.client.0.smithi090.stdout:7/915: mknod db/d17/d29/d38/d3e/d46/c133 0 2024-04-24T08:25:13.895 INFO:tasks.workunit.client.1.smithi149.stdout:6/981: symlink d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/d13f/l14e 0 2024-04-24T08:25:13.899 INFO:tasks.workunit.client.0.smithi090.stdout:7/916: truncate db/d17/d29/d32/d5a/d75/f79 1719301 0 2024-04-24T08:25:13.905 INFO:tasks.workunit.client.0.smithi090.stdout:4/787: dwrite d3/d8e/ddf/fd6 [0,4194304] 0 2024-04-24T08:25:13.907 INFO:tasks.workunit.client.0.smithi090.stdout:5/890: creat d8/da/d14/d2f/d112/f11f x:0 0 0 2024-04-24T08:25:13.912 INFO:tasks.workunit.client.0.smithi090.stdout:4/788: dread d3/d8e/ddf/d5c/f66 [0,4194304] 0 2024-04-24T08:25:13.914 INFO:tasks.workunit.client.0.smithi090.stdout:0/825: fsync d1a/d4b/d53/d79/d82/f8d 0 2024-04-24T08:25:13.914 INFO:tasks.workunit.client.0.smithi090.stdout:3/821: link d2/df/d1a/d37/d53/d60/d9b/lec d2/df/d5b/l117 0 2024-04-24T08:25:13.915 INFO:tasks.workunit.client.0.smithi090.stdout:8/796: creat d2/da/d13/d64/db2/d109/f10a x:0 0 0 2024-04-24T08:25:13.915 INFO:tasks.workunit.client.0.smithi090.stdout:4/789: truncate d3/d8e/fb7 1206902 0 2024-04-24T08:25:13.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/822: fdatasync d2/d15/d36/d6f/d9d/fa4 0 2024-04-24T08:25:13.918 INFO:tasks.workunit.client.0.smithi090.stdout:1/813: unlink d3/d5/da/d94/da6/d115/d30/d9d/da2/fb5 0 2024-04-24T08:25:13.918 INFO:tasks.workunit.client.0.smithi090.stdout:3/823: chown d2/d90/fa1 55851681 1 2024-04-24T08:25:13.919 INFO:tasks.workunit.client.0.smithi090.stdout:1/814: stat d3/d5/d3e/da3 0 2024-04-24T08:25:13.922 INFO:tasks.workunit.client.1.smithi149.stdout:2/940: dread d0/d6/db/d39/f8e [0,4194304] 0 2024-04-24T08:25:13.923 INFO:tasks.workunit.client.0.smithi090.stdout:8/797: dread d2/da/d13/d64/db2/fd7 [0,4194304] 0 2024-04-24T08:25:13.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/941: creat d0/d6/db/d27/f122 x:0 0 0 2024-04-24T08:25:13.929 INFO:tasks.workunit.client.0.smithi090.stdout:2/896: write d0/d7/d57/f66 [2744224,37402] 0 2024-04-24T08:25:13.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/942: mknod d0/d6c/c123 0 2024-04-24T08:25:13.936 INFO:tasks.workunit.client.0.smithi090.stdout:7/917: mknod db/d17/d39/c134 0 2024-04-24T08:25:13.939 INFO:tasks.workunit.client.1.smithi149.stdout:6/982: dwrite d0/db7/d112/f105 [0,4194304] 0 2024-04-24T08:25:13.949 INFO:tasks.workunit.client.1.smithi149.stdout:6/983: symlink d0/d1a/d28/d31/d4c/d80/dbd/l14f 0 2024-04-24T08:25:13.949 INFO:tasks.workunit.client.0.smithi090.stdout:3/824: dread d2/d15/f1d [0,4194304] 0 2024-04-24T08:25:13.952 INFO:tasks.workunit.client.0.smithi090.stdout:1/815: mknod d3/d1d/d34/d73/dd8/d109/c11d 0 2024-04-24T08:25:13.952 INFO:tasks.workunit.client.0.smithi090.stdout:7/918: read - db/d17/d29/d38/d99/fef zero size 2024-04-24T08:25:13.953 INFO:tasks.workunit.client.0.smithi090.stdout:1/816: chown d3/d1d/d34/f98 290381 1 2024-04-24T08:25:13.953 INFO:tasks.workunit.client.0.smithi090.stdout:7/919: chown db/d17/d39/d59 802927922 1 2024-04-24T08:25:13.955 INFO:tasks.workunit.client.0.smithi090.stdout:1/817: write d3/d5/da/d97/fd7 [173490,70805] 0 2024-04-24T08:25:13.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/897: mkdir d0/d7/d28/d124/d12f 0 2024-04-24T08:25:13.959 INFO:tasks.workunit.client.0.smithi090.stdout:1/818: truncate d3/d5/d3e/f117 1090245 0 2024-04-24T08:25:13.959 INFO:tasks.workunit.client.0.smithi090.stdout:7/920: read db/d17/d29/f42 [4007982,85797] 0 2024-04-24T08:25:13.959 INFO:tasks.workunit.client.0.smithi090.stdout:4/790: creat d3/d8e/ddf/ffb x:0 0 0 2024-04-24T08:25:13.960 INFO:tasks.workunit.client.0.smithi090.stdout:3/825: creat d2/d90/f118 x:0 0 0 2024-04-24T08:25:13.961 INFO:tasks.workunit.client.0.smithi090.stdout:3/826: readlink d2/df/l2d 0 2024-04-24T08:25:13.961 INFO:tasks.workunit.client.1.smithi149.stdout:6/984: link d0/d1a/d28/d31/d4c/d76/f13d d0/d1a/d28/d31/d4c/d4e/db8/dea/f150 0 2024-04-24T08:25:13.962 INFO:tasks.workunit.client.0.smithi090.stdout:8/798: dread d2/d62/f93 [0,4194304] 0 2024-04-24T08:25:13.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/827: chown d2/d25/f108 0 1 2024-04-24T08:25:13.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/828: stat d2/df/d21/c68 0 2024-04-24T08:25:13.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/898: truncate d0/feb 483223 0 2024-04-24T08:25:13.968 INFO:tasks.workunit.client.0.smithi090.stdout:6/874: sync 2024-04-24T08:25:13.969 INFO:tasks.workunit.client.0.smithi090.stdout:8/799: write d2/da/d13/d64/db2/d109/f10a [258916,92322] 0 2024-04-24T08:25:13.969 INFO:tasks.workunit.client.0.smithi090.stdout:7/921: fdatasync db/f10 0 2024-04-24T08:25:13.975 INFO:tasks.workunit.client.0.smithi090.stdout:0/826: write d1a/d4b/d53/d79/d3c/f91 [597569,35103] 0 2024-04-24T08:25:13.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/827: fsync d1a/d4b/d53/d79/d69/d90/dbc/ff1 0 2024-04-24T08:25:13.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/828: chown d1a/fef 113 1 2024-04-24T08:25:13.982 INFO:tasks.workunit.client.1.smithi149.stdout:2/943: dwrite d0/d6/db/d39/d3d/d44/d57/fed [0,4194304] 0 2024-04-24T08:25:13.984 INFO:tasks.workunit.client.1.smithi149.stdout:6/985: dwrite d0/d1a/d28/d31/d4c/d4e/d62/d7a/fad [4194304,4194304] 0 2024-04-24T08:25:13.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/986: unlink d0/d1a/d28/f2c 0 2024-04-24T08:25:13.994 INFO:tasks.workunit.client.1.smithi149.stdout:2/944: link d0/d6/db/d27/c69 d0/d6/db/d28/d7f/d8f/c124 0 2024-04-24T08:25:13.996 INFO:tasks.workunit.client.1.smithi149.stdout:6/987: mknod d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/dec/c151 0 2024-04-24T08:25:14.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/945: chown d0/d6/db/d28/f2c 0 1 2024-04-24T08:25:14.003 INFO:tasks.workunit.client.1.smithi149.stdout:2/946: chown d0/d6/db/d39/d3d/fd6 4264290 1 2024-04-24T08:25:14.008 INFO:tasks.workunit.client.1.smithi149.stdout:6/988: mkdir d0/d1a/d28/d31/d4c/d4e/dba/d152 0 2024-04-24T08:25:14.009 INFO:tasks.workunit.client.1.smithi149.stdout:2/947: write d0/d6/db/d27/d81/d9f/ffb [757250,106846] 0 2024-04-24T08:25:14.012 INFO:tasks.workunit.client.1.smithi149.stdout:6/989: chown d0/d1a/d28/d31/d4c/d4e/d62/d7a/d93/f94 29865315 1 2024-04-24T08:25:14.016 INFO:tasks.workunit.client.1.smithi149.stdout:2/948: read - d0/d6/db/d27/d81/db4/f121 zero size 2024-04-24T08:25:14.017 INFO:tasks.workunit.client.0.smithi090.stdout:7/922: truncate db/d17/d39/d59/ff0 556243 0 2024-04-24T08:25:14.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/829: chown d1a/d4b/d53/d79/d69/d78/ce4 0 1 2024-04-24T08:25:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/891: truncate d8/d59/dc7/d7a/d50/f53 281016 0 2024-04-24T08:25:14.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/830: chown d1a/d4b/d53/d79/d58/fa5 248861 1 2024-04-24T08:25:14.027 INFO:tasks.workunit.client.0.smithi090.stdout:2/899: dwrite d0/d7/d28/daa/ddc/f109 [0,4194304] 0 2024-04-24T08:25:14.029 INFO:tasks.workunit.client.0.smithi090.stdout:6/875: getdents d8/d1d/d1f 0 2024-04-24T08:25:14.031 INFO:tasks.workunit.client.1.smithi149.stdout:6/990: dwrite d0/f10 [0,4194304] 0 2024-04-24T08:25:14.034 INFO:tasks.workunit.client.0.smithi090.stdout:5/892: read d8/d59/dc7/d7a/d50/f53 [142744,37633] 0 2024-04-24T08:25:14.038 INFO:tasks.workunit.client.0.smithi090.stdout:2/900: creat d0/d5e/de2/f130 x:0 0 0 2024-04-24T08:25:14.038 INFO:tasks.workunit.client.0.smithi090.stdout:6/876: read - d8/d1d/d1f/d3e/d44/d90/d106/d38/fb2 zero size 2024-04-24T08:25:14.039 INFO:tasks.workunit.client.0.smithi090.stdout:0/831: symlink d1a/d4b/d53/l117 0 2024-04-24T08:25:14.040 INFO:tasks.workunit.client.1.smithi149.stdout:2/949: dwrite d0/d6/db/d27/f5f [4194304,4194304] 0 2024-04-24T08:25:14.040 INFO:tasks.workunit.client.1.smithi149.stdout:6/991: write d0/d1a/d28/d31/d4c/d80/d6f/dc2/fd3 [1070732,123394] 0 2024-04-24T08:25:14.046 INFO:tasks.workunit.client.1.smithi149.stdout:2/950: stat d0/d6/db/d27/d64/d66/db1/dc9/d102/cce 0 2024-04-24T08:25:14.047 INFO:tasks.workunit.client.0.smithi090.stdout:1/819: dwrite d3/d5/da/d94/da6/d115/d30/d9d/da2/fde [0,4194304] 0 2024-04-24T08:25:14.048 INFO:tasks.workunit.client.0.smithi090.stdout:4/791: dwrite d3/d8e/ddf/d5c/db5/f68 [0,4194304] 0 2024-04-24T08:25:14.049 INFO:tasks.workunit.client.0.smithi090.stdout:0/832: write d1a/fcd [5728504,40893] 0 2024-04-24T08:25:14.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/992: getdents d0/d1a/d28/d31/d4c/d4e/dba/d152 0 2024-04-24T08:25:14.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/993: stat d0/d1a/d28/d31 0 2024-04-24T08:25:14.059 INFO:tasks.workunit.client.1.smithi149.stdout:6/994: symlink d0/d1a/d28/d31/df4/d13a/l153 0 2024-04-24T08:25:14.059 INFO:tasks.workunit.client.0.smithi090.stdout:4/792: creat d3/d8e/ddf/d5c/db5/d69/ffc x:0 0 0 2024-04-24T08:25:14.060 INFO:tasks.workunit.client.0.smithi090.stdout:0/833: chown d1a/d4b/d53/d79/d69/d8a/leb 3 1 2024-04-24T08:25:14.065 INFO:tasks.workunit.client.0.smithi090.stdout:1/820: rename d3/d5/da/d94/da6/d115/f106 to d3/d1d/d34/d73/dd8/d109/f11e 0 2024-04-24T08:25:14.066 INFO:tasks.workunit.client.1.smithi149.stdout:6/995: creat d0/d1a/d28/d31/d4c/d80/dbd/f154 x:0 0 0 2024-04-24T08:25:14.068 INFO:tasks.workunit.client.0.smithi090.stdout:5/893: write d8/d59/dc7/d7a/da1/fde [140928,32713] 0 2024-04-24T08:25:14.068 INFO:tasks.workunit.client.0.smithi090.stdout:3/829: dread d2/df/d1a/d37/d59/dcb/fd2 [0,4194304] 0 2024-04-24T08:25:14.070 INFO:tasks.workunit.client.1.smithi149.stdout:6/996: mkdir d0/db7/d112/d155 0 2024-04-24T08:25:14.071 INFO:tasks.workunit.client.1.smithi149.stdout:6/997: dread - d0/db7/fda zero size 2024-04-24T08:25:14.073 INFO:tasks.workunit.client.1.smithi149.stdout:6/998: dread - d0/d1a/d28/d31/d4c/d80/d6f/fcb zero size 2024-04-24T08:25:14.076 INFO:tasks.workunit.client.1.smithi149.stdout:6/999: stat d0/d1a/d28/d31/d4c/d4e/d62/fa6 0 2024-04-24T08:25:14.078 INFO:tasks.workunit.client.0.smithi090.stdout:9/818: sync 2024-04-24T08:25:14.079 INFO:tasks.workunit.client.0.smithi090.stdout:3/830: write d2/d25/d33/f80 [3020481,82958] 0 2024-04-24T08:25:14.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/877: dwrite d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/ffe [0,4194304] 0 2024-04-24T08:25:14.094 INFO:tasks.workunit.client.0.smithi090.stdout:1/821: rename d3/d5/da/d94/da6/d115/d30/dc5 to d3/d5/d58/d11f 0 2024-04-24T08:25:14.095 INFO:tasks.workunit.client.0.smithi090.stdout:4/793: dwrite d3/d24/f2b [4194304,4194304] 0 2024-04-24T08:25:14.098 INFO:tasks.workunit.client.0.smithi090.stdout:9/819: creat d0/dc/d24/d7c/d4d/f10a x:0 0 0 2024-04-24T08:25:14.101 INFO:tasks.workunit.client.0.smithi090.stdout:0/834: dwrite d1a/d4b/f10f [0,4194304] 0 2024-04-24T08:25:14.105 INFO:tasks.workunit.client.0.smithi090.stdout:1/822: chown d3/d5/da/d94/da6/d115/c66 8864315 1 2024-04-24T08:25:14.110 INFO:tasks.workunit.client.0.smithi090.stdout:9/820: chown d0/dc/d24/d2d/d49/d62/dcb/f104 282046037 1 2024-04-24T08:25:14.111 INFO:tasks.workunit.client.0.smithi090.stdout:4/794: mkdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/dfd 0 2024-04-24T08:25:14.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/894: dwrite d8/d59/dc7/d7a/d50/ffc [0,4194304] 0 2024-04-24T08:25:14.119 INFO:tasks.workunit.client.0.smithi090.stdout:1/823: rmdir d3/d5/da/d94/da6/d115/d30/d9d/da2 39 2024-04-24T08:25:14.121 INFO:tasks.workunit.client.0.smithi090.stdout:1/824: stat d3/d5/da/d94/f92 0 2024-04-24T08:25:14.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/895: rmdir d8/d59/dc7/d7a/d50/d9f 39 2024-04-24T08:25:14.125 INFO:tasks.workunit.client.0.smithi090.stdout:5/896: fdatasync d8/da/d14/fd3 0 2024-04-24T08:25:14.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/835: dwrite d1a/d4b/d53/d79/d69/d90/dbc/f103 [0,4194304] 0 2024-04-24T08:25:14.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/800: sync 2024-04-24T08:25:14.136 INFO:tasks.workunit.client.0.smithi090.stdout:4/795: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/f60 [1468807,22676] 0 2024-04-24T08:25:14.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/831: dread d2/d25/d33/f80 [0,4194304] 0 2024-04-24T08:25:14.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/801: fdatasync d2/d62/d6b/f7d 0 2024-04-24T08:25:14.140 INFO:tasks.workunit.client.0.smithi090.stdout:9/821: truncate d0/d19/f1a 2055958 0 2024-04-24T08:25:14.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/897: write d8/da/d14/d2f/d112/dcb/f22 [5358306,79848] 0 2024-04-24T08:25:14.145 INFO:tasks.workunit.client.0.smithi090.stdout:0/836: mknod d1a/c118 0 2024-04-24T08:25:14.149 INFO:tasks.workunit.client.0.smithi090.stdout:6/878: dread d8/d1d/d1f/d8c/fdf [0,4194304] 0 2024-04-24T08:25:14.149 INFO:tasks.workunit.client.0.smithi090.stdout:3/832: symlink d2/d25/d33/dc8/l119 0 2024-04-24T08:25:14.150 INFO:tasks.workunit.client.0.smithi090.stdout:4/796: dread d3/d24/f2b [4194304,4194304] 0 2024-04-24T08:25:14.150 INFO:tasks.workunit.client.0.smithi090.stdout:5/898: creat d8/da/d14/d2f/d112/f120 x:0 0 0 2024-04-24T08:25:14.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/833: chown d2/d15/d36/d6f/d9d/l9f 0 1 2024-04-24T08:25:14.156 INFO:tasks.workunit.client.0.smithi090.stdout:5/899: write d8/d59/dc7/d7a/da1/fde [707583,63656] 0 2024-04-24T08:25:14.160 INFO:tasks.workunit.client.0.smithi090.stdout:6/879: rename d8/d1d/d1f/d3e/d4f/d7e/daa/cf5 to d8/d1d/d2f/d85/c110 0 2024-04-24T08:25:14.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/822: write d0/dc/d24/d7c/d4d/d32/ff3 [4007089,15011] 0 2024-04-24T08:25:14.168 INFO:tasks.workunit.client.0.smithi090.stdout:4/797: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f35 [1358063,16902] 0 2024-04-24T08:25:14.170 INFO:tasks.workunit.client.0.smithi090.stdout:5/900: write d8/f9 [986544,121087] 0 2024-04-24T08:25:14.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/837: dwrite f18 [0,4194304] 0 2024-04-24T08:25:14.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/823: symlink d0/d19/dfc/l10b 0 2024-04-24T08:25:14.176 INFO:tasks.workunit.client.0.smithi090.stdout:4/798: symlink d3/d87/lfe 0 2024-04-24T08:25:14.180 INFO:tasks.workunit.client.0.smithi090.stdout:7/923: sync 2024-04-24T08:25:14.180 INFO:tasks.workunit.client.0.smithi090.stdout:1/825: dread d3/d5/da/d97/d9c/fe2 [0,4194304] 0 2024-04-24T08:25:14.181 INFO:tasks.workunit.client.0.smithi090.stdout:5/901: fsync d8/da/d14/d2f/d112/dcb/d21/fee 0 2024-04-24T08:25:14.185 INFO:tasks.workunit.client.0.smithi090.stdout:7/924: fdatasync db/d54/d83/f117 0 2024-04-24T08:25:14.185 INFO:tasks.workunit.client.0.smithi090.stdout:8/802: dread d2/d62/d83/faf [0,4194304] 0 2024-04-24T08:25:14.186 INFO:tasks.workunit.client.0.smithi090.stdout:3/834: dread d2/f14 [0,4194304] 0 2024-04-24T08:25:14.189 INFO:tasks.workunit.client.0.smithi090.stdout:4/799: creat d3/d8e/ddf/d5c/db5/d7c/fff x:0 0 0 2024-04-24T08:25:14.194 INFO:tasks.workunit.client.0.smithi090.stdout:5/902: creat d8/d25/f121 x:0 0 0 2024-04-24T08:25:14.194 INFO:tasks.workunit.client.0.smithi090.stdout:1/826: unlink d3/d5/da/dc/df4/lf7 0 2024-04-24T08:25:14.194 INFO:tasks.workunit.client.0.smithi090.stdout:7/925: fsync db/d17/d29/d32/d5a/f8d 0 2024-04-24T08:25:14.196 INFO:tasks.workunit.client.0.smithi090.stdout:6/880: dread d8/d1d/d1f/f58 [0,4194304] 0 2024-04-24T08:25:14.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/838: dwrite d1a/d4b/f113 [0,4194304] 0 2024-04-24T08:25:14.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/903: mknod d8/d99/c122 0 2024-04-24T08:25:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:7/926: unlink db/f15 0 2024-04-24T08:25:14.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/881: stat d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/cab 0 2024-04-24T08:25:14.204 INFO:tasks.workunit.client.0.smithi090.stdout:5/904: write d8/da/d14/d2f/d112/dcb/ff7 [938713,48897] 0 2024-04-24T08:25:14.204 INFO:tasks.workunit.client.0.smithi090.stdout:3/835: creat d2/df/d5b/d105/f11a x:0 0 0 2024-04-24T08:25:14.205 INFO:tasks.workunit.client.0.smithi090.stdout:0/839: mkdir d1a/d4b/d53/d79/d35/d8e/d119 0 2024-04-24T08:25:14.208 INFO:tasks.workunit.client.0.smithi090.stdout:8/803: link d2/da/d13/d64/d24/dad/f102 d2/da/d13/d64/d50/f10b 0 2024-04-24T08:25:14.209 INFO:tasks.workunit.client.0.smithi090.stdout:7/927: chown db/d17/d29/d32/d5a/d75/lb9 4648198 1 2024-04-24T08:25:14.209 INFO:tasks.workunit.client.0.smithi090.stdout:9/824: truncate d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f9d 1030117 0 2024-04-24T08:25:14.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/836: write d2/d25/f108 [977982,6089] 0 2024-04-24T08:25:14.210 INFO:tasks.workunit.client.0.smithi090.stdout:5/905: creat d8/da/f123 x:0 0 0 2024-04-24T08:25:14.216 INFO:tasks.workunit.client.0.smithi090.stdout:5/906: symlink d8/d99/l124 0 2024-04-24T08:25:14.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/837: write d2/df/d1a/d37/d4b/f112 [859734,8109] 0 2024-04-24T08:25:14.218 INFO:tasks.workunit.client.0.smithi090.stdout:8/804: chown d2/d62/d83/da3/ld4 453383595 1 2024-04-24T08:25:14.219 INFO:tasks.workunit.client.0.smithi090.stdout:0/840: getdents d1a/d4b/d53/d79/d35/dea 0 2024-04-24T08:25:14.223 INFO:tasks.workunit.client.0.smithi090.stdout:9/825: mknod d0/dc/d24/d7c/d4d/d32/d9f/d72/c10c 0 2024-04-24T08:25:14.223 INFO:tasks.workunit.client.0.smithi090.stdout:1/827: dwrite d3/d5/da/dc/d80/fe4 [0,4194304] 0 2024-04-24T08:25:14.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/907: creat d8/d59/dc7/d7a/f125 x:0 0 0 2024-04-24T08:25:14.226 INFO:tasks.workunit.client.0.smithi090.stdout:8/805: stat d2/da/d13/d64/d50/f10b 0 2024-04-24T08:25:14.227 INFO:tasks.workunit.client.0.smithi090.stdout:3/838: chown d2/df/d1a/d37/d4b 18143 1 2024-04-24T08:25:14.227 INFO:tasks.workunit.client.0.smithi090.stdout:4/800: dread d3/d8e/f97 [0,4194304] 0 2024-04-24T08:25:14.229 INFO:tasks.workunit.client.0.smithi090.stdout:5/908: stat d8/da/d3a/d2d/l115 0 2024-04-24T08:25:14.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/841: rename d1a/c31 to d1a/d4b/d53/d79/d69/d90/df2/c11a 0 2024-04-24T08:25:14.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/882: dwrite d8/d1d/d1f/fac [0,4194304] 0 2024-04-24T08:25:14.237 INFO:tasks.workunit.client.0.smithi090.stdout:3/839: mknod d2/df/d1a/d37/d59/dcb/c11b 0 2024-04-24T08:25:14.240 INFO:tasks.workunit.client.0.smithi090.stdout:8/806: truncate d2/da/d13/d64/f6d 142986 0 2024-04-24T08:25:14.241 INFO:tasks.workunit.client.0.smithi090.stdout:0/842: mknod d1a/d4b/d53/d57/c11b 0 2024-04-24T08:25:14.252 INFO:tasks.workunit.client.0.smithi090.stdout:1/828: dwrite d3/d68/d7f/fda [4194304,4194304] 0 2024-04-24T08:25:14.256 INFO:tasks.workunit.client.0.smithi090.stdout:0/843: creat d1a/d4b/d53/d79/d82/d9d/d116/f11c x:0 0 0 2024-04-24T08:25:14.257 INFO:tasks.workunit.client.0.smithi090.stdout:3/840: write d2/d15/fbf [1617356,31943] 0 2024-04-24T08:25:14.260 INFO:tasks.workunit.client.0.smithi090.stdout:7/928: dread db/dbd/f116 [0,4194304] 0 2024-04-24T08:25:14.262 INFO:tasks.workunit.client.0.smithi090.stdout:1/829: truncate d3/fb4 788236 0 2024-04-24T08:25:14.263 INFO:tasks.workunit.client.0.smithi090.stdout:5/909: dwrite d8/d76/dad/fd1 [0,4194304] 0 2024-04-24T08:25:14.264 INFO:tasks.workunit.client.0.smithi090.stdout:1/830: chown d3/f4 230590073 1 2024-04-24T08:25:14.264 INFO:tasks.workunit.client.0.smithi090.stdout:4/801: dwrite d3/d8e/ddf/d1d/f46 [4194304,4194304] 0 2024-04-24T08:25:14.266 INFO:tasks.workunit.client.0.smithi090.stdout:3/841: stat d2/df/d57/d4f/fe8 0 2024-04-24T08:25:14.268 INFO:tasks.workunit.client.0.smithi090.stdout:5/910: fdatasync d8/d59/dc7/d7a/f125 0 2024-04-24T08:25:14.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/883: dwrite d8/d1d/d1f/d3e/d8f/fcc [0,4194304] 0 2024-04-24T08:25:14.271 INFO:tasks.workunit.client.0.smithi090.stdout:0/844: dread - d1a/d4b/d53/d79/d58/d96/fb1 zero size 2024-04-24T08:25:14.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/831: rmdir d3/d5/da/d97/d9c 39 2024-04-24T08:25:14.276 INFO:tasks.workunit.client.0.smithi090.stdout:1/832: write d3/d5/da/d97/fbf [3592860,81857] 0 2024-04-24T08:25:14.277 INFO:tasks.workunit.client.0.smithi090.stdout:5/911: mknod d8/da/d14/d2f/d44/c126 0 2024-04-24T08:25:14.278 INFO:tasks.workunit.client.0.smithi090.stdout:0/845: rmdir d1a/d4b/d53/d79 39 2024-04-24T08:25:14.283 INFO:tasks.workunit.client.0.smithi090.stdout:6/884: symlink d8/d1d/d1f/d3e/d44/d90/d106/d38/dfd/l111 0 2024-04-24T08:25:14.284 INFO:tasks.workunit.client.0.smithi090.stdout:7/929: write db/d90/faf [2477503,97351] 0 2024-04-24T08:25:14.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/833: symlink d3/d5/da/l120 0 2024-04-24T08:25:14.288 INFO:tasks.workunit.client.0.smithi090.stdout:0/846: mkdir d1a/d11d 0 2024-04-24T08:25:14.295 INFO:tasks.workunit.client.0.smithi090.stdout:3/842: dwrite d2/df/d57/d4f/f72 [0,4194304] 0 2024-04-24T08:25:14.296 INFO:tasks.workunit.client.0.smithi090.stdout:7/930: chown db/d17/d29/d32/d5a/d75/f79 2 1 2024-04-24T08:25:14.298 INFO:tasks.workunit.client.0.smithi090.stdout:8/807: dread d2/f10 [0,4194304] 0 2024-04-24T08:25:14.298 INFO:tasks.workunit.client.0.smithi090.stdout:1/834: symlink d3/d1d/d54/l121 0 2024-04-24T08:25:14.299 INFO:tasks.workunit.client.0.smithi090.stdout:3/843: fsync d2/d25/d33/f81 0 2024-04-24T08:25:14.300 INFO:tasks.workunit.client.0.smithi090.stdout:0/847: dread - d1a/d4b/d53/d79/d58/d96/fb1 zero size 2024-04-24T08:25:14.301 INFO:tasks.workunit.client.0.smithi090.stdout:5/912: truncate d8/da/d3a/f43 1296068 0 2024-04-24T08:25:14.305 INFO:tasks.workunit.client.0.smithi090.stdout:4/802: dwrite d3/d8e/ddf/d5c/db5/f36 [0,4194304] 0 2024-04-24T08:25:14.306 INFO:tasks.workunit.client.0.smithi090.stdout:7/931: dread db/d90/f91 [0,4194304] 0 2024-04-24T08:25:14.311 INFO:tasks.workunit.client.0.smithi090.stdout:7/932: chown db/d17/d29/d38/d99/dfb/cfd 147050 1 2024-04-24T08:25:14.311 INFO:tasks.workunit.client.0.smithi090.stdout:0/848: creat d1a/d4b/d53/d79/d58/d96/f11e x:0 0 0 2024-04-24T08:25:14.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/849: readlink d1a/l45 0 2024-04-24T08:25:14.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/803: creat d3/d8e/ddf/d5c/db5/d7c/f100 x:0 0 0 2024-04-24T08:25:14.315 INFO:tasks.workunit.client.0.smithi090.stdout:5/913: mknod d8/da/d14/d2f/d112/dcb/c127 0 2024-04-24T08:25:14.315 INFO:tasks.workunit.client.0.smithi090.stdout:7/933: symlink db/d17/d29/d38/d3e/l135 0 2024-04-24T08:25:14.316 INFO:tasks.workunit.client.0.smithi090.stdout:5/914: dread - d8/da/d3a/d2d/f116 zero size 2024-04-24T08:25:14.320 INFO:tasks.workunit.client.0.smithi090.stdout:1/835: creat d3/d5/da/dc/d80/deb/d111/f122 x:0 0 0 2024-04-24T08:25:14.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/850: rename d1a/f1e to d1a/d4b/d53/d79/d82/d9d/f11f 0 2024-04-24T08:25:14.408 INFO:tasks.workunit.client.0.smithi090.stdout:6/885: write d8/d1d/d1f/d3e/d44/d90/d106/fd3 [546435,52806] 0 2024-04-24T08:25:14.410 INFO:tasks.workunit.client.0.smithi090.stdout:8/808: truncate d2/da/d13/d64/d24/db3/fc2 1844611 0 2024-04-24T08:25:14.412 INFO:tasks.workunit.client.0.smithi090.stdout:0/851: chown f18 0 1 2024-04-24T08:25:14.425 INFO:tasks.workunit.client.0.smithi090.stdout:5/915: dwrite d8/da/d3a/f55 [0,4194304] 0 2024-04-24T08:25:14.428 INFO:tasks.workunit.client.0.smithi090.stdout:5/916: stat d8/da/d14/d2f/d112/dcb/d21/le2 0 2024-04-24T08:25:14.430 INFO:tasks.workunit.client.0.smithi090.stdout:4/804: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/f101 x:0 0 0 2024-04-24T08:25:14.435 INFO:tasks.workunit.client.0.smithi090.stdout:1/836: creat d3/d68/d7f/d11b/f123 x:0 0 0 2024-04-24T08:25:14.435 INFO:tasks.workunit.client.0.smithi090.stdout:7/934: link db/d17/d39/ld9 db/d17/d29/l136 0 2024-04-24T08:25:14.437 INFO:tasks.workunit.client.0.smithi090.stdout:6/886: symlink d8/d1d/d1f/d3e/d44/d90/da2/d10c/l112 0 2024-04-24T08:25:14.442 INFO:tasks.workunit.client.0.smithi090.stdout:4/805: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/f8d [882595,100987] 0 2024-04-24T08:25:14.443 INFO:tasks.workunit.client.0.smithi090.stdout:2/901: dread d0/d7/d67/d7f/d31/f56 [0,4194304] 0 2024-04-24T08:25:14.445 INFO:tasks.workunit.client.0.smithi090.stdout:2/902: chown d0/d14 11344 1 2024-04-24T08:25:14.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/887: creat d8/d1d/d2f/d85/f113 x:0 0 0 2024-04-24T08:25:14.446 INFO:tasks.workunit.client.0.smithi090.stdout:4/806: chown d3/d8e/ddf/d5c/db5/l31 111082 1 2024-04-24T08:25:14.449 INFO:tasks.workunit.client.0.smithi090.stdout:9/826: dread d0/dc/d24/d2d/d49/d81/f9a [0,4194304] 0 2024-04-24T08:25:14.449 INFO:tasks.workunit.client.0.smithi090.stdout:6/888: read d8/d1d/fcf [1099680,125679] 0 2024-04-24T08:25:14.450 INFO:tasks.workunit.client.0.smithi090.stdout:9/827: dread - d0/dc/f64 zero size 2024-04-24T08:25:14.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/917: rename d8/d76/fcf to d8/d59/dc7/d7a/d50/d9f/f128 0 2024-04-24T08:25:14.454 INFO:tasks.workunit.client.0.smithi090.stdout:1/837: creat d3/d5/da/d97/d9c/dc4/dcc/f124 x:0 0 0 2024-04-24T08:25:14.464 INFO:tasks.workunit.client.0.smithi090.stdout:5/918: symlink d8/da/d14/d2f/d112/da2/l129 0 2024-04-24T08:25:14.464 INFO:tasks.workunit.client.0.smithi090.stdout:3/844: sync 2024-04-24T08:25:14.465 INFO:tasks.workunit.client.0.smithi090.stdout:3/845: rename d2 to d2/df/d21/d11c 22 2024-04-24T08:25:14.466 INFO:tasks.workunit.client.0.smithi090.stdout:2/903: mknod d0/d14/d36/dfe/c131 0 2024-04-24T08:25:14.467 INFO:tasks.workunit.client.0.smithi090.stdout:1/838: truncate d3/d5/f28 2043644 0 2024-04-24T08:25:14.471 INFO:tasks.workunit.client.0.smithi090.stdout:4/807: mkdir d3/d8e/ddf/d5c/db5/d77/dec/d102 0 2024-04-24T08:25:14.490 INFO:tasks.workunit.client.0.smithi090.stdout:6/889: creat d8/d1d/d1f/d3e/d10a/f114 x:0 0 0 2024-04-24T08:25:14.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/890: chown d8/d1d/d1f/d3e/fd6 108 1 2024-04-24T08:25:14.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/809: write d2/da/f35 [2249756,7510] 0 2024-04-24T08:25:14.505 INFO:tasks.workunit.client.0.smithi090.stdout:3/846: link d2/d25/d33/c46 d2/d84/c11d 0 2024-04-24T08:25:14.509 INFO:tasks.workunit.client.0.smithi090.stdout:0/852: dwrite d1a/d4b/d53/d79/d35/d62/fc3 [0,4194304] 0 2024-04-24T08:25:14.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/919: chown d8/d59/dc7/d7a/d50/d9f/f128 7 1 2024-04-24T08:25:14.513 INFO:tasks.workunit.client.0.smithi090.stdout:1/839: truncate d3/f4 1963192 0 2024-04-24T08:25:14.513 INFO:tasks.workunit.client.0.smithi090.stdout:3/847: dread d2/df/d1a/f6d [0,4194304] 0 2024-04-24T08:25:14.516 INFO:tasks.workunit.client.1.smithi149.stdout:2/951: sync 2024-04-24T08:25:14.520 INFO:tasks.workunit.client.0.smithi090.stdout:8/810: stat d2/l11 0 2024-04-24T08:25:14.524 INFO:tasks.workunit.client.0.smithi090.stdout:2/904: dread d0/d7/d28/daa/def/d50/d96/f98 [0,4194304] 0 2024-04-24T08:25:14.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/905: write d0/d7/de1/f114 [2709631,22373] 0 2024-04-24T08:25:14.527 INFO:tasks.workunit.client.0.smithi090.stdout:6/891: dwrite d8/d1d/fcf [0,4194304] 0 2024-04-24T08:25:14.535 INFO:tasks.workunit.client.0.smithi090.stdout:7/935: rename db/d17/d29/c12e to db/d17/d29/d38/d99/c137 0 2024-04-24T08:25:14.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/808: rmdir d3/d8e 39 2024-04-24T08:25:14.546 INFO:tasks.workunit.client.0.smithi090.stdout:1/840: stat d3/d5/da/d94/da6/d115/d30/d9d/da2/l119 0 2024-04-24T08:25:14.547 INFO:tasks.workunit.client.0.smithi090.stdout:9/828: dwrite d0/d19/f1a [0,4194304] 0 2024-04-24T08:25:14.548 INFO:tasks.workunit.client.0.smithi090.stdout:1/841: chown d3/d1d/d34/d5a/cdc 4133 1 2024-04-24T08:25:14.548 INFO:tasks.workunit.client.0.smithi090.stdout:2/906: mkdir d0/d14/d36/dce/dee/d132 0 2024-04-24T08:25:14.556 INFO:tasks.workunit.client.0.smithi090.stdout:4/809: write d3/fc1 [952689,12458] 0 2024-04-24T08:25:14.557 INFO:tasks.workunit.client.0.smithi090.stdout:0/853: creat d1a/d4b/d53/d79/d3c/f120 x:0 0 0 2024-04-24T08:25:14.561 INFO:tasks.workunit.client.0.smithi090.stdout:0/854: write d1a/d4b/d53/d79/d35/d62/fab [129353,123102] 0 2024-04-24T08:25:14.564 INFO:tasks.workunit.client.1.smithi149.stdout:2/952: symlink d0/d6/db/d27/d81/l125 0 2024-04-24T08:25:14.565 INFO:tasks.workunit.client.0.smithi090.stdout:6/892: creat d8/d1d/d29/d102/f115 x:0 0 0 2024-04-24T08:25:14.566 INFO:tasks.workunit.client.0.smithi090.stdout:9/829: dwrite d0/dc/d24/d7c/d4d/d8c/fdd [0,4194304] 0 2024-04-24T08:25:14.567 INFO:tasks.workunit.client.0.smithi090.stdout:5/920: link d8/da/d3a/d2d/d37/l54 d8/d59/dc7/d7a/dbc/l12a 0 2024-04-24T08:25:14.570 INFO:tasks.workunit.client.1.smithi149.stdout:2/953: truncate d0/d6/db/f1f 1623994 0 2024-04-24T08:25:14.573 INFO:tasks.workunit.client.0.smithi090.stdout:1/842: mknod d3/d5/c125 0 2024-04-24T08:25:14.577 INFO:tasks.workunit.client.1.smithi149.stdout:2/954: symlink d0/d6/db/d27/d81/d9f/dba/l126 0 2024-04-24T08:25:14.580 INFO:tasks.workunit.client.0.smithi090.stdout:6/893: symlink d8/d1d/d1f/d3e/d44/d90/da2/l116 0 2024-04-24T08:25:14.580 INFO:tasks.workunit.client.0.smithi090.stdout:5/921: readlink d8/da/d14/d2f/d112/dcb/l2a 0 2024-04-24T08:25:14.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/810: symlink d3/d24/l103 0 2024-04-24T08:25:14.587 INFO:tasks.workunit.client.0.smithi090.stdout:2/907: rename d0/d14/d36/dce/dee to d0/d7/d28/daa/ddc/d12a/d133 0 2024-04-24T08:25:14.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/830: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/f6f [0,4194304] 0 2024-04-24T08:25:14.589 INFO:tasks.workunit.client.1.smithi149.stdout:2/955: dread d0/d6/db/d39/d3d/d44/d75/fff [0,4194304] 0 2024-04-24T08:25:14.589 INFO:tasks.workunit.client.0.smithi090.stdout:1/843: creat d3/d68/d7f/f126 x:0 0 0 2024-04-24T08:25:14.591 INFO:tasks.workunit.client.0.smithi090.stdout:6/894: rmdir d8/d1d/d1f/d3e/d8f 39 2024-04-24T08:25:14.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/811: mknod d3/d87/da6/c104 0 2024-04-24T08:25:14.592 INFO:tasks.workunit.client.0.smithi090.stdout:6/895: stat d8/d1d/d1f/d3e/d44/d90/d106/d38 0 2024-04-24T08:25:14.593 INFO:tasks.workunit.client.0.smithi090.stdout:6/896: stat d8/c77 0 2024-04-24T08:25:14.594 INFO:tasks.workunit.client.0.smithi090.stdout:5/922: rename d8/d59/dc7/d7a/da1/f108 to d8/da/d14/d2f/d112/dcb/f12b 0 2024-04-24T08:25:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:5/923: readlink d8/d99/lbb 0 2024-04-24T08:25:14.597 INFO:tasks.workunit.client.0.smithi090.stdout:7/936: symlink db/d17/d29/d32/d5a/d75/l138 0 2024-04-24T08:25:14.598 INFO:tasks.workunit.client.0.smithi090.stdout:9/831: creat d0/dc/d24/d2d/d49/d62/def/f10d x:0 0 0 2024-04-24T08:25:14.599 INFO:tasks.workunit.client.0.smithi090.stdout:9/832: dread - d0/dc/f64 zero size 2024-04-24T08:25:14.601 INFO:tasks.workunit.client.0.smithi090.stdout:6/897: creat d8/d1d/d1f/d3e/d44/d90/da2/d10c/f117 x:0 0 0 2024-04-24T08:25:14.603 INFO:tasks.workunit.client.0.smithi090.stdout:5/924: mkdir d8/da/d14/d2f/d112/dcb/d21/d4c/d12c 0 2024-04-24T08:25:14.605 INFO:tasks.workunit.client.0.smithi090.stdout:0/855: write d1a/d4b/d53/d79/d58/fa5 [589366,63563] 0 2024-04-24T08:25:14.607 INFO:tasks.workunit.client.0.smithi090.stdout:5/925: chown d8/da/d14/d2f/d112/dcb/d21/d4c/f109 12 1 2024-04-24T08:25:14.609 INFO:tasks.workunit.client.0.smithi090.stdout:6/898: chown d8/d1d/d1f/d3e/d10a/d19/l6b 7 1 2024-04-24T08:25:14.609 INFO:tasks.workunit.client.0.smithi090.stdout:2/908: creat d0/d7/d67/d7f/d31/f134 x:0 0 0 2024-04-24T08:25:14.610 INFO:tasks.workunit.client.0.smithi090.stdout:0/856: dread d1a/d4b/d53/d79/d3c/fad [0,4194304] 0 2024-04-24T08:25:14.613 INFO:tasks.workunit.client.0.smithi090.stdout:6/899: write d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/ff7 [483982,62633] 0 2024-04-24T08:25:14.614 INFO:tasks.workunit.client.0.smithi090.stdout:1/844: rename d3/d5/da/d94/da6/d115/c8a to d3/d5/da/d94/da6/d115/d30/d9d/c127 0 2024-04-24T08:25:14.615 INFO:tasks.workunit.client.0.smithi090.stdout:9/833: creat d0/d2a/d106/f10e x:0 0 0 2024-04-24T08:25:14.617 INFO:tasks.workunit.client.0.smithi090.stdout:5/926: mknod d8/d9c/c12d 0 2024-04-24T08:25:14.618 INFO:tasks.workunit.client.1.smithi149.stdout:2/956: write d0/d6/db/d39/d3d/fb6 [2171947,45037] 0 2024-04-24T08:25:14.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/812: rmdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/dfd 0 2024-04-24T08:25:14.621 INFO:tasks.workunit.client.1.smithi149.stdout:2/957: stat d0/d6c/c123 0 2024-04-24T08:25:14.622 INFO:tasks.workunit.client.0.smithi090.stdout:1/845: mknod d3/d5/d58/d11f/c128 0 2024-04-24T08:25:14.623 INFO:tasks.workunit.client.0.smithi090.stdout:6/900: symlink d8/d1d/d1f/d3e/d10a/d19/dea/l118 0 2024-04-24T08:25:14.623 INFO:tasks.workunit.client.0.smithi090.stdout:8/811: sync 2024-04-24T08:25:14.623 INFO:tasks.workunit.client.0.smithi090.stdout:4/813: write d3/d8e/ddf/d1d/dcb/ff8 [674424,18289] 0 2024-04-24T08:25:14.624 INFO:tasks.workunit.client.0.smithi090.stdout:3/848: sync 2024-04-24T08:25:14.624 INFO:tasks.workunit.client.0.smithi090.stdout:2/909: dread d0/d7/d67/d7f/d31/f56 [0,4194304] 0 2024-04-24T08:25:14.625 INFO:tasks.workunit.client.0.smithi090.stdout:9/834: unlink d0/dc/d24/d7c/d4d/d32/l83 0 2024-04-24T08:25:14.627 INFO:tasks.workunit.client.0.smithi090.stdout:5/927: mknod d8/da/d14/d2f/d112/dcb/d21/d4c/c12e 0 2024-04-24T08:25:14.627 INFO:tasks.workunit.client.0.smithi090.stdout:1/846: mkdir d3/d68/df2/d129 0 2024-04-24T08:25:14.628 INFO:tasks.workunit.client.0.smithi090.stdout:8/812: dread - d2/da/d13/d64/d24/f92 zero size 2024-04-24T08:25:14.628 INFO:tasks.workunit.client.0.smithi090.stdout:4/814: stat d3/d8e/ddf/d5c/db5/f30 0 2024-04-24T08:25:14.632 INFO:tasks.workunit.client.0.smithi090.stdout:5/928: creat d8/d99/f12f x:0 0 0 2024-04-24T08:25:14.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/901: write d8/d1d/d1f/d3e/d10a/d19/f64 [4930862,96947] 0 2024-04-24T08:25:14.635 INFO:tasks.workunit.client.0.smithi090.stdout:2/910: dread d0/d7/d67/d7f/d31/f41 [0,4194304] 0 2024-04-24T08:25:14.638 INFO:tasks.workunit.client.0.smithi090.stdout:5/929: creat d8/d56/f130 x:0 0 0 2024-04-24T08:25:14.639 INFO:tasks.workunit.client.0.smithi090.stdout:8/813: mkdir d2/da/d13/d64/d24/dad/d10c 0 2024-04-24T08:25:14.639 INFO:tasks.workunit.client.0.smithi090.stdout:6/902: write d8/d1d/d29/fe9 [658096,122973] 0 2024-04-24T08:25:14.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/815: read d3/d8e/ddf/d5c/db5/f9a [2243532,71203] 0 2024-04-24T08:25:14.641 INFO:tasks.workunit.client.0.smithi090.stdout:2/911: chown d0/d7/d6f/fc6 0 1 2024-04-24T08:25:14.643 INFO:tasks.workunit.client.0.smithi090.stdout:7/937: dwrite db/d17/d29/d38/d3e/d46/fed [0,4194304] 0 2024-04-24T08:25:14.644 INFO:tasks.workunit.client.0.smithi090.stdout:5/930: stat d8/da/d14/d2f/d112/da2 0 2024-04-24T08:25:14.644 INFO:tasks.workunit.client.0.smithi090.stdout:8/814: dread - d2/d62/d83/da3/deb/fec zero size 2024-04-24T08:25:14.644 INFO:tasks.workunit.client.0.smithi090.stdout:0/857: dread f8 [0,4194304] 0 2024-04-24T08:25:14.650 INFO:tasks.workunit.client.0.smithi090.stdout:6/903: symlink d8/d1d/d1f/d3e/d44/d90/da2/l119 0 2024-04-24T08:25:14.653 INFO:tasks.workunit.client.0.smithi090.stdout:0/858: fsync d1a/d4b/d53/d79/d58/db4/f109 0 2024-04-24T08:25:14.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/815: creat d2/da/db0/f10d x:0 0 0 2024-04-24T08:25:14.657 INFO:tasks.workunit.client.0.smithi090.stdout:7/938: truncate db/d17/f28 4830742 0 2024-04-24T08:25:14.659 INFO:tasks.workunit.client.0.smithi090.stdout:2/912: rename d0/d14/d36 to d0/d14/d17/dc2/d135 0 2024-04-24T08:25:14.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/816: mknod d2/d62/d83/da3/c10e 0 2024-04-24T08:25:14.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/817: readlink d2/d62/d6b/d9c/dd8/lea 0 2024-04-24T08:25:14.665 INFO:tasks.workunit.client.0.smithi090.stdout:7/939: unlink db/d17/d29/f42 0 2024-04-24T08:25:14.666 INFO:tasks.workunit.client.0.smithi090.stdout:3/849: dread d2/d15/d36/d6f/d92/fac [0,4194304] 0 2024-04-24T08:25:14.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/835: dread d0/d19/f28 [0,4194304] 0 2024-04-24T08:25:14.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/913: mkdir d0/d7/d57/d79/d116/d136 0 2024-04-24T08:25:14.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/859: creat d1a/f121 x:0 0 0 2024-04-24T08:25:14.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/818: symlink d2/d62/d83/da3/deb/l10f 0 2024-04-24T08:25:14.672 INFO:tasks.workunit.client.0.smithi090.stdout:2/914: readlink d0/d7/l2f 0 2024-04-24T08:25:14.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/836: rmdir d0/dc/d24/d2d/d49/d62/dcb 39 2024-04-24T08:25:14.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/847: dread d3/d5/da/dc/f87 [0,4194304] 0 2024-04-24T08:25:14.675 INFO:tasks.workunit.client.0.smithi090.stdout:2/915: dread - d0/d7/de1/ff8 zero size 2024-04-24T08:25:14.676 INFO:tasks.workunit.client.0.smithi090.stdout:3/850: fdatasync d2/df/d1a/d37/d53/d60/d9b/dca/fd9 0 2024-04-24T08:25:14.678 INFO:tasks.workunit.client.0.smithi090.stdout:2/916: dread - d0/fad zero size 2024-04-24T08:25:14.679 INFO:tasks.workunit.client.0.smithi090.stdout:9/837: chown d0/dc/d24/d7c/d4d/d8c/de2/lca 0 1 2024-04-24T08:25:14.679 INFO:tasks.workunit.client.0.smithi090.stdout:1/848: chown d3/d5/da/d94/da6/d115/l5f 107 1 2024-04-24T08:25:14.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/860: unlink d1a/d4b/d53/d79/d35/dea/dfe/fd4 0 2024-04-24T08:25:14.681 INFO:tasks.workunit.client.0.smithi090.stdout:0/861: readlink d1a/d4b/d53/d79/d58/db4/lc2 0 2024-04-24T08:25:14.683 INFO:tasks.workunit.client.0.smithi090.stdout:8/819: rename d2/da/d13/d64/f16 to d2/d62/d6b/f110 0 2024-04-24T08:25:14.684 INFO:tasks.workunit.client.0.smithi090.stdout:2/917: fdatasync d0/fad 0 2024-04-24T08:25:14.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/838: fsync d0/dc/d24/d2d/d49/d62/f68 0 2024-04-24T08:25:14.688 INFO:tasks.workunit.client.0.smithi090.stdout:4/816: dread d3/d8e/ddf/d5c/db5/d77/fa7 [0,4194304] 0 2024-04-24T08:25:14.688 INFO:tasks.workunit.client.0.smithi090.stdout:6/904: dread d8/d1d/d1f/f5d [0,4194304] 0 2024-04-24T08:25:14.690 INFO:tasks.workunit.client.0.smithi090.stdout:3/851: mknod d2/df/d1a/c11e 0 2024-04-24T08:25:14.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/918: chown d0/f22 235 1 2024-04-24T08:25:14.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/820: mkdir d2/da/db0/d111 0 2024-04-24T08:25:14.694 INFO:tasks.workunit.client.0.smithi090.stdout:6/905: truncate d8/d1d/d29/df0/ff2 603950 0 2024-04-24T08:25:14.697 INFO:tasks.workunit.client.0.smithi090.stdout:6/906: chown d8/d1d/d1f/d3e/d10a/d19/l22 3004 1 2024-04-24T08:25:14.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/839: rename d0/d19/la7 to d0/dc/d24/d7c/l10f 0 2024-04-24T08:25:14.699 INFO:tasks.workunit.client.0.smithi090.stdout:7/940: dread db/d17/d29/d38/d3e/fb7 [0,4194304] 0 2024-04-24T08:25:14.701 INFO:tasks.workunit.client.0.smithi090.stdout:5/931: dwrite d8/da/d14/d2f/d112/dcb/f1e [0,4194304] 0 2024-04-24T08:25:14.701 INFO:tasks.workunit.client.0.smithi090.stdout:7/941: fsync db/dbd/d112/f12a 0 2024-04-24T08:25:14.706 INFO:tasks.workunit.client.0.smithi090.stdout:2/919: rename d0/d5e/de2/l123 to d0/d7/d6f/l137 0 2024-04-24T08:25:14.708 INFO:tasks.workunit.client.0.smithi090.stdout:1/849: write d3/d5/d3e/da3/fc8 [827016,5418] 0 2024-04-24T08:25:14.711 INFO:tasks.workunit.client.0.smithi090.stdout:5/932: mknod d8/d59/dc7/d7a/da1/c131 0 2024-04-24T08:25:14.712 INFO:tasks.workunit.client.0.smithi090.stdout:6/907: symlink d8/d1d/d1f/d3e/l11a 0 2024-04-24T08:25:14.713 INFO:tasks.workunit.client.0.smithi090.stdout:7/942: write db/d17/d29/d32/d5a/d75/fa4 [3285298,93438] 0 2024-04-24T08:25:14.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/821: rename d2/da/d13/d34/f40 to d2/da/d13/d64/db2/d107/f112 0 2024-04-24T08:25:14.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/817: write d3/ff6 [429615,57394] 0 2024-04-24T08:25:14.719 INFO:tasks.workunit.client.0.smithi090.stdout:2/920: symlink d0/d14/d17/d11f/l138 0 2024-04-24T08:25:14.720 INFO:tasks.workunit.client.0.smithi090.stdout:7/943: write db/d54/d83/f132 [248106,71843] 0 2024-04-24T08:25:14.721 INFO:tasks.workunit.client.0.smithi090.stdout:5/933: creat d8/da/d3a/d2d/d37/f132 x:0 0 0 2024-04-24T08:25:14.724 INFO:tasks.workunit.client.0.smithi090.stdout:1/850: truncate d3/d5/d3e/da3/fd0 515752 0 2024-04-24T08:25:14.724 INFO:tasks.workunit.client.0.smithi090.stdout:0/862: dwrite d1a/d4b/d53/d79/fcf [0,4194304] 0 2024-04-24T08:25:14.725 INFO:tasks.workunit.client.0.smithi090.stdout:5/934: stat d8/da/d14/d2f/d112/da2/l129 0 2024-04-24T08:25:14.726 INFO:tasks.workunit.client.0.smithi090.stdout:8/822: creat d2/da/d13/d64/db2/dbb/f113 x:0 0 0 2024-04-24T08:25:14.727 INFO:tasks.workunit.client.0.smithi090.stdout:2/921: creat d0/d5e/de2/f139 x:0 0 0 2024-04-24T08:25:14.729 INFO:tasks.workunit.client.0.smithi090.stdout:7/944: mkdir db/de9/d139 0 2024-04-24T08:25:14.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/852: dwrite d2/df/d1a/d37/d4b/fa8 [4194304,4194304] 0 2024-04-24T08:25:14.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/823: write d2/d62/d83/da3/fbd [1127452,81303] 0 2024-04-24T08:25:14.731 INFO:tasks.workunit.client.0.smithi090.stdout:2/922: write d0/d14/d17/f18 [2040931,52404] 0 2024-04-24T08:25:14.735 INFO:tasks.workunit.client.0.smithi090.stdout:6/908: unlink d8/d1d/d1f/d3e/d4f/c76 0 2024-04-24T08:25:14.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/840: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110 0 2024-04-24T08:25:14.735 INFO:tasks.workunit.client.0.smithi090.stdout:6/909: dread - d8/feb zero size 2024-04-24T08:25:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:4/818: mkdir d3/d105 0 2024-04-24T08:25:14.737 INFO:tasks.workunit.client.0.smithi090.stdout:6/910: chown d8/d1d/d1f/f9c 402720733 1 2024-04-24T08:25:14.738 INFO:tasks.workunit.client.0.smithi090.stdout:5/935: creat d8/d7c/da9/d118/f133 x:0 0 0 2024-04-24T08:25:14.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/863: truncate d1a/d4b/d53/d79/fa9 1465622 0 2024-04-24T08:25:14.745 INFO:tasks.workunit.client.0.smithi090.stdout:7/945: creat db/d17/d29/d32/d5a/d75/d11f/dae/f13a x:0 0 0 2024-04-24T08:25:14.746 INFO:tasks.workunit.client.0.smithi090.stdout:3/853: dwrite d2/df/d21/f30 [0,4194304] 0 2024-04-24T08:25:14.747 INFO:tasks.workunit.client.0.smithi090.stdout:9/841: symlink d0/dc/d24/d2d/d49/d81/l111 0 2024-04-24T08:25:14.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/851: unlink d3/d1d/d34/c95 0 2024-04-24T08:25:14.748 INFO:tasks.workunit.client.0.smithi090.stdout:3/854: fsync d2/df/d1a/d37/d53/fe7 0 2024-04-24T08:25:14.749 INFO:tasks.workunit.client.0.smithi090.stdout:4/819: unlink d3/d24/d92/fc2 0 2024-04-24T08:25:14.751 INFO:tasks.workunit.client.0.smithi090.stdout:5/936: creat d8/d59/dc7/d7a/da1/dae/f134 x:0 0 0 2024-04-24T08:25:14.751 INFO:tasks.workunit.client.0.smithi090.stdout:3/855: dread - d2/d15/d36/d6f/d9d/ff0 zero size 2024-04-24T08:25:14.753 INFO:tasks.workunit.client.0.smithi090.stdout:0/864: mkdir d1a/d4b/d53/d57/d122 0 2024-04-24T08:25:14.754 INFO:tasks.workunit.client.0.smithi090.stdout:7/946: creat db/de9/f13b x:0 0 0 2024-04-24T08:25:14.756 INFO:tasks.workunit.client.0.smithi090.stdout:1/852: creat d3/d5/da/dc/df4/f12a x:0 0 0 2024-04-24T08:25:14.757 INFO:tasks.workunit.client.0.smithi090.stdout:0/865: truncate d1a/d4b/d53/d79/d58/fd3 735460 0 2024-04-24T08:25:14.758 INFO:tasks.workunit.client.0.smithi090.stdout:4/820: truncate d3/d8e/ddf/d5c/f47 812572 0 2024-04-24T08:25:14.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/842: mknod d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/c112 0 2024-04-24T08:25:14.762 INFO:tasks.workunit.client.0.smithi090.stdout:6/911: getdents d8/d1d/d1f/d3e/d4f/d7e/daa 0 2024-04-24T08:25:14.763 INFO:tasks.workunit.client.0.smithi090.stdout:0/866: creat d1a/d4b/d53/d79/d82/f123 x:0 0 0 2024-04-24T08:25:14.764 INFO:tasks.workunit.client.0.smithi090.stdout:9/843: read - d0/dc/d24/d2d/d49/d62/dcb/dcd/ffb zero size 2024-04-24T08:25:14.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/821: symlink d3/d8e/ddf/d5c/db5/d69/l106 0 2024-04-24T08:25:14.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/912: fsync d8/f3f 0 2024-04-24T08:25:14.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/867: chown d1a/d4b/d53/d79/d69/d78/d9b/cd9 20 1 2024-04-24T08:25:14.768 INFO:tasks.workunit.client.0.smithi090.stdout:5/937: dread d8/d59/dc7/d7b/f80 [0,4194304] 0 2024-04-24T08:25:14.772 INFO:tasks.workunit.client.0.smithi090.stdout:5/938: fdatasync d8/da/d14/d2f/d112/dcb/fa5 0 2024-04-24T08:25:14.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/844: rename d0/l42 to d0/dc/d24/d2d/dda/de1/d103/l113 0 2024-04-24T08:25:14.775 INFO:tasks.workunit.client.0.smithi090.stdout:1/853: dwrite d3/d5/da/d94/da6/d115/d30/d9d/fd3 [4194304,4194304] 0 2024-04-24T08:25:14.780 INFO:tasks.workunit.client.0.smithi090.stdout:6/913: symlink d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/l11b 0 2024-04-24T08:25:14.782 INFO:tasks.workunit.client.0.smithi090.stdout:9/845: truncate d0/dc/f64 570780 0 2024-04-24T08:25:14.783 INFO:tasks.workunit.client.0.smithi090.stdout:6/914: write d8/d1d/d1f/d3e/d7a/fef [257024,58670] 0 2024-04-24T08:25:14.783 INFO:tasks.workunit.client.0.smithi090.stdout:8/824: write d2/d62/d6b/d9c/dd8/fe1 [988928,69624] 0 2024-04-24T08:25:14.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/825: chown d2/d62/d6b/c77 70 1 2024-04-24T08:25:14.791 INFO:tasks.workunit.client.0.smithi090.stdout:5/939: dwrite d8/da/d14/d2f/d112/dcb/de3/f105 [0,4194304] 0 2024-04-24T08:25:14.797 INFO:tasks.workunit.client.0.smithi090.stdout:0/868: getdents d1a/d4b/d53/d79/d69/d78/d9b 0 2024-04-24T08:25:14.802 INFO:tasks.workunit.client.0.smithi090.stdout:2/923: dwrite d0/d2e/f6a [4194304,4194304] 0 2024-04-24T08:25:14.803 INFO:tasks.workunit.client.0.smithi090.stdout:3/856: write d2/df/f19 [1043830,102724] 0 2024-04-24T08:25:14.807 INFO:tasks.workunit.client.0.smithi090.stdout:3/857: chown d2/d84/c104 674 1 2024-04-24T08:25:14.808 INFO:tasks.workunit.client.0.smithi090.stdout:7/947: write db/d17/d29/d32/f3d [4936718,46363] 0 2024-04-24T08:25:14.809 INFO:tasks.workunit.client.0.smithi090.stdout:0/869: creat d1a/d4b/d53/d79/d69/d90/dcc/f124 x:0 0 0 2024-04-24T08:25:14.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/940: dread d8/d76/dad/fd1 [0,4194304] 0 2024-04-24T08:25:14.814 INFO:tasks.workunit.client.0.smithi090.stdout:0/870: write d1a/d4b/d53/d79/d58/de7/f100 [85008,6222] 0 2024-04-24T08:25:14.815 INFO:tasks.workunit.client.0.smithi090.stdout:2/924: link d0/lb d0/d7/d28/daa/ddc/d12a/d133/l13a 0 2024-04-24T08:25:14.817 INFO:tasks.workunit.client.0.smithi090.stdout:7/948: mknod db/d17/d29/d32/c13c 0 2024-04-24T08:25:14.819 INFO:tasks.workunit.client.0.smithi090.stdout:5/941: mknod d8/d9c/c135 0 2024-04-24T08:25:14.821 INFO:tasks.workunit.client.0.smithi090.stdout:2/925: fdatasync d0/d7/d67/d7f/f87 0 2024-04-24T08:25:14.824 INFO:tasks.workunit.client.0.smithi090.stdout:5/942: chown d8/d59/dc7/d7a/d50/f68 3463 1 2024-04-24T08:25:14.825 INFO:tasks.workunit.client.0.smithi090.stdout:9/846: write d0/dc/d24/d2d/d49/d81/d87/f8a [4706449,91682] 0 2024-04-24T08:25:14.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/949: rename db/d17/d29/d38/d99/c137 to db/d54/ddb/c13d 0 2024-04-24T08:25:14.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/826: write d2/da/d13/d64/d24/db3/fc2 [2564500,31229] 0 2024-04-24T08:25:14.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/822: dwrite d3/d8e/ddf/d5c/f3d [0,4194304] 0 2024-04-24T08:25:14.831 INFO:tasks.workunit.client.0.smithi090.stdout:5/943: mknod d8/d25/c136 0 2024-04-24T08:25:14.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/858: write d2/d84/fa3 [584493,34828] 0 2024-04-24T08:25:14.842 INFO:tasks.workunit.client.0.smithi090.stdout:7/950: symlink db/d17/d29/dc0/l13e 0 2024-04-24T08:25:14.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/827: readlink d2/da/d13/d64/d50/la6 0 2024-04-24T08:25:14.843 INFO:tasks.workunit.client.0.smithi090.stdout:2/926: rename d0/d7/d67/d7f/d31/d90/f108 to d0/d7/d67/d7f/d31/d90/f13b 0 2024-04-24T08:25:14.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/823: chown d3/d8e/ddf/d5c/db5/db6/ffa 0 1 2024-04-24T08:25:14.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/847: write d0/dc/d24/d7c/d4d/d32/ff3 [1196871,18551] 0 2024-04-24T08:25:14.849 INFO:tasks.workunit.client.0.smithi090.stdout:1/854: dwrite d3/d5/da/dc/f3a [4194304,4194304] 0 2024-04-24T08:25:14.849 INFO:tasks.workunit.client.0.smithi090.stdout:6/915: dwrite d8/d1d/d1f/d3e/fd6 [0,4194304] 0 2024-04-24T08:25:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:7/951: dread - db/d17/d29/d32/fa3 zero size 2024-04-24T08:25:14.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/848: readlink d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/lbf 0 2024-04-24T08:25:14.859 INFO:tasks.workunit.client.0.smithi090.stdout:4/824: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f107 x:0 0 0 2024-04-24T08:25:14.860 INFO:tasks.workunit.client.0.smithi090.stdout:7/952: write db/d54/d78/ffe [57773,77662] 0 2024-04-24T08:25:14.860 INFO:tasks.workunit.client.0.smithi090.stdout:8/828: truncate d2/d62/d6b/f90 295416 0 2024-04-24T08:25:14.862 INFO:tasks.workunit.client.0.smithi090.stdout:7/953: chown db/d54 296783 1 2024-04-24T08:25:14.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/855: dread - d3/d5/da/d94/da6/d115/d30/d9d/da2/fd6 zero size 2024-04-24T08:25:14.865 INFO:tasks.workunit.client.0.smithi090.stdout:6/916: creat d8/d1d/d1f/d3e/d44/d90/d106/d38/dfd/f11c x:0 0 0 2024-04-24T08:25:14.866 INFO:tasks.workunit.client.0.smithi090.stdout:5/944: dwrite d8/da/d3a/d2d/f82 [0,4194304] 0 2024-04-24T08:25:14.867 INFO:tasks.workunit.client.0.smithi090.stdout:9/849: read d0/dc/d24/d7c/f82 [123139,79551] 0 2024-04-24T08:25:14.877 INFO:tasks.workunit.client.0.smithi090.stdout:8/829: truncate d2/d62/f6e 542359 0 2024-04-24T08:25:14.878 INFO:tasks.workunit.client.0.smithi090.stdout:9/850: chown d0/da4/cdb 318577 1 2024-04-24T08:25:14.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/825: rename d3/d8e/fdd to d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/f108 0 2024-04-24T08:25:14.879 INFO:tasks.workunit.client.0.smithi090.stdout:2/927: dread d0/d7/d67/d7f/d31/d46/fb7 [0,4194304] 0 2024-04-24T08:25:14.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/830: stat d2/da/d13/d60/fb1 0 2024-04-24T08:25:14.880 INFO:tasks.workunit.client.0.smithi090.stdout:0/871: dread d1a/d4b/d53/d79/d58/db4/fed [0,4194304] 0 2024-04-24T08:25:14.880 INFO:tasks.workunit.client.0.smithi090.stdout:9/851: chown d0/dc/d24/d2d/d49/d62 0 1 2024-04-24T08:25:14.883 INFO:tasks.workunit.client.0.smithi090.stdout:2/928: write d0/d2e/f10a [908375,59135] 0 2024-04-24T08:25:14.883 INFO:tasks.workunit.client.0.smithi090.stdout:3/859: dread d2/df/d57/d4f/f86 [0,4194304] 0 2024-04-24T08:25:14.883 INFO:tasks.workunit.client.0.smithi090.stdout:8/831: fsync d2/da/d13/d60/d91/ff8 0 2024-04-24T08:25:14.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/826: read d3/d8e/ddf/d1d/fa1 [3706332,57533] 0 2024-04-24T08:25:14.886 INFO:tasks.workunit.client.0.smithi090.stdout:5/945: getdents d8/d7c/da9/d118/da6/d11d 0 2024-04-24T08:25:14.887 INFO:tasks.workunit.client.0.smithi090.stdout:0/872: write d1a/d4b/d53/d79/d35/f42 [2362621,6859] 0 2024-04-24T08:25:14.889 INFO:tasks.workunit.client.0.smithi090.stdout:1/856: creat d3/d1d/d34/f12b x:0 0 0 2024-04-24T08:25:14.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/946: chown d8/da/d3a/d2d/d37/dec 7 1 2024-04-24T08:25:14.893 INFO:tasks.workunit.client.0.smithi090.stdout:6/917: write d8/d1d/fa3 [1034390,14291] 0 2024-04-24T08:25:14.894 INFO:tasks.workunit.client.0.smithi090.stdout:9/852: symlink d0/dc/d24/l114 0 2024-04-24T08:25:14.894 INFO:tasks.workunit.client.0.smithi090.stdout:7/954: write db/d17/d29/d32/fe4 [970230,53129] 0 2024-04-24T08:25:14.896 INFO:tasks.workunit.client.0.smithi090.stdout:8/832: creat d2/da/d13/d64/db2/dfe/f114 x:0 0 0 2024-04-24T08:25:14.899 INFO:tasks.workunit.client.0.smithi090.stdout:3/860: write d2/d15/f2e [639264,35745] 0 2024-04-24T08:25:14.900 INFO:tasks.workunit.client.0.smithi090.stdout:0/873: mknod d1a/d4b/d53/d79/d35/dea/dfe/c125 0 2024-04-24T08:25:14.901 INFO:tasks.workunit.client.0.smithi090.stdout:1/857: mkdir d3/d5/da/dc/d80/deb/d111/d12c 0 2024-04-24T08:25:14.901 INFO:tasks.workunit.client.0.smithi090.stdout:7/955: readlink db/d17/d29/d32/d5a/d75/d11f/dae/lce 0 2024-04-24T08:25:14.902 INFO:tasks.workunit.client.0.smithi090.stdout:6/918: rmdir d8/d1d/d1f/d3e/d10a 39 2024-04-24T08:25:14.904 INFO:tasks.workunit.client.0.smithi090.stdout:2/929: getdents d0/da7 0 2024-04-24T08:25:14.908 INFO:tasks.workunit.client.1.smithi149.stdout:2/958: sync 2024-04-24T08:25:14.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/930: chown d0/d7/d28/daa/ff5 6 1 2024-04-24T08:25:14.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/827: dwrite d3/d8e/fed [0,4194304] 0 2024-04-24T08:25:14.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/861: write d2/d15/d36/d6f/f9a [521256,63930] 0 2024-04-24T08:25:14.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/853: rename d0/dc/d24/d2d/d49/d62/dcb/dec/c105 to d0/dc/d24/d2d/dda/de1/c115 0 2024-04-24T08:25:14.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/874: mknod d1a/d4b/d53/d79/d69/d90/d10e/c126 0 2024-04-24T08:25:14.918 INFO:tasks.workunit.client.0.smithi090.stdout:3/862: truncate d2/d25/f8c 268465 0 2024-04-24T08:25:14.919 INFO:tasks.workunit.client.0.smithi090.stdout:2/931: mkdir d0/d13c 0 2024-04-24T08:25:14.924 INFO:tasks.workunit.client.0.smithi090.stdout:2/932: unlink d0/d7/d6f/c9b 0 2024-04-24T08:25:14.926 INFO:tasks.workunit.client.1.smithi149.stdout:2/959: dwrite d0/ff4 [0,4194304] 0 2024-04-24T08:25:14.927 INFO:tasks.workunit.client.0.smithi090.stdout:1/858: dwrite d3/d5/f16 [0,4194304] 0 2024-04-24T08:25:14.929 INFO:tasks.workunit.client.0.smithi090.stdout:7/956: dwrite db/d17/d29/d38/d3e/fe7 [0,4194304] 0 2024-04-24T08:25:14.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/960: write d0/d6/db/d28/db0/fc0 [385089,121671] 0 2024-04-24T08:25:14.933 INFO:tasks.workunit.client.1.smithi149.stdout:2/961: write d0/d6/db/f1c [2080504,100818] 0 2024-04-24T08:25:14.936 INFO:tasks.workunit.client.0.smithi090.stdout:7/957: chown db/d17/d29/d32/d5a/d75/d11f/la7 0 1 2024-04-24T08:25:14.937 INFO:tasks.workunit.client.0.smithi090.stdout:1/859: creat d3/d5/d10e/f12d x:0 0 0 2024-04-24T08:25:14.939 INFO:tasks.workunit.client.0.smithi090.stdout:6/919: dread d8/d1d/d1f/f89 [0,4194304] 0 2024-04-24T08:25:14.939 INFO:tasks.workunit.client.0.smithi090.stdout:3/863: link d2/df/d21/c8b d2/df/d1a/d37/dc0/de6/dfa/c11f 0 2024-04-24T08:25:14.939 INFO:tasks.workunit.client.0.smithi090.stdout:1/860: fdatasync d3/d5/d3e/da3/fc8 0 2024-04-24T08:25:14.939 INFO:tasks.workunit.client.1.smithi149.stdout:2/962: creat d0/d6/db/d27/f127 x:0 0 0 2024-04-24T08:25:14.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/861: read d3/d5/da/dc/fab [2965825,33557] 0 2024-04-24T08:25:14.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/864: mkdir d2/d120 0 2024-04-24T08:25:14.944 INFO:tasks.workunit.client.1.smithi149.stdout:2/963: creat d0/d6/db/d39/d3d/d44/df0/f128 x:0 0 0 2024-04-24T08:25:14.944 INFO:tasks.workunit.client.0.smithi090.stdout:1/862: readlink d3/d1d/d54/l62 0 2024-04-24T08:25:14.945 INFO:tasks.workunit.client.1.smithi149.stdout:2/964: chown d0/d6/db/d28/f2c 409 1 2024-04-24T08:25:14.946 INFO:tasks.workunit.client.0.smithi090.stdout:6/920: creat d8/d1d/d1f/d3e/d44/d90/d106/d38/f11d x:0 0 0 2024-04-24T08:25:14.949 INFO:tasks.workunit.client.0.smithi090.stdout:3/865: creat d2/df/d1a/d37/d59/d87/db0/f121 x:0 0 0 2024-04-24T08:25:14.950 INFO:tasks.workunit.client.0.smithi090.stdout:1/863: creat d3/d5/da/dc/d80/deb/d111/f12e x:0 0 0 2024-04-24T08:25:14.952 INFO:tasks.workunit.client.1.smithi149.stdout:2/965: getdents d0/d6/db/d27/d64/d66/db1/dcf/dd4 0 2024-04-24T08:25:14.954 INFO:tasks.workunit.client.0.smithi090.stdout:1/864: symlink d3/d5/d58/d11f/d102/l12f 0 2024-04-24T08:25:14.957 INFO:tasks.workunit.client.1.smithi149.stdout:2/966: creat d0/d6/db/d27/d64/d66/db1/f129 x:0 0 0 2024-04-24T08:25:14.961 INFO:tasks.workunit.client.1.smithi149.stdout:2/967: mknod d0/d6/db/d27/d64/d66/db1/dc9/d102/c12a 0 2024-04-24T08:25:14.963 INFO:tasks.workunit.client.0.smithi090.stdout:6/921: dwrite d8/d1d/d1f/d3e/d4d/f99 [0,4194304] 0 2024-04-24T08:25:14.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/866: dwrite d2/df/d1a/d37/d4b/f112 [0,4194304] 0 2024-04-24T08:25:14.966 INFO:tasks.workunit.client.1.smithi149.stdout:2/968: symlink d0/d6/db/d28/d7f/d8f/db9/ddf/de3/l12b 0 2024-04-24T08:25:14.967 INFO:tasks.workunit.client.0.smithi090.stdout:3/867: chown d2/df/d1a/d37/f8a 4349755 1 2024-04-24T08:25:14.971 INFO:tasks.workunit.client.1.smithi149.stdout:2/969: mknod d0/d6/db/d27/d64/d66/c12c 0 2024-04-24T08:25:14.974 INFO:tasks.workunit.client.0.smithi090.stdout:3/868: mknod d2/df/d1a/d37/d59/d87/c122 0 2024-04-24T08:25:14.981 INFO:tasks.workunit.client.0.smithi090.stdout:3/869: creat d2/df/d57/d114/f123 x:0 0 0 2024-04-24T08:25:14.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/875: write d1a/d4b/d53/d79/fb2 [1928213,105477] 0 2024-04-24T08:25:14.993 INFO:tasks.workunit.client.0.smithi090.stdout:4/828: dread d3/fc4 [0,4194304] 0 2024-04-24T08:25:15.002 INFO:tasks.workunit.client.1.smithi149.stdout:2/970: dwrite d0/d6/db/d27/f114 [0,4194304] 0 2024-04-24T08:25:15.017 INFO:tasks.workunit.client.1.smithi149.stdout:2/971: dwrite d0/d6/db/d27/d81/db4/f11f [0,4194304] 0 2024-04-24T08:25:15.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/876: creat d1a/d11d/f127 x:0 0 0 2024-04-24T08:25:15.021 INFO:tasks.workunit.client.1.smithi149.stdout:2/972: unlink d0/d6/db/d27/d81/db4/cfd 0 2024-04-24T08:25:15.023 INFO:tasks.workunit.client.0.smithi090.stdout:3/870: dread d2/df/fa5 [0,4194304] 0 2024-04-24T08:25:15.023 INFO:tasks.workunit.client.0.smithi090.stdout:7/958: dread db/d17/d39/f41 [0,4194304] 0 2024-04-24T08:25:15.023 INFO:tasks.workunit.client.1.smithi149.stdout:2/973: chown d0/d6/db/d39/d78/lcb 11 1 2024-04-24T08:25:15.028 INFO:tasks.workunit.client.0.smithi090.stdout:9/854: dwrite d0/dc/d24/d2d/d49/d62/f68 [0,4194304] 0 2024-04-24T08:25:15.030 INFO:tasks.workunit.client.0.smithi090.stdout:7/959: creat db/d17/d29/d38/d99/f13f x:0 0 0 2024-04-24T08:25:15.031 INFO:tasks.workunit.client.1.smithi149.stdout:2/974: mknod d0/d6/db/d27/d81/d9f/c12d 0 2024-04-24T08:25:15.032 INFO:tasks.workunit.client.0.smithi090.stdout:7/960: write db/d17/d29/d38/d3e/fb8 [808233,100285] 0 2024-04-24T08:25:15.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/871: mkdir d2/d25/d124 0 2024-04-24T08:25:15.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/872: read - d2/d15/fe0 zero size 2024-04-24T08:25:15.040 INFO:tasks.workunit.client.1.smithi149.stdout:2/975: mknod d0/d6/db/d39/d78/d7c/c12e 0 2024-04-24T08:25:15.045 INFO:tasks.workunit.client.1.smithi149.stdout:2/976: write d0/d6/db/d27/f114 [1407751,25284] 0 2024-04-24T08:25:15.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/977: rename d0/d6/db/d28/f6f to d0/d6/f12f 0 2024-04-24T08:25:15.055 INFO:tasks.workunit.client.0.smithi090.stdout:3/873: link d2/df/d1a/d37/d53/d60/ld3 d2/d25/l125 0 2024-04-24T08:25:15.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/874: fdatasync d2/d15/d36/d6f/d92/fa9 0 2024-04-24T08:25:15.063 INFO:tasks.workunit.client.0.smithi090.stdout:1/865: truncate d3/d1d/d34/d5a/fec 498328 0 2024-04-24T08:25:15.065 INFO:tasks.workunit.client.0.smithi090.stdout:6/922: write d8/d1d/d1f/d3e/d44/f82 [3269853,47015] 0 2024-04-24T08:25:15.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/961: dread db/d54/db1/fe2 [0,4194304] 0 2024-04-24T08:25:15.075 INFO:tasks.workunit.client.0.smithi090.stdout:7/962: mkdir db/d54/d78/d140 0 2024-04-24T08:25:15.075 INFO:tasks.workunit.client.0.smithi090.stdout:0/877: write d1a/d4b/d53/d57/fa4 [432100,5131] 0 2024-04-24T08:25:15.078 INFO:tasks.workunit.client.0.smithi090.stdout:6/923: rename d8/l13 to d8/d1d/d1f/d3e/d10a/d19/dea/l11e 0 2024-04-24T08:25:15.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/878: write d1a/d4b/d53/d79/d35/d62/fc3 [4989015,99574] 0 2024-04-24T08:25:15.083 INFO:tasks.workunit.client.0.smithi090.stdout:7/963: creat db/d17/d29/d32/d5a/d75/f141 x:0 0 0 2024-04-24T08:25:15.084 INFO:tasks.workunit.client.0.smithi090.stdout:1/866: link d3/d5/d3e/da3/fd0 d3/d5/da/d97/d9c/f130 0 2024-04-24T08:25:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:9/855: dread d0/dc/d24/d7c/d4d/d32/ff3 [0,4194304] 0 2024-04-24T08:25:15.087 INFO:tasks.workunit.client.0.smithi090.stdout:7/964: truncate db/d17/d29/d38/d3e/d46/f103 111224 0 2024-04-24T08:25:15.088 INFO:tasks.workunit.client.0.smithi090.stdout:1/867: write d3/d5/da/d94/d10b/f11a [160957,40205] 0 2024-04-24T08:25:15.089 INFO:tasks.workunit.client.1.smithi149.stdout:2/978: dread d0/d6/db/d28/f3a [0,4194304] 0 2024-04-24T08:25:15.096 INFO:tasks.workunit.client.1.smithi149.stdout:2/979: getdents d0/d6/db/d39/d3d/d44/d75 0 2024-04-24T08:25:15.100 INFO:tasks.workunit.client.1.smithi149.stdout:2/980: symlink d0/d6/db/d27/d81/db4/l130 0 2024-04-24T08:25:15.113 INFO:tasks.workunit.client.1.smithi149.stdout:2/981: dwrite d0/d6/db/d27/f5f [4194304,4194304] 0 2024-04-24T08:25:15.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/982: mkdir d0/d6/db/d39/d3d/d44/d131 0 2024-04-24T08:25:15.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/983: unlink d0/d6/db/d39/d3d/d44/df0/df8/c10f 0 2024-04-24T08:25:15.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/879: mknod d1a/d4b/d53/d79/d69/c128 0 2024-04-24T08:25:15.126 INFO:tasks.workunit.client.0.smithi090.stdout:5/947: sync 2024-04-24T08:25:15.128 INFO:tasks.workunit.client.1.smithi149.stdout:2/984: chown d0/d6/db/d28/db0/cde 451309 1 2024-04-24T08:25:15.134 INFO:tasks.workunit.client.0.smithi090.stdout:0/880: creat d1a/d4b/d53/d79/d82/f129 x:0 0 0 2024-04-24T08:25:15.134 INFO:tasks.workunit.client.1.smithi149.stdout:2/985: creat d0/d6/db/d28/d7f/d8f/db9/ddf/de3/f132 x:0 0 0 2024-04-24T08:25:15.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/948: dread - d8/d7c/ff1 zero size 2024-04-24T08:25:15.135 INFO:tasks.workunit.client.0.smithi090.stdout:6/924: dwrite d8/d1d/d1f/d3e/d44/d90/d106/fd3 [0,4194304] 0 2024-04-24T08:25:15.136 INFO:tasks.workunit.client.0.smithi090.stdout:1/868: truncate d3/d5/da/d94/da6/d115/fd4 4270909 0 2024-04-24T08:25:15.138 INFO:tasks.workunit.client.0.smithi090.stdout:0/881: write f18 [905114,13096] 0 2024-04-24T08:25:15.138 INFO:tasks.workunit.client.1.smithi149.stdout:2/986: rename d0/d6/db/d28/d7f/ff7 to d0/d6/db/f133 0 2024-04-24T08:25:15.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/949: stat d8/d7c/da9/d118/da6/d11d 0 2024-04-24T08:25:15.143 INFO:tasks.workunit.client.0.smithi090.stdout:6/925: fdatasync d8/d1d/fa3 0 2024-04-24T08:25:15.144 INFO:tasks.workunit.client.0.smithi090.stdout:7/965: dwrite db/dbd/f107 [0,4194304] 0 2024-04-24T08:25:15.145 INFO:tasks.workunit.client.0.smithi090.stdout:3/875: dread d2/df/d5b/f63 [4194304,4194304] 0 2024-04-24T08:25:15.149 INFO:tasks.workunit.client.0.smithi090.stdout:9/856: dwrite d0/d2a/f38 [0,4194304] 0 2024-04-24T08:25:15.155 INFO:tasks.workunit.client.0.smithi090.stdout:5/950: mknod d8/da/d3a/c137 0 2024-04-24T08:25:15.155 INFO:tasks.workunit.client.0.smithi090.stdout:0/882: mkdir d1a/d4b/d53/d79/d82/d9d/d116/d12a 0 2024-04-24T08:25:15.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/869: write d3/d5/da/d94/da6/d115/d30/d9d/da2/fde [1321644,74217] 0 2024-04-24T08:25:15.155 INFO:tasks.workunit.client.0.smithi090.stdout:6/926: mkdir d8/d1d/d1f/d11f 0 2024-04-24T08:25:15.163 INFO:tasks.workunit.client.0.smithi090.stdout:6/927: symlink d8/d1d/d2f/d85/l120 0 2024-04-24T08:25:15.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/966: rename db/d17/d29/d38/d3e/la1 to db/d17/d29/d38/d99/l142 0 2024-04-24T08:25:15.167 INFO:tasks.workunit.client.0.smithi090.stdout:9/857: mknod d0/dc/d24/d2d/d49/d62/dcb/dcd/c116 0 2024-04-24T08:25:15.167 INFO:tasks.workunit.client.0.smithi090.stdout:7/967: fdatasync db/d17/d29/d32/dd4/ff4 0 2024-04-24T08:25:15.168 INFO:tasks.workunit.client.0.smithi090.stdout:5/951: getdents d8/da/d14/d2f/d112/dcb/de3/d117 0 2024-04-24T08:25:15.169 INFO:tasks.workunit.client.0.smithi090.stdout:7/968: chown db/fd 1647775 1 2024-04-24T08:25:15.173 INFO:tasks.workunit.client.0.smithi090.stdout:7/969: dread - db/de9/f13b zero size 2024-04-24T08:25:15.173 INFO:tasks.workunit.client.0.smithi090.stdout:0/883: getdents d1a 0 2024-04-24T08:25:15.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/970: creat db/de9/f143 x:0 0 0 2024-04-24T08:25:15.179 INFO:tasks.workunit.client.0.smithi090.stdout:0/884: creat d1a/d4b/d53/d79/d58/f12b x:0 0 0 2024-04-24T08:25:15.180 INFO:tasks.workunit.client.0.smithi090.stdout:7/971: fdatasync db/d17/d39/f65 0 2024-04-24T08:25:15.183 INFO:tasks.workunit.client.0.smithi090.stdout:1/870: dwrite d3/d5/d3e/da3/fc8 [0,4194304] 0 2024-04-24T08:25:15.185 INFO:tasks.workunit.client.0.smithi090.stdout:3/876: write d2/d25/d33/f44 [1845595,91170] 0 2024-04-24T08:25:15.186 INFO:tasks.workunit.client.0.smithi090.stdout:7/972: creat db/dbd/f144 x:0 0 0 2024-04-24T08:25:15.187 INFO:tasks.workunit.client.0.smithi090.stdout:9/858: dread d0/d2a/f38 [0,4194304] 0 2024-04-24T08:25:15.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/885: rename d1a/f3f to d1a/d4b/d53/d79/d35/dbd/f12c 0 2024-04-24T08:25:15.190 INFO:tasks.workunit.client.1.smithi149.stdout:2/987: dwrite d0/d6/db/f133 [0,4194304] 0 2024-04-24T08:25:15.191 INFO:tasks.workunit.client.0.smithi090.stdout:7/973: chown db/d17/d29/d38/d99/fef 198 1 2024-04-24T08:25:15.192 INFO:tasks.workunit.client.0.smithi090.stdout:6/928: write d8/d1d/d1f/d3e/d8f/fcc [251769,69198] 0 2024-04-24T08:25:15.193 INFO:tasks.workunit.client.0.smithi090.stdout:5/952: dwrite d8/d59/dc7/d7b/ff9 [0,4194304] 0 2024-04-24T08:25:15.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/871: creat d3/d5/da/dc/df4/f131 x:0 0 0 2024-04-24T08:25:15.197 INFO:tasks.workunit.client.0.smithi090.stdout:3/877: mkdir d2/d15/d36/d6f/d126 0 2024-04-24T08:25:15.200 INFO:tasks.workunit.client.0.smithi090.stdout:9/859: creat d0/dc/d24/d2d/d49/dfa/f117 x:0 0 0 2024-04-24T08:25:15.201 INFO:tasks.workunit.client.0.smithi090.stdout:3/878: write d2/df/d1a/d37/d53/fe1 [848912,78092] 0 2024-04-24T08:25:15.201 INFO:tasks.workunit.client.0.smithi090.stdout:5/953: rename d8/d76/dad/ff6 to d8/d9c/f138 0 2024-04-24T08:25:15.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/988: creat d0/d6/f134 x:0 0 0 2024-04-24T08:25:15.205 INFO:tasks.workunit.client.0.smithi090.stdout:1/872: fdatasync d3/d68/f8f 0 2024-04-24T08:25:15.206 INFO:tasks.workunit.client.0.smithi090.stdout:5/954: write d8/f9 [2044973,96938] 0 2024-04-24T08:25:15.208 INFO:tasks.workunit.client.0.smithi090.stdout:5/955: chown d8/f10 962289 1 2024-04-24T08:25:15.213 INFO:tasks.workunit.client.0.smithi090.stdout:9/860: fsync d0/dc/d24/d7c/f57 0 2024-04-24T08:25:15.214 INFO:tasks.workunit.client.0.smithi090.stdout:6/929: dread d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/fbb [0,4194304] 0 2024-04-24T08:25:15.216 INFO:tasks.workunit.client.0.smithi090.stdout:7/974: link db/d54/d83/c9e db/d54/ddb/c145 0 2024-04-24T08:25:15.217 INFO:tasks.workunit.client.0.smithi090.stdout:7/975: chown db/d54/ddb/ff6 7892 1 2024-04-24T08:25:15.220 INFO:tasks.workunit.client.0.smithi090.stdout:3/879: creat d2/df/db5/deb/f127 x:0 0 0 2024-04-24T08:25:15.220 INFO:tasks.workunit.client.0.smithi090.stdout:7/976: chown db/d17/d29/d32/d5a/d75/l138 164837 1 2024-04-24T08:25:15.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/873: fdatasync d3/d68/d7f/f9e 0 2024-04-24T08:25:15.224 INFO:tasks.workunit.client.0.smithi090.stdout:2/933: sync 2024-04-24T08:25:15.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/829: sync 2024-04-24T08:25:15.224 INFO:tasks.workunit.client.0.smithi090.stdout:8/833: sync 2024-04-24T08:25:15.226 INFO:tasks.workunit.client.0.smithi090.stdout:7/977: fdatasync db/d17/d29/d38/d99/f120 0 2024-04-24T08:25:15.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/874: read d3/d5/da/dc/d80/fba [18855,79471] 0 2024-04-24T08:25:15.234 INFO:tasks.workunit.client.0.smithi090.stdout:0/886: dread d1a/f52 [0,4194304] 0 2024-04-24T08:25:15.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/834: truncate d2/da/db0/ff5 401275 0 2024-04-24T08:25:15.235 INFO:tasks.workunit.client.0.smithi090.stdout:7/978: mkdir db/d17/d29/d32/d5a/d146 0 2024-04-24T08:25:15.236 INFO:tasks.workunit.client.0.smithi090.stdout:4/830: unlink d3/d8e/ddf/d5c/db5/db6/fe8 0 2024-04-24T08:25:15.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/875: dread - d3/d5/da/d97/d9c/dc4/ffb zero size 2024-04-24T08:25:15.242 INFO:tasks.workunit.client.0.smithi090.stdout:6/930: dwrite d8/d1d/d1f/d3e/d10a/d19/f64 [0,4194304] 0 2024-04-24T08:25:15.245 INFO:tasks.workunit.client.0.smithi090.stdout:7/979: write db/d17/d29/d32/d5a/f119 [465813,29405] 0 2024-04-24T08:25:15.245 INFO:tasks.workunit.client.1.smithi149.stdout:2/989: dwrite d0/f2 [0,4194304] 0 2024-04-24T08:25:15.249 INFO:tasks.workunit.client.1.smithi149.stdout:2/990: mkdir d0/d6/db/d39/d3d/d44/df0/df8/d135 0 2024-04-24T08:25:15.250 INFO:tasks.workunit.client.0.smithi090.stdout:0/887: dread - d1a/d4b/d53/d79/d58/d96/f11e zero size 2024-04-24T08:25:15.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/835: read - d2/da/d13/d64/d50/fb7 zero size 2024-04-24T08:25:15.253 INFO:tasks.workunit.client.0.smithi090.stdout:3/880: dwrite d2/df/d57/d4f/fe8 [4194304,4194304] 0 2024-04-24T08:25:15.253 INFO:tasks.workunit.client.0.smithi090.stdout:9/861: dwrite d0/d2a/d106/f10e [0,4194304] 0 2024-04-24T08:25:15.254 INFO:tasks.workunit.client.1.smithi149.stdout:2/991: stat d0/d6/db/d39/d3d/d44/d75/fff 0 2024-04-24T08:25:15.257 INFO:tasks.workunit.client.1.smithi149.stdout:2/992: write d0/d6/db/d39/d3d/fd6 [50942,115816] 0 2024-04-24T08:25:15.263 INFO:tasks.workunit.client.0.smithi090.stdout:1/876: creat d3/d5/da/dc/df4/f132 x:0 0 0 2024-04-24T08:25:15.265 INFO:tasks.workunit.client.0.smithi090.stdout:8/836: dread - d2/da/db0/f10d zero size 2024-04-24T08:25:15.265 INFO:tasks.workunit.client.0.smithi090.stdout:6/931: creat d8/d1d/d29/df0/f121 x:0 0 0 2024-04-24T08:25:15.269 INFO:tasks.workunit.client.0.smithi090.stdout:3/881: dread - d2/df/d1a/d37/d59/d87/ffb zero size 2024-04-24T08:25:15.272 INFO:tasks.workunit.client.0.smithi090.stdout:2/934: dwrite d0/d14/d17/f62 [0,4194304] 0 2024-04-24T08:25:15.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/956: dwrite d8/da/d3a/d2d/d37/f88 [0,4194304] 0 2024-04-24T08:25:15.280 INFO:tasks.workunit.client.0.smithi090.stdout:7/980: dread db/d17/d29/d32/dd4/ff4 [0,4194304] 0 2024-04-24T08:25:15.280 INFO:tasks.workunit.client.0.smithi090.stdout:6/932: stat d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/fec 0 2024-04-24T08:25:15.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/862: chown d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d92/fff 991413 1 2024-04-24T08:25:15.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/877: rename d3/d1d/d34/d73/cc6 to d3/d5/da/dc/df4/c133 0 2024-04-24T08:25:15.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/957: dread - d8/d99/f12f zero size 2024-04-24T08:25:15.286 INFO:tasks.workunit.client.1.smithi149.stdout:2/993: truncate d0/d6/db/d27/f47 2043076 0 2024-04-24T08:25:15.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/878: write d3/f41 [799624,37691] 0 2024-04-24T08:25:15.287 INFO:tasks.workunit.client.0.smithi090.stdout:0/888: link d1a/d4b/d53/d79/d82/d9d/lda d1a/d4b/d53/d79/d58/l12d 0 2024-04-24T08:25:15.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/958: chown d8/f31 44252 1 2024-04-24T08:25:15.289 INFO:tasks.workunit.client.0.smithi090.stdout:2/935: symlink d0/d14/d17/dc2/d135/dfe/l13d 0 2024-04-24T08:25:15.290 INFO:tasks.workunit.client.0.smithi090.stdout:8/837: write d2/d62/d6b/f90 [674916,46810] 0 2024-04-24T08:25:15.292 INFO:tasks.workunit.client.0.smithi090.stdout:3/882: mkdir d2/d15/d36/d6f/d126/d128 0 2024-04-24T08:25:15.294 INFO:tasks.workunit.client.0.smithi090.stdout:6/933: fsync d8/d1d/d1f/d3e/d44/fc0 0 2024-04-24T08:25:15.297 INFO:tasks.workunit.client.1.smithi149.stdout:2/994: symlink d0/d6/db/d28/d7f/d8f/db9/ddf/l136 0 2024-04-24T08:25:15.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/889: mkdir d1a/d4b/d53/d57/d12e 0 2024-04-24T08:25:15.301 INFO:tasks.workunit.client.0.smithi090.stdout:1/879: creat d3/d5/da/d94/da6/d115/d30/f134 x:0 0 0 2024-04-24T08:25:15.306 INFO:tasks.workunit.client.0.smithi090.stdout:9/863: link d0/dc/d24/d7c/d4d/d32/ff3 d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 0 2024-04-24T08:25:15.307 INFO:tasks.workunit.client.0.smithi090.stdout:7/981: dwrite db/dbd/f144 [0,4194304] 0 2024-04-24T08:25:15.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/880: truncate d3/d5/da/d97/d9c/dc4/ffb 1006110 0 2024-04-24T08:25:15.310 INFO:tasks.workunit.client.1.smithi149.stdout:2/995: getdents d0/d6/db/d27/d64/d66/db1/dc9 0 2024-04-24T08:25:15.312 INFO:tasks.workunit.client.0.smithi090.stdout:8/838: rename d2/d62/d6b/f110 to d2/da/d13/d64/d24/dad/f115 0 2024-04-24T08:25:15.314 INFO:tasks.workunit.client.0.smithi090.stdout:9/864: dread - d0/dc/f9c zero size 2024-04-24T08:25:15.315 INFO:tasks.workunit.client.0.smithi090.stdout:5/959: write d8/f31 [7474817,109662] 0 2024-04-24T08:25:15.319 INFO:tasks.workunit.client.0.smithi090.stdout:3/883: link d2/df/d1a/d37/d59/dcb/c11b d2/df/d1a/d37/d53/d60/c129 0 2024-04-24T08:25:15.319 INFO:tasks.workunit.client.0.smithi090.stdout:5/960: chown d8/d59/dc7/d7b/ff9 3100385 1 2024-04-24T08:25:15.323 INFO:tasks.workunit.client.0.smithi090.stdout:3/884: fdatasync d2/df/d1a/d37/dc0/ff6 0 2024-04-24T08:25:15.325 INFO:tasks.workunit.client.1.smithi149.stdout:2/996: getdents d0/d6/db/d27/d64 0 2024-04-24T08:25:15.327 INFO:tasks.workunit.client.1.smithi149.stdout:2/997: stat d0/d6/c89 0 2024-04-24T08:25:15.329 INFO:tasks.workunit.client.0.smithi090.stdout:7/982: unlink db/d17/d29/d32/f40 0 2024-04-24T08:25:15.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/934: dwrite d8/d1d/d1f/d3e/d44/d90/da2/fcd [0,4194304] 0 2024-04-24T08:25:15.331 INFO:tasks.workunit.client.0.smithi090.stdout:5/961: creat d8/d59/dc7/d7a/da1/dae/f139 x:0 0 0 2024-04-24T08:25:15.333 INFO:tasks.workunit.client.0.smithi090.stdout:7/983: truncate db/d17/d29/d38/d3e/fe7 5173551 0 2024-04-24T08:25:15.336 INFO:tasks.workunit.client.1.smithi149.stdout:2/998: creat d0/d6/db/d28/d115/f137 x:0 0 0 2024-04-24T08:25:15.336 INFO:tasks.workunit.client.0.smithi090.stdout:0/890: link d1a/d4b/d53/d79/d69/d8a/fa2 d1a/d4b/d53/d79/d3c/f12f 0 2024-04-24T08:25:15.336 INFO:tasks.workunit.client.0.smithi090.stdout:5/962: chown d8/d76/dad/fdb 981 1 2024-04-24T08:25:15.337 INFO:tasks.workunit.client.0.smithi090.stdout:7/984: write db/d54/f108 [2066785,23774] 0 2024-04-24T08:25:15.340 INFO:tasks.workunit.client.0.smithi090.stdout:9/865: link d0/dc/d24/d7c/d4d/d32/d9f/d72/c10c d0/dc/d24/d2d/d49/d62/dcb/c119 0 2024-04-24T08:25:15.341 INFO:tasks.workunit.client.1.smithi149.stdout:2/999: creat d0/d6/db/d28/d7f/d11e/f138 x:0 0 0 2024-04-24T08:25:15.341 INFO:tasks.workunit.client.0.smithi090.stdout:5/963: fsync d8/d7c/da9/fc6 0 2024-04-24T08:25:15.343 INFO:tasks.workunit.client.0.smithi090.stdout:8/839: dread d2/da/d13/d64/db2/dbb/fc0 [0,4194304] 0 2024-04-24T08:25:15.348 INFO:tasks.workunit.client.0.smithi090.stdout:0/891: read d1a/d4b/d53/d79/d58/d96/f10d [3190739,122234] 0 2024-04-24T08:25:15.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/985: truncate db/d17/d29/d38/d99/f105 540316 0 2024-04-24T08:25:15.350 INFO:tasks.workunit.client.1.smithi149.stderr:+ rm -rf -- ./tmp.o5tDYj6NmN 2024-04-24T08:25:15.351 INFO:tasks.workunit.client.0.smithi090.stdout:9/866: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f85 [340351,34340] 0 2024-04-24T08:25:15.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/840: write d2/da/d13/d64/d50/f57 [3078367,7946] 0 2024-04-24T08:25:15.351 INFO:tasks.workunit.client.0.smithi090.stdout:3/885: dwrite d2/df/d1a/d37/d53/f65 [0,4194304] 0 2024-04-24T08:25:15.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/935: dwrite d8/d1d/d1f/d3e/fd6 [0,4194304] 0 2024-04-24T08:25:15.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/881: dread d3/d5/da/dc/f40 [0,4194304] 0 2024-04-24T08:25:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/892: write d1a/d4b/d53/d79/d69/d78/fd7 [423710,109372] 0 2024-04-24T08:25:15.360 INFO:tasks.workunit.client.0.smithi090.stdout:5/964: truncate d8/da/d14/d2f/f47 2785639 0 2024-04-24T08:25:15.362 INFO:tasks.workunit.client.0.smithi090.stdout:9/867: dread d0/dc/f64 [0,4194304] 0 2024-04-24T08:25:15.362 INFO:tasks.workunit.client.0.smithi090.stdout:7/986: truncate db/d17/d29/d38/d3e/d46/f103 841047 0 2024-04-24T08:25:15.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/936: write d8/d1d/d1f/fb6 [2595856,107263] 0 2024-04-24T08:25:15.367 INFO:tasks.workunit.client.0.smithi090.stdout:3/886: link d2/l78 d2/d25/d124/l12a 0 2024-04-24T08:25:15.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/937: stat d8/d1d/d1f/d3e/d44/d90/cb9 0 2024-04-24T08:25:15.371 INFO:tasks.workunit.client.0.smithi090.stdout:1/882: write d3/d1d/d34/d73/ff6 [796550,15470] 0 2024-04-24T08:25:15.372 INFO:tasks.workunit.client.0.smithi090.stdout:6/938: dread - d8/d1d/d29/fe3 zero size 2024-04-24T08:25:15.373 INFO:tasks.workunit.client.0.smithi090.stdout:8/841: truncate d2/da/d13/d64/f4e 2514726 0 2024-04-24T08:25:15.373 INFO:tasks.workunit.client.0.smithi090.stdout:9/868: unlink d0/dc/d24/d2d/d49/c71 0 2024-04-24T08:25:15.374 INFO:tasks.workunit.client.0.smithi090.stdout:0/893: truncate d1a/d4b/d53/d79/d82/f8d 467057 0 2024-04-24T08:25:15.375 INFO:tasks.workunit.client.0.smithi090.stdout:3/887: rmdir d2/d15/dc4 39 2024-04-24T08:25:15.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/965: mknod d8/d59/dc7/c13a 0 2024-04-24T08:25:15.378 INFO:tasks.workunit.client.0.smithi090.stdout:6/939: mknod d8/d1d/d1f/d3e/d7a/c122 0 2024-04-24T08:25:15.379 INFO:tasks.workunit.client.0.smithi090.stdout:3/888: write d2/d90/f118 [891035,29705] 0 2024-04-24T08:25:15.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/869: symlink d0/dc/d24/d7c/d4d/d32/l11a 0 2024-04-24T08:25:15.385 INFO:tasks.workunit.client.0.smithi090.stdout:7/987: dwrite db/d17/d29/d32/f3d [0,4194304] 0 2024-04-24T08:25:15.387 INFO:tasks.workunit.client.0.smithi090.stdout:9/870: dread - d0/dc/d24/d7c/d4d/ff6 zero size 2024-04-24T08:25:15.387 INFO:tasks.workunit.client.0.smithi090.stdout:0/894: symlink d1a/d4b/d53/d79/d69/d78/d9b/l130 0 2024-04-24T08:25:15.392 INFO:tasks.workunit.client.0.smithi090.stdout:3/889: unlink d2/d15/d36/d6f/d9d/ff0 0 2024-04-24T08:25:15.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/842: rename d2/c14 to d2/da/d13/d64/d24/dad/d10c/c116 0 2024-04-24T08:25:15.402 INFO:tasks.workunit.client.0.smithi090.stdout:7/988: dwrite db/dab/fb3 [0,4194304] 0 2024-04-24T08:25:15.404 INFO:tasks.workunit.client.0.smithi090.stdout:6/940: dwrite d8/d1d/f6f [0,4194304] 0 2024-04-24T08:25:15.413 INFO:tasks.workunit.client.0.smithi090.stdout:8/843: mkdir d2/da/d13/d60/d91/d117 0 2024-04-24T08:25:15.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/883: write d3/d5/da/d94/da6/d115/fd4 [854959,93788] 0 2024-04-24T08:25:15.417 INFO:tasks.workunit.client.0.smithi090.stdout:3/890: chown d2/d15/dc4/f10b 9239731 1 2024-04-24T08:25:15.418 INFO:tasks.workunit.client.0.smithi090.stdout:8/844: dread - d2/da/d13/d64/d50/f10b zero size 2024-04-24T08:25:15.418 INFO:tasks.workunit.client.0.smithi090.stdout:6/941: chown d8/d1d/d1f/d3e/d44/d90/da2/c101 799 1 2024-04-24T08:25:15.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/966: dread d8/faf [0,4194304] 0 2024-04-24T08:25:15.428 INFO:tasks.workunit.client.0.smithi090.stdout:9/871: dwrite d0/dc/d24/d2d/d49/fb3 [4194304,4194304] 0 2024-04-24T08:25:15.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/989: mknod db/d17/c147 0 2024-04-24T08:25:15.441 INFO:tasks.workunit.client.0.smithi090.stdout:7/990: truncate db/d54/d78/f10e 189085 0 2024-04-24T08:25:15.443 INFO:tasks.workunit.client.0.smithi090.stdout:1/884: mknod d3/d5/da/d94/da6/d115/d30/d9d/da2/c135 0 2024-04-24T08:25:15.470 INFO:tasks.workunit.client.0.smithi090.stdout:5/967: chown d8/d9c/fc5 2070708327 1 2024-04-24T08:25:15.471 INFO:tasks.workunit.client.0.smithi090.stdout:1/885: fsync d3/d68/d7f/d11b/f123 0 2024-04-24T08:25:15.471 INFO:tasks.workunit.client.0.smithi090.stdout:9/872: read d0/d19/f46 [50899,101844] 0 2024-04-24T08:25:15.475 INFO:tasks.workunit.client.0.smithi090.stdout:0/895: getdents d1a/d11d 0 2024-04-24T08:25:15.493 INFO:tasks.workunit.client.0.smithi090.stdout:9/873: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110/d11b 0 2024-04-24T08:25:15.494 INFO:tasks.workunit.client.0.smithi090.stdout:1/886: rename d3/d68/c10a to d3/d5/da/c136 0 2024-04-24T08:25:15.498 INFO:tasks.workunit.client.0.smithi090.stdout:0/896: rename d1a/d4b/d53/d79/d35/d62/fc3 to d1a/d4b/d53/d79/d35/f131 0 2024-04-24T08:25:15.499 INFO:tasks.workunit.client.0.smithi090.stdout:3/891: dwrite d2/d15/d36/d6f/d7a/f8f [4194304,4194304] 0 2024-04-24T08:25:15.501 INFO:tasks.workunit.client.0.smithi090.stdout:5/968: dwrite f6 [0,4194304] 0 2024-04-24T08:25:15.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/991: dread db/d17/d29/d32/f57 [0,4194304] 0 2024-04-24T08:25:15.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/969: chown d8/l32 65897404 1 2024-04-24T08:25:15.512 INFO:tasks.workunit.client.0.smithi090.stdout:9/874: read - d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/fb7 zero size 2024-04-24T08:25:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:3/892: mkdir d2/df/d1a/d37/d53/d60/d9b/d12b 0 2024-04-24T08:25:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/845: dwrite d2/da/d13/d64/d24/db3/db4/dcd/de8/ffd [0,4194304] 0 2024-04-24T08:25:15.517 INFO:tasks.workunit.client.0.smithi090.stdout:3/893: dread - d2/d15/dc4/f10b zero size 2024-04-24T08:25:15.518 INFO:tasks.workunit.client.0.smithi090.stdout:1/887: read d3/d5/da/d94/da6/d115/d30/d9d/da2/fde [58108,64229] 0 2024-04-24T08:25:15.522 INFO:tasks.workunit.client.0.smithi090.stdout:7/992: rename db/d17/d29/d32/d5a/d75/l138 to db/de9/l148 0 2024-04-24T08:25:15.527 INFO:tasks.workunit.client.0.smithi090.stdout:3/894: dread - d2/df/d1a/d37/d59/d87/ffb zero size 2024-04-24T08:25:15.529 INFO:tasks.workunit.client.0.smithi090.stdout:0/897: dwrite d1a/d4b/d53/d79/fcf [0,4194304] 0 2024-04-24T08:25:15.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/970: dwrite d8/d59/dc7/d7a/d50/f53 [0,4194304] 0 2024-04-24T08:25:15.536 INFO:tasks.workunit.client.0.smithi090.stdout:7/993: mknod db/d17/d29/d32/d5a/d75/c149 0 2024-04-24T08:25:15.536 INFO:tasks.workunit.client.0.smithi090.stdout:8/846: dread d2/d62/f6e [0,4194304] 0 2024-04-24T08:25:15.537 INFO:tasks.workunit.client.0.smithi090.stdout:7/994: chown db/d54/ddb 226 1 2024-04-24T08:25:15.538 INFO:tasks.workunit.client.0.smithi090.stdout:3/895: creat d2/df/d1a/d37/dc0/f12c x:0 0 0 2024-04-24T08:25:15.544 INFO:tasks.workunit.client.0.smithi090.stdout:0/898: symlink d1a/d4b/d53/d79/d69/d90/d10e/l132 0 2024-04-24T08:25:15.546 INFO:tasks.workunit.client.0.smithi090.stdout:0/899: chown d1a/d4b/d53/d79/d82/f123 10250471 1 2024-04-24T08:25:15.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/831: sync 2024-04-24T08:25:15.550 INFO:tasks.workunit.client.0.smithi090.stdout:2/936: sync 2024-04-24T08:25:15.555 INFO:tasks.workunit.client.0.smithi090.stdout:8/847: unlink d2/l8b 0 2024-04-24T08:25:15.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/995: dread db/d17/d29/d38/f53 [4194304,4194304] 0 2024-04-24T08:25:15.558 INFO:tasks.workunit.client.0.smithi090.stdout:5/971: dwrite d8/d7c/da9/d118/da6/fcc [0,4194304] 0 2024-04-24T08:25:15.562 INFO:tasks.workunit.client.0.smithi090.stdout:3/896: getdents d2/d15/d36 0 2024-04-24T08:25:15.563 INFO:tasks.workunit.client.0.smithi090.stdout:1/888: write d3/d5/da/d94/da6/d115/d30/d9d/f113 [1863234,19918] 0 2024-04-24T08:25:15.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/972: symlink d8/d25/l13b 0 2024-04-24T08:25:15.567 INFO:tasks.workunit.client.0.smithi090.stdout:7/996: symlink db/de9/l14a 0 2024-04-24T08:25:15.568 INFO:tasks.workunit.client.0.smithi090.stdout:1/889: read - d3/d5/da/d94/da6/d115/d30/f7e zero size 2024-04-24T08:25:15.570 INFO:tasks.workunit.client.0.smithi090.stdout:5/973: chown d8/da/d3a/d2d/c3c 7108972 1 2024-04-24T08:25:15.571 INFO:tasks.workunit.client.0.smithi090.stdout:2/937: creat d0/d7/d67/d7f/d31/f13e x:0 0 0 2024-04-24T08:25:15.574 INFO:tasks.workunit.client.0.smithi090.stdout:3/897: mknod d2/df/d21/c12d 0 2024-04-24T08:25:15.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/875: dwrite d0/f53 [0,4194304] 0 2024-04-24T08:25:15.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/997: rename db/d17/d29/d32/d5a/d75/d101/f106 to db/d54/ddb/f14b 0 2024-04-24T08:25:15.584 INFO:tasks.workunit.client.0.smithi090.stdout:9/876: creat d0/dc/d24/d7c/f11c x:0 0 0 2024-04-24T08:25:15.585 INFO:tasks.workunit.client.0.smithi090.stdout:0/900: dwrite d1a/d4b/d53/d79/d58/db4/fed [0,4194304] 0 2024-04-24T08:25:15.587 INFO:tasks.workunit.client.0.smithi090.stdout:1/890: rename d3/d1d/d34/c11c to d3/d5/da/d97/d9c/c137 0 2024-04-24T08:25:15.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/832: dwrite d3/d8e/ddf/d5c/f66 [0,4194304] 0 2024-04-24T08:25:15.595 INFO:tasks.workunit.client.0.smithi090.stdout:9/877: chown d0/dc/f29 2 1 2024-04-24T08:25:15.597 INFO:tasks.workunit.client.0.smithi090.stdout:8/848: dwrite d2/da/d13/f52 [0,4194304] 0 2024-04-24T08:25:15.600 INFO:tasks.workunit.client.0.smithi090.stdout:9/878: chown d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96 0 1 2024-04-24T08:25:15.601 INFO:tasks.workunit.client.0.smithi090.stdout:4/833: truncate d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/ff3 753020 0 2024-04-24T08:25:15.603 INFO:tasks.workunit.client.0.smithi090.stdout:0/901: mknod d1a/d4b/d53/d79/d35/dea/dfe/c133 0 2024-04-24T08:25:15.609 INFO:tasks.workunit.client.0.smithi090.stdout:3/898: dread d2/d15/f2e [0,4194304] 0 2024-04-24T08:25:15.609 INFO:tasks.workunit.client.0.smithi090.stdout:5/974: rename d8/d7c/da9/fc6 to d8/da/d14/d2f/d112/dcb/de3/f13c 0 2024-04-24T08:25:15.612 INFO:tasks.workunit.client.0.smithi090.stdout:1/891: creat d3/d68/d7f/f138 x:0 0 0 2024-04-24T08:25:15.614 INFO:tasks.workunit.client.0.smithi090.stdout:2/938: creat d0/d7/d67/d7f/d31/d46/f13f x:0 0 0 2024-04-24T08:25:15.616 INFO:tasks.workunit.client.0.smithi090.stdout:6/942: sync 2024-04-24T08:25:15.616 INFO:tasks.workunit.client.0.smithi090.stdout:8/849: write d2/da/d13/f4b [1994619,48723] 0 2024-04-24T08:25:15.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/850: chown d2/da/c78 86 1 2024-04-24T08:25:15.622 INFO:tasks.workunit.client.0.smithi090.stdout:5/975: rmdir d8/d7c/da9/d118 39 2024-04-24T08:25:15.622 INFO:tasks.workunit.client.0.smithi090.stdout:0/902: dwrite d1a/d4b/d53/d79/d35/f42 [0,4194304] 0 2024-04-24T08:25:15.622 INFO:tasks.workunit.client.0.smithi090.stdout:7/998: write db/de9/ff5 [536597,98742] 0 2024-04-24T08:25:15.623 INFO:tasks.workunit.client.0.smithi090.stdout:3/899: dread d2/df/d1a/d37/d53/f70 [0,4194304] 0 2024-04-24T08:25:15.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/892: chown d3/d5/da/dc/d80/fff 195 1 2024-04-24T08:25:15.626 INFO:tasks.workunit.client.0.smithi090.stdout:0/903: write d1a/d4b/d53/d79/d35/db8/f10c [477962,23389] 0 2024-04-24T08:25:15.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/893: chown d3/d5/da/dc/d80/fba 461205988 1 2024-04-24T08:25:15.633 INFO:tasks.workunit.client.0.smithi090.stdout:5/976: symlink d8/da/d3a/d2d/d37/dec/dfe/l13d 0 2024-04-24T08:25:15.633 INFO:tasks.workunit.client.0.smithi090.stdout:3/900: truncate d2/df/db5/deb/f127 237798 0 2024-04-24T08:25:15.634 INFO:tasks.workunit.client.0.smithi090.stdout:6/943: fdatasync d8/d1d/d1f/f58 0 2024-04-24T08:25:15.634 INFO:tasks.workunit.client.0.smithi090.stdout:8/851: mknod d2/da/db0/d111/c118 0 2024-04-24T08:25:15.636 INFO:tasks.workunit.client.0.smithi090.stdout:8/852: write d2/f9d [326233,85285] 0 2024-04-24T08:25:15.643 INFO:tasks.workunit.client.0.smithi090.stdout:7/999: getdents db/dab/d109/d129 0 2024-04-24T08:25:15.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/977: unlink d8/da/d3a/d2d/d37/lb3 0 2024-04-24T08:25:15.653 INFO:tasks.workunit.client.0.smithi090.stdout:2/939: write d0/d7/f94 [124742,4308] 0 2024-04-24T08:25:15.654 INFO:tasks.workunit.client.0.smithi090.stdout:5/978: fsync d8/d99/f11e 0 2024-04-24T08:25:15.656 INFO:tasks.workunit.client.0.smithi090.stdout:4/834: truncate d3/d8e/fed 809910 0 2024-04-24T08:25:15.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/940: chown d0/fad 9 1 2024-04-24T08:25:15.657 INFO:tasks.workunit.client.0.smithi090.stdout:0/904: creat d1a/d4b/f134 x:0 0 0 2024-04-24T08:25:15.658 INFO:tasks.workunit.client.0.smithi090.stdout:9/879: dwrite d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf [0,4194304] 0 2024-04-24T08:25:15.663 INFO:tasks.workunit.client.0.smithi090.stdout:1/894: rename d3/d5/da/dc/df4/c133 to d3/d5/da/d94/d10b/c139 0 2024-04-24T08:25:15.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/853: mkdir d2/da/d119 0 2024-04-24T08:25:15.667 INFO:tasks.workunit.client.0.smithi090.stdout:2/941: readlink d0/d2e/l3e 0 2024-04-24T08:25:15.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/901: dread d2/df/d57/d4f/fe8 [0,4194304] 0 2024-04-24T08:25:15.671 INFO:tasks.workunit.client.0.smithi090.stdout:4/835: dwrite d3/d8e/ddf/fef [0,4194304] 0 2024-04-24T08:25:15.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/979: creat d8/da/d3a/d2d/d37/dec/f13e x:0 0 0 2024-04-24T08:25:15.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/880: rmdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110 39 2024-04-24T08:25:15.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/905: rmdir d1a/d4b 39 2024-04-24T08:25:15.684 INFO:tasks.workunit.client.0.smithi090.stdout:8/854: chown d2/da/d13/d64/d24/db3/db4/lf3 66 1 2024-04-24T08:25:15.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/944: truncate d8/d1d/d1f/d3e/d4d/f99 3010035 0 2024-04-24T08:25:15.689 INFO:tasks.workunit.client.0.smithi090.stdout:4/836: rmdir d3/d8e/ddf/d5c/db5/d69/d8b/db2 39 2024-04-24T08:25:15.690 INFO:tasks.workunit.client.0.smithi090.stdout:5/980: symlink d8/da/d3a/d2d/l13f 0 2024-04-24T08:25:15.691 INFO:tasks.workunit.client.0.smithi090.stdout:4/837: chown d3/d24/f44 62603261 1 2024-04-24T08:25:15.693 INFO:tasks.workunit.client.0.smithi090.stdout:5/981: fsync d8/da/d14/d2f/d112/dcb/f1e 0 2024-04-24T08:25:15.700 INFO:tasks.workunit.client.0.smithi090.stdout:8/855: rmdir d2/da/db0/d111 39 2024-04-24T08:25:15.701 INFO:tasks.workunit.client.0.smithi090.stdout:6/945: rmdir d8/d1d/d1f/d3e/d44/d90/d106 39 2024-04-24T08:25:15.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/902: dwrite d2/df/d1a/d37/f55 [0,4194304] 0 2024-04-24T08:25:15.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/881: dread d0/d19/f1a [0,4194304] 0 2024-04-24T08:25:15.704 INFO:tasks.workunit.client.0.smithi090.stdout:1/895: dwrite d3/d5/d3e/da3/fb2 [0,4194304] 0 2024-04-24T08:25:15.717 INFO:tasks.workunit.client.0.smithi090.stdout:5/982: dread d8/d7c/da9/d118/f98 [0,4194304] 0 2024-04-24T08:25:15.720 INFO:tasks.workunit.client.0.smithi090.stdout:2/942: rename d0/d7/d28/l35 to d0/d7/de1/l140 0 2024-04-24T08:25:15.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/856: mkdir d2/da/d13/d64/d24/db3/db4/dcd/d11a 0 2024-04-24T08:25:15.726 INFO:tasks.workunit.client.0.smithi090.stdout:3/903: creat d2/d25/d33/f12e x:0 0 0 2024-04-24T08:25:15.729 INFO:tasks.workunit.client.0.smithi090.stdout:9/882: fsync d0/dc/d24/d7c/d4d/f52 0 2024-04-24T08:25:15.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/943: fsync d0/f3b 0 2024-04-24T08:25:15.736 INFO:tasks.workunit.client.0.smithi090.stdout:2/944: chown d0/d7/d28/daa/ddc/d12a/d133/l6c 125123386 1 2024-04-24T08:25:15.741 INFO:tasks.workunit.client.0.smithi090.stdout:3/904: dread d2/df/d1a/f6d [0,4194304] 0 2024-04-24T08:25:15.741 INFO:tasks.workunit.client.0.smithi090.stdout:9/883: creat d0/dc/d24/d7c/f11d x:0 0 0 2024-04-24T08:25:15.743 INFO:tasks.workunit.client.0.smithi090.stdout:3/905: chown d2/df/d57/d4f/l66 0 1 2024-04-24T08:25:15.745 INFO:tasks.workunit.client.0.smithi090.stdout:3/906: write d2/d15/d36/d6f/d9d/fa4 [2379063,121845] 0 2024-04-24T08:25:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/946: mknod d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/df6/c123 0 2024-04-24T08:25:15.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/947: stat d8/d1d/d1f/d3e/d10a/d19/dd2/f104 0 2024-04-24T08:25:15.761 INFO:tasks.workunit.client.0.smithi090.stdout:8/857: symlink d2/da/d13/d64/l11b 0 2024-04-24T08:25:15.763 INFO:tasks.workunit.client.0.smithi090.stdout:8/858: chown d2/d62/fbc 51 1 2024-04-24T08:25:15.774 INFO:tasks.workunit.client.0.smithi090.stdout:5/983: dwrite d8/da/d14/f94 [0,4194304] 0 2024-04-24T08:25:15.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:15 smithi149 ceph-mon[31282]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 65 MiB/s rd, 160 MiB/s wr, 349 op/s 2024-04-24T08:25:15.790 INFO:tasks.workunit.client.0.smithi090.stdout:8/859: dwrite d2/d62/d6b/f90 [0,4194304] 0 2024-04-24T08:25:15.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:15 smithi090 ceph-mon[25016]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 65 MiB/s rd, 160 MiB/s wr, 349 op/s 2024-04-24T08:25:15.908 INFO:tasks.workunit.client.0.smithi090.stdout:3/907: mknod d2/d15/dc4/c12f 0 2024-04-24T08:25:15.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/908: dread d2/df/d1a/f6d [0,4194304] 0 2024-04-24T08:25:15.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/884: mknod d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110/d11b/c11e 0 2024-04-24T08:25:15.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/885: dread - d0/dc/d24/d7c/d4d/ff6 zero size 2024-04-24T08:25:15.941 INFO:tasks.workunit.client.0.smithi090.stdout:9/886: read - d0/dc/d24/d7c/f11d zero size 2024-04-24T08:25:15.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/909: getdents d2/df/d5b/d105 0 2024-04-24T08:25:15.943 INFO:tasks.workunit.client.0.smithi090.stdout:2/945: dwrite d0/d7/d6f/fc6 [0,4194304] 0 2024-04-24T08:25:15.949 INFO:tasks.workunit.client.0.smithi090.stdout:3/910: write d2/df/d21/ffc [409114,6693] 0 2024-04-24T08:25:15.959 INFO:tasks.workunit.client.0.smithi090.stdout:2/946: mkdir d0/d7/d57/d79/d141 0 2024-04-24T08:25:15.960 INFO:tasks.workunit.client.0.smithi090.stdout:3/911: truncate d2/d15/d36/d6f/f9a 980537 0 2024-04-24T08:25:15.965 INFO:tasks.workunit.client.0.smithi090.stdout:9/887: creat d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110/d11b/f11f x:0 0 0 2024-04-24T08:25:15.970 INFO:tasks.workunit.client.0.smithi090.stdout:9/888: dread - d0/dc/d24/d2d/d49/dfa/f117 zero size 2024-04-24T08:25:15.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/912: creat d2/df/d1a/d37/d59/d98/dfd/f130 x:0 0 0 2024-04-24T08:25:16.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/947: dwrite d0/d5e/ffc [0,4194304] 0 2024-04-24T08:25:16.003 INFO:tasks.workunit.client.0.smithi090.stdout:9/889: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 [0,4194304] 0 2024-04-24T08:25:16.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/890: chown d0/l8 3 1 2024-04-24T08:25:16.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/948: fdatasync d0/f9e 0 2024-04-24T08:25:16.066 INFO:tasks.workunit.client.0.smithi090.stdout:6/948: write d8/d1d/d1f/d3e/d44/fe2 [1047935,89577] 0 2024-04-24T08:25:16.082 INFO:tasks.workunit.client.0.smithi090.stdout:6/949: link d8/d1d/d1f/d3e/d44/d90/d106/d38/f11d d8/d1d/d1f/dc2/f124 0 2024-04-24T08:25:16.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/950: mknod d8/d1d/d2f/c125 0 2024-04-24T08:25:16.109 INFO:tasks.workunit.client.0.smithi090.stdout:4/838: rename d3/d24/d92 to d3/d8e/ddf/d109 0 2024-04-24T08:25:16.115 INFO:tasks.workunit.client.0.smithi090.stdout:4/839: mknod d3/d24/c10a 0 2024-04-24T08:25:16.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/896: rename d3/d5/ca0 to d3/d68/c13a 0 2024-04-24T08:25:16.119 INFO:tasks.workunit.client.0.smithi090.stdout:2/949: truncate d0/d7/de1/f114 498003 0 2024-04-24T08:25:16.122 INFO:tasks.workunit.client.0.smithi090.stdout:4/840: fsync d3/d8e/ddf/d5c/f3d 0 2024-04-24T08:25:16.123 INFO:tasks.workunit.client.0.smithi090.stdout:3/913: dwrite d2/df/d1a/d37/d59/dcb/fd2 [0,4194304] 0 2024-04-24T08:25:16.143 INFO:tasks.workunit.client.0.smithi090.stdout:0/906: rename d1a/d4b/d53/d79/d3c/f91 to d1a/d4b/d53/d79/d35/dea/f135 0 2024-04-24T08:25:16.148 INFO:tasks.workunit.client.0.smithi090.stdout:4/841: fsync d3/d8e/ddf/fbc 0 2024-04-24T08:25:16.153 INFO:tasks.workunit.client.0.smithi090.stdout:2/950: mknod d0/d7/d28/d124/dcd/c142 0 2024-04-24T08:25:16.154 INFO:tasks.workunit.client.0.smithi090.stdout:8/860: rename d2/da/d13/d64/d24/db3/db4/lf3 to d2/da/d13/d64/db2/d107/l11c 0 2024-04-24T08:25:16.155 INFO:tasks.workunit.client.0.smithi090.stdout:0/907: fsync d1a/d4b/d53/d79/d35/f42 0 2024-04-24T08:25:16.155 INFO:tasks.workunit.client.0.smithi090.stdout:5/984: link d8/d59/dc7/d7b/f80 d8/da/d3a/d2d/f140 0 2024-04-24T08:25:16.162 INFO:tasks.workunit.client.0.smithi090.stdout:5/985: fdatasync d8/f9e 0 2024-04-24T08:25:16.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/951: stat d0/d7/d28/daa/def/c6d 0 2024-04-24T08:25:16.163 INFO:tasks.workunit.client.0.smithi090.stdout:1/897: rename d3/d1d/d34/d73/dd8/d109/c11d to d3/d68/c13b 0 2024-04-24T08:25:16.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/842: write d3/d24/f2b [6075038,25427] 0 2024-04-24T08:25:16.168 INFO:tasks.workunit.client.0.smithi090.stdout:3/914: creat d2/d15/f131 x:0 0 0 2024-04-24T08:25:16.173 INFO:tasks.workunit.client.0.smithi090.stdout:8/861: symlink d2/da/d13/d60/l11d 0 2024-04-24T08:25:16.174 INFO:tasks.workunit.client.0.smithi090.stdout:9/891: sync 2024-04-24T08:25:16.176 INFO:tasks.workunit.client.0.smithi090.stdout:5/986: rmdir d8/da/d14/d2f/d112/dcb 39 2024-04-24T08:25:16.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/915: chown d2/d25/d124/l12a 396772864 1 2024-04-24T08:25:16.180 INFO:tasks.workunit.client.0.smithi090.stdout:1/898: creat d3/d5/da/dc/d80/deb/f13c x:0 0 0 2024-04-24T08:25:16.187 INFO:tasks.workunit.client.0.smithi090.stdout:0/908: write d1a/d4b/d53/d79/d69/d90/fdc [517812,107549] 0 2024-04-24T08:25:16.192 INFO:tasks.workunit.client.0.smithi090.stdout:1/899: read d3/d1d/d34/d5a/f5b [3190245,110590] 0 2024-04-24T08:25:16.193 INFO:tasks.workunit.client.0.smithi090.stdout:4/843: mkdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b 0 2024-04-24T08:25:16.194 INFO:tasks.workunit.client.0.smithi090.stdout:5/987: mkdir d8/d59/dc7/d141 0 2024-04-24T08:25:16.198 INFO:tasks.workunit.client.0.smithi090.stdout:8/862: creat d2/da/d13/d60/f11e x:0 0 0 2024-04-24T08:25:16.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/892: dread d0/f53 [0,4194304] 0 2024-04-24T08:25:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/988: fdatasync d8/faf 0 2024-04-24T08:25:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/844: stat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/fae 0 2024-04-24T08:25:16.227 INFO:tasks.workunit.client.0.smithi090.stdout:1/900: unlink d3/d5/da/d94/da6/d115/l7b 0 2024-04-24T08:25:16.227 INFO:tasks.workunit.client.0.smithi090.stdout:2/952: dwrite d0/f9e [0,4194304] 0 2024-04-24T08:25:16.230 INFO:tasks.workunit.client.0.smithi090.stdout:8/863: unlink d2/da/d13/d64/d24/c89 0 2024-04-24T08:25:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:5/989: chown d8/da/d14/ld2 1836752 1 2024-04-24T08:25:16.240 INFO:tasks.workunit.client.0.smithi090.stdout:2/953: symlink d0/d7/d28/daa/l143 0 2024-04-24T08:25:16.241 INFO:tasks.workunit.client.0.smithi090.stdout:8/864: write d2/f3 [8643450,30639] 0 2024-04-24T08:25:16.242 INFO:tasks.workunit.client.0.smithi090.stdout:4/845: symlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/l10c 0 2024-04-24T08:25:16.246 INFO:tasks.workunit.client.0.smithi090.stdout:9/893: symlink d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/l120 0 2024-04-24T08:25:16.246 INFO:tasks.workunit.client.0.smithi090.stdout:2/954: rmdir d0/d7 39 2024-04-24T08:25:16.249 INFO:tasks.workunit.client.0.smithi090.stdout:3/916: dwrite d2/fa [4194304,4194304] 0 2024-04-24T08:25:16.249 INFO:tasks.workunit.client.0.smithi090.stdout:4/846: dread - d3/d8e/ddf/f94 zero size 2024-04-24T08:25:16.264 INFO:tasks.workunit.client.0.smithi090.stdout:1/901: dwrite d3/d5/da/d97/d9c/fc0 [0,4194304] 0 2024-04-24T08:25:16.264 INFO:tasks.workunit.client.0.smithi090.stdout:9/894: mknod d0/dc/d24/d2d/d49/d62/dcb/dcd/c121 0 2024-04-24T08:25:16.267 INFO:tasks.workunit.client.0.smithi090.stdout:9/895: readlink d0/dc/d24/d7c/d4d/le4 0 2024-04-24T08:25:16.269 INFO:tasks.workunit.client.0.smithi090.stdout:1/902: chown d3/d1d/d34/d73/lf5 981900172 1 2024-04-24T08:25:16.276 INFO:tasks.workunit.client.0.smithi090.stdout:8/865: rename d2/d62/d83/da3/deb/l10f to d2/d62/l11f 0 2024-04-24T08:25:16.276 INFO:tasks.workunit.client.0.smithi090.stdout:1/903: fdatasync f0 0 2024-04-24T08:25:16.276 INFO:tasks.workunit.client.0.smithi090.stdout:6/951: sync 2024-04-24T08:25:16.279 INFO:tasks.workunit.client.0.smithi090.stdout:2/955: getdents d0/d14/d17/dfa 0 2024-04-24T08:25:16.283 INFO:tasks.workunit.client.0.smithi090.stdout:6/952: chown d8/d1d/d1f/d3e/d10a/d19/l2d 2567 1 2024-04-24T08:25:16.285 INFO:tasks.workunit.client.0.smithi090.stdout:4/847: dwrite d3/d87/f9f [0,4194304] 0 2024-04-24T08:25:16.288 INFO:tasks.workunit.client.0.smithi090.stdout:4/848: write d3/d8e/ddf/f70 [4277686,80967] 0 2024-04-24T08:25:16.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/917: dread d2/df/d1a/d37/f8a [0,4194304] 0 2024-04-24T08:25:16.288 INFO:tasks.workunit.client.0.smithi090.stdout:2/956: creat d0/d14/d17/dc2/dff/f144 x:0 0 0 2024-04-24T08:25:16.289 INFO:tasks.workunit.client.0.smithi090.stdout:9/896: rename d0/dc/d24 to d0/dc/d24/d2d/d49/d122 22 2024-04-24T08:25:16.290 INFO:tasks.workunit.client.0.smithi090.stdout:4/849: stat d3/d8e/ce5 0 2024-04-24T08:25:16.292 INFO:tasks.workunit.client.0.smithi090.stdout:9/897: chown d0/d2a/f8e 6 1 2024-04-24T08:25:16.293 INFO:tasks.workunit.client.0.smithi090.stdout:4/850: fsync d3/d8e/ddf/d5c/db5/d69/d8b/db2/f7e 0 2024-04-24T08:25:16.296 INFO:tasks.workunit.client.0.smithi090.stdout:6/953: symlink d8/d1d/d1f/d3e/d44/d90/d106/l126 0 2024-04-24T08:25:16.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/918: write d2/d25/d33/f80 [2572216,33469] 0 2024-04-24T08:25:16.299 INFO:tasks.workunit.client.0.smithi090.stdout:9/898: symlink d0/dc/d24/d7c/d4d/d8c/l123 0 2024-04-24T08:25:16.300 INFO:tasks.workunit.client.0.smithi090.stdout:8/866: link d2/da/d13/d64/f86 d2/d62/d6b/d9c/f120 0 2024-04-24T08:25:16.306 INFO:tasks.workunit.client.0.smithi090.stdout:6/954: creat d8/d1d/d1f/d3e/d7a/f127 x:0 0 0 2024-04-24T08:25:16.307 INFO:tasks.workunit.client.0.smithi090.stdout:3/919: readlink d2/d25/d33/l96 0 2024-04-24T08:25:16.311 INFO:tasks.workunit.client.0.smithi090.stdout:8/867: creat d2/da/d13/d60/d91/f121 x:0 0 0 2024-04-24T08:25:16.315 INFO:tasks.workunit.client.0.smithi090.stdout:2/957: rename d0/d7/d57/lbc to d0/d7/d67/d7f/d31/l145 0 2024-04-24T08:25:16.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/958: write d0/d7/de1/f43 [3350241,45234] 0 2024-04-24T08:25:16.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/955: symlink d8/d1d/d1f/d3e/d10a/d19/l128 0 2024-04-24T08:25:16.321 INFO:tasks.workunit.client.0.smithi090.stdout:4/851: link d3/dd0/lee d3/d8e/ddf/d5c/db5/d69/l10d 0 2024-04-24T08:25:16.322 INFO:tasks.workunit.client.0.smithi090.stdout:3/920: dwrite d2/d90/fe4 [0,4194304] 0 2024-04-24T08:25:16.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/959: chown d0/d7/d28/daa/db1/lf9 0 1 2024-04-24T08:25:16.324 INFO:tasks.workunit.client.0.smithi090.stdout:9/899: creat d0/dc/d24/d7c/d4d/d32/d9f/d72/f124 x:0 0 0 2024-04-24T08:25:16.324 INFO:tasks.workunit.client.0.smithi090.stdout:2/960: chown d0/d14/d17/l25 103 1 2024-04-24T08:25:16.325 INFO:tasks.workunit.client.0.smithi090.stdout:2/961: chown d0/d7/d67/f72 1483396 1 2024-04-24T08:25:16.326 INFO:tasks.workunit.client.0.smithi090.stdout:6/956: mknod d8/d1d/d1f/d3e/d44/c129 0 2024-04-24T08:25:16.328 INFO:tasks.workunit.client.0.smithi090.stdout:6/957: readlink d8/d1d/d1f/d3e/d10a/d19/l6b 0 2024-04-24T08:25:16.328 INFO:tasks.workunit.client.0.smithi090.stdout:8/868: write d2/da/d13/d64/d50/f10b [151454,127360] 0 2024-04-24T08:25:16.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/921: rmdir d2/d25/d33/dc8 39 2024-04-24T08:25:16.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/852: creat d3/d8e/ddf/d5c/db5/db6/f10e x:0 0 0 2024-04-24T08:25:16.330 INFO:tasks.workunit.client.0.smithi090.stdout:3/922: dread - d2/d25/d33/f12e zero size 2024-04-24T08:25:16.332 INFO:tasks.workunit.client.0.smithi090.stdout:9/900: mkdir d0/dc/d24/d2d/dda/de1/d125 0 2024-04-24T08:25:16.332 INFO:tasks.workunit.client.0.smithi090.stdout:3/923: fdatasync d2/d15/d36/d6f/d92/fa9 0 2024-04-24T08:25:16.335 INFO:tasks.workunit.client.0.smithi090.stdout:8/869: symlink d2/da/d13/d60/d91/l122 0 2024-04-24T08:25:16.336 INFO:tasks.workunit.client.0.smithi090.stdout:4/853: mknod d3/d8e/c10f 0 2024-04-24T08:25:16.339 INFO:tasks.workunit.client.0.smithi090.stdout:9/901: unlink d0/dc/d24/d7c/d4d/d32/d9f/d78/l99 0 2024-04-24T08:25:16.340 INFO:tasks.workunit.client.0.smithi090.stdout:3/924: fdatasync d2/f14 0 2024-04-24T08:25:16.341 INFO:tasks.workunit.client.0.smithi090.stdout:2/962: rename d0/d7/d6f to d0/d7/d67/d7f/d31/d146 0 2024-04-24T08:25:16.347 INFO:tasks.workunit.client.0.smithi090.stdout:9/902: rmdir d0/dc/d24/d2d/d49/dfa 39 2024-04-24T08:25:16.348 INFO:tasks.workunit.client.0.smithi090.stdout:9/903: dread - d0/dc/d24/d7c/f11c zero size 2024-04-24T08:25:16.353 INFO:tasks.workunit.client.0.smithi090.stdout:9/904: truncate d0/dc/d24/d7c/d4d/d32/d9f/d72/de9/f107 589747 0 2024-04-24T08:25:16.404 INFO:tasks.workunit.client.0.smithi090.stdout:4/854: rename d3/d8e/ddf/f70 to d3/d8e/ddf/f110 0 2024-04-24T08:25:16.407 INFO:tasks.workunit.client.0.smithi090.stdout:8/870: getdents d2/da/d13/d64/db2/dbb 0 2024-04-24T08:25:16.408 INFO:tasks.workunit.client.0.smithi090.stdout:2/963: write d0/d7/d67/d7f/fd7 [730653,122607] 0 2024-04-24T08:25:16.422 INFO:tasks.workunit.client.0.smithi090.stdout:6/958: dwrite d8/d1d/d1f/d3e/d44/d90/da2/fa4 [0,4194304] 0 2024-04-24T08:25:16.424 INFO:tasks.workunit.client.0.smithi090.stdout:2/964: chown d0/d7/d28/daa/def/f8d 196 1 2024-04-24T08:25:16.430 INFO:tasks.workunit.client.0.smithi090.stdout:9/905: dwrite d0/d19/f25 [4194304,4194304] 0 2024-04-24T08:25:16.446 INFO:tasks.workunit.client.0.smithi090.stdout:4/855: dwrite d3/d8e/ddf/d5c/f7f [0,4194304] 0 2024-04-24T08:25:16.456 INFO:tasks.workunit.client.0.smithi090.stdout:8/871: creat d2/d62/d71/f123 x:0 0 0 2024-04-24T08:25:16.460 INFO:tasks.workunit.client.0.smithi090.stdout:4/856: stat d3/d8e/ddf/d5c/db5/d69/l10d 0 2024-04-24T08:25:16.464 INFO:tasks.workunit.client.0.smithi090.stdout:2/965: rename d0/l12 to d0/d7/d57/l147 0 2024-04-24T08:25:16.468 INFO:tasks.workunit.client.0.smithi090.stdout:8/872: rename d2/d62/l9a to d2/da/d13/d60/d76/l124 0 2024-04-24T08:25:16.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/966: truncate d0/d7/d67/f72 1074186 0 2024-04-24T08:25:16.471 INFO:tasks.workunit.client.0.smithi090.stdout:2/967: readlink d0/d14/d17/lb8 0 2024-04-24T08:25:16.473 INFO:tasks.workunit.client.0.smithi090.stdout:8/873: mkdir d2/da/d13/d64/d24/dad/d10c/d125 0 2024-04-24T08:25:16.473 INFO:tasks.workunit.client.0.smithi090.stdout:2/968: write d0/d7/f95 [5242814,21092] 0 2024-04-24T08:25:16.477 INFO:tasks.workunit.client.0.smithi090.stdout:8/874: chown d2/da/d13/d64/d24/db3/db4/fdd 15384 1 2024-04-24T08:25:16.481 INFO:tasks.workunit.client.0.smithi090.stdout:2/969: mkdir d0/d7/d67/d7f/d31/d90/d148 0 2024-04-24T08:25:16.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/970: rmdir d0/d7/d57/d79 39 2024-04-24T08:25:16.496 INFO:tasks.workunit.client.0.smithi090.stdout:4/857: dread d3/d8e/ddf/d5c/db5/d69/d8b/db2/f85 [0,4194304] 0 2024-04-24T08:25:16.500 INFO:tasks.workunit.client.0.smithi090.stdout:6/959: write d8/d1d/d1f/d3e/d44/d90/da2/d10c/f10d [1103442,80960] 0 2024-04-24T08:25:16.506 INFO:tasks.workunit.client.0.smithi090.stdout:0/909: sync 2024-04-24T08:25:16.506 INFO:tasks.workunit.client.0.smithi090.stdout:5/990: sync 2024-04-24T08:25:16.509 INFO:tasks.workunit.client.0.smithi090.stdout:6/960: rename d8/d1d/ddb to d8/d1d/d1f/dc2/d12a 0 2024-04-24T08:25:16.510 INFO:tasks.workunit.client.0.smithi090.stdout:9/906: dwrite d0/d19/fa0 [0,4194304] 0 2024-04-24T08:25:16.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/961: dread - d8/d1d/d1f/dc2/f124 zero size 2024-04-24T08:25:16.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/910: chown d1a/d4b/d53/d79/d69/ca8 27096 1 2024-04-24T08:25:16.542 INFO:tasks.workunit.client.0.smithi090.stdout:2/971: dwrite d0/d7/d67/d7f/fd7 [0,4194304] 0 2024-04-24T08:25:16.544 INFO:tasks.workunit.client.0.smithi090.stdout:8/875: write d2/da/d13/d64/f86 [3620601,106218] 0 2024-04-24T08:25:16.549 INFO:tasks.workunit.client.0.smithi090.stdout:0/911: dread - d1a/f121 zero size 2024-04-24T08:25:16.551 INFO:tasks.workunit.client.0.smithi090.stdout:6/962: chown d8/d1d/d1f/d3e/d10a/d19/f27 3365 1 2024-04-24T08:25:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:4/858: dwrite d3/d8e/ddf/d1d/f27 [4194304,4194304] 0 2024-04-24T08:25:16.558 INFO:tasks.workunit.client.0.smithi090.stdout:8/876: truncate d2/da/d13/d64/db2/dfe/f114 13382 0 2024-04-24T08:25:16.565 INFO:tasks.workunit.client.0.smithi090.stdout:2/972: truncate d0/d7/f32 4429481 0 2024-04-24T08:25:16.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/859: link d3/d8e/ddf/d5c/db5/d69/d8b/db2/f101 d3/d105/f111 0 2024-04-24T08:25:16.572 INFO:tasks.workunit.client.0.smithi090.stdout:9/907: link d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/c89 d0/dc/d24/d2d/d49/d62/dcb/dec/c126 0 2024-04-24T08:25:16.573 INFO:tasks.workunit.client.0.smithi090.stdout:2/973: mkdir d0/d7/d28/daa/def/d50/d149 0 2024-04-24T08:25:16.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/877: getdents d2/d62/d83 0 2024-04-24T08:25:16.577 INFO:tasks.workunit.client.0.smithi090.stdout:8/878: stat d2/da/f35 0 2024-04-24T08:25:16.583 INFO:tasks.workunit.client.0.smithi090.stdout:2/974: rmdir d0/d5e/de2 39 2024-04-24T08:25:16.584 INFO:tasks.workunit.client.0.smithi090.stdout:0/912: write d1a/d4b/d53/d79/d69/d78/d9b/fb0 [712920,109495] 0 2024-04-24T08:25:16.591 INFO:tasks.workunit.client.0.smithi090.stdout:0/913: read ff [6500309,533] 0 2024-04-24T08:25:16.596 INFO:tasks.workunit.client.0.smithi090.stdout:2/975: truncate d0/d14/d17/f62 4083903 0 2024-04-24T08:25:16.600 INFO:tasks.workunit.client.0.smithi090.stdout:4/860: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/f42 [4194304,4194304] 0 2024-04-24T08:25:16.605 INFO:tasks.workunit.client.0.smithi090.stdout:2/976: creat d0/d7/d57/f14a x:0 0 0 2024-04-24T08:25:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:6/963: dread d8/d1d/d1f/d3e/d44/f87 [0,4194304] 0 2024-04-24T08:25:16.608 INFO:tasks.workunit.client.0.smithi090.stdout:3/925: sync 2024-04-24T08:25:16.609 INFO:tasks.workunit.client.0.smithi090.stdout:1/904: sync 2024-04-24T08:25:16.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/879: dwrite d2/da/d13/ff0 [0,4194304] 0 2024-04-24T08:25:16.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/861: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/cc5 541 1 2024-04-24T08:25:16.611 INFO:tasks.workunit.client.0.smithi090.stdout:3/926: chown d2/df/db5/deb 230649 1 2024-04-24T08:25:16.612 INFO:tasks.workunit.client.0.smithi090.stdout:9/908: dread d0/dc/f29 [0,4194304] 0 2024-04-24T08:25:16.615 INFO:tasks.workunit.client.0.smithi090.stdout:6/964: mknod d8/d1d/d29/d102/c12b 0 2024-04-24T08:25:16.617 INFO:tasks.workunit.client.0.smithi090.stdout:1/905: creat d3/d5/d58/d11f/d102/f13d x:0 0 0 2024-04-24T08:25:16.619 INFO:tasks.workunit.client.0.smithi090.stdout:2/977: truncate d0/d7/fd 2023515 0 2024-04-24T08:25:16.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/862: dread - d3/d8e/ddf/d5c/db5/f84 zero size 2024-04-24T08:25:16.621 INFO:tasks.workunit.client.0.smithi090.stdout:3/927: write d2/df/d57/d4f/f86 [4247514,21381] 0 2024-04-24T08:25:16.623 INFO:tasks.workunit.client.0.smithi090.stdout:2/978: write d0/d2e/f119 [465193,107912] 0 2024-04-24T08:25:16.627 INFO:tasks.workunit.client.0.smithi090.stdout:8/880: creat d2/da/db0/d111/f126 x:0 0 0 2024-04-24T08:25:16.628 INFO:tasks.workunit.client.0.smithi090.stdout:6/965: creat d8/d1d/d1f/d3e/d44/d90/d106/d81/f12c x:0 0 0 2024-04-24T08:25:16.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/863: symlink d3/d87/l112 0 2024-04-24T08:25:16.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/909: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 [0,4194304] 0 2024-04-24T08:25:16.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/864: fdatasync d3/d24/f2b 0 2024-04-24T08:25:16.634 INFO:tasks.workunit.client.0.smithi090.stdout:1/906: mkdir d3/d68/df2/d129/d13e 0 2024-04-24T08:25:16.637 INFO:tasks.workunit.client.0.smithi090.stdout:1/907: dread - d3/d5/da/d97/d9c/dc4/dcc/f124 zero size 2024-04-24T08:25:16.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/908: fdatasync d3/d5/da/d97/d9c/fc0 0 2024-04-24T08:25:16.641 INFO:tasks.workunit.client.0.smithi090.stdout:1/909: stat d3/d5/d3e 0 2024-04-24T08:25:16.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/966: rename d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/fe6 to d8/d1d/d1f/d3e/d44/d90/f12d 0 2024-04-24T08:25:16.644 INFO:tasks.workunit.client.0.smithi090.stdout:6/967: stat d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/fbd 0 2024-04-24T08:25:16.652 INFO:tasks.workunit.client.0.smithi090.stdout:9/910: dwrite d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/fb2 [0,4194304] 0 2024-04-24T08:25:16.652 INFO:tasks.workunit.client.0.smithi090.stdout:6/968: mknod d8/d1d/d1f/d3e/d44/d90/d106/d38/c12e 0 2024-04-24T08:25:16.657 INFO:tasks.workunit.client.0.smithi090.stdout:6/969: write d8/d1d/d1f/d3e/fd6 [4098044,27254] 0 2024-04-24T08:25:16.658 INFO:tasks.workunit.client.0.smithi090.stdout:9/911: dread - d0/dc/d24/d2d/d49/d62/dcb/dcd/ffb zero size 2024-04-24T08:25:16.662 INFO:tasks.workunit.client.0.smithi090.stdout:9/912: fsync d0/dc/d24/d2d/d49/fed 0 2024-04-24T08:25:16.666 INFO:tasks.workunit.client.0.smithi090.stdout:4/865: truncate d3/d8e/ddf/d5c/db5/d69/d8b/db2/f60 6160970 0 2024-04-24T08:25:16.667 INFO:tasks.workunit.client.0.smithi090.stdout:2/979: dwrite d0/d7/d28/d124/fe4 [4194304,4194304] 0 2024-04-24T08:25:16.672 INFO:tasks.workunit.client.0.smithi090.stdout:9/913: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/fc4 [3612027,55598] 0 2024-04-24T08:25:16.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/928: dread d2/df/d1a/d37/d53/f74 [0,4194304] 0 2024-04-24T08:25:16.679 INFO:tasks.workunit.client.0.smithi090.stdout:8/881: dwrite d2/da/d13/f5c [0,4194304] 0 2024-04-24T08:25:16.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/910: dwrite d3/d5/d3e/f18 [0,4194304] 0 2024-04-24T08:25:16.684 INFO:tasks.workunit.client.0.smithi090.stdout:4/866: creat d3/d87/f113 x:0 0 0 2024-04-24T08:25:16.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/970: link d8/d1d/d29/cc6 d8/d1d/d1f/d3e/d7a/c12f 0 2024-04-24T08:25:16.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/980: dread d0/d7/d28/daa/def/d50/d96/fcf [0,4194304] 0 2024-04-24T08:25:16.695 INFO:tasks.workunit.client.0.smithi090.stdout:5/991: sync 2024-04-24T08:25:16.695 INFO:tasks.workunit.client.0.smithi090.stdout:0/914: sync 2024-04-24T08:25:16.700 INFO:tasks.workunit.client.0.smithi090.stdout:1/911: creat d3/d5/da/dc/f13f x:0 0 0 2024-04-24T08:25:16.700 INFO:tasks.workunit.client.0.smithi090.stdout:5/992: write d8/d59/dc7/d7a/da1/dae/f139 [477846,90763] 0 2024-04-24T08:25:16.702 INFO:tasks.workunit.client.0.smithi090.stdout:6/971: mkdir d8/d1d/d1f/d3e/d4f/d7e/daa/d130 0 2024-04-24T08:25:16.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/882: dwrite d2/da/d13/d64/d24/db3/db4/dcd/de8/ffd [0,4194304] 0 2024-04-24T08:25:16.704 INFO:tasks.workunit.client.0.smithi090.stdout:0/915: dread d1a/d4b/d53/d79/d58/de7/f100 [0,4194304] 0 2024-04-24T08:25:16.708 INFO:tasks.workunit.client.0.smithi090.stdout:3/929: link d2/df/d1a/d37/d4b/fe5 d2/d15/d36/f132 0 2024-04-24T08:25:16.712 INFO:tasks.workunit.client.0.smithi090.stdout:2/981: fsync d0/d7/d57/d79/fd6 0 2024-04-24T08:25:16.713 INFO:tasks.workunit.client.0.smithi090.stdout:1/912: truncate d3/d5/da/f64 2860915 0 2024-04-24T08:25:16.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/883: creat d2/da/d13/d64/db2/dbb/f127 x:0 0 0 2024-04-24T08:25:16.714 INFO:tasks.workunit.client.0.smithi090.stdout:9/914: write d0/dc/d24/d2d/f80 [333558,101426] 0 2024-04-24T08:25:16.718 INFO:tasks.workunit.client.0.smithi090.stdout:4/867: write d3/d8e/ddf/d1d/fa1 [2203495,104274] 0 2024-04-24T08:25:16.724 INFO:tasks.workunit.client.0.smithi090.stdout:4/868: dread d3/d8e/ddf/d5c/db5/d69/fe7 [0,4194304] 0 2024-04-24T08:25:16.726 INFO:tasks.workunit.client.0.smithi090.stdout:9/915: read d0/dc/d24/d7c/f86 [129354,23195] 0 2024-04-24T08:25:16.727 INFO:tasks.workunit.client.0.smithi090.stdout:2/982: rename d0/d13c to d0/d14/d17/dc2/dcb/d14b 0 2024-04-24T08:25:16.729 INFO:tasks.workunit.client.0.smithi090.stdout:2/983: chown d0/d7/d57/dea/fbe 18336 1 2024-04-24T08:25:16.730 INFO:tasks.workunit.client.0.smithi090.stdout:1/913: dwrite d3/d1d/ff1 [0,4194304] 0 2024-04-24T08:25:16.737 INFO:tasks.workunit.client.0.smithi090.stdout:3/930: readlink d2/d15/d36/d6f/d9d/l9f 0 2024-04-24T08:25:16.745 INFO:tasks.workunit.client.0.smithi090.stdout:6/972: truncate d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/fbd 533856 0 2024-04-24T08:25:16.746 INFO:tasks.workunit.client.0.smithi090.stdout:9/916: fdatasync d0/dc/d24/d7c/f82 0 2024-04-24T08:25:16.747 INFO:tasks.workunit.client.0.smithi090.stdout:2/984: mknod d0/d7/d28/daa/db1/c14c 0 2024-04-24T08:25:16.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/914: mkdir d3/d5/da/d97/d9c/dc4/dcc/d140 0 2024-04-24T08:25:16.750 INFO:tasks.workunit.client.0.smithi090.stdout:8/884: creat d2/da/f128 x:0 0 0 2024-04-24T08:25:16.751 INFO:tasks.workunit.client.0.smithi090.stdout:3/931: mkdir d2/df/d5b/d105/d133 0 2024-04-24T08:25:16.756 INFO:tasks.workunit.client.0.smithi090.stdout:4/869: getdents d3/d8e/ddf/d109 0 2024-04-24T08:25:16.760 INFO:tasks.workunit.client.0.smithi090.stdout:8/885: mkdir d2/da/d119/d129 0 2024-04-24T08:25:16.763 INFO:tasks.workunit.client.0.smithi090.stdout:2/985: getdents d0/d7 0 2024-04-24T08:25:16.765 INFO:tasks.workunit.client.0.smithi090.stdout:2/986: fsync d0/d7/de1/ff8 0 2024-04-24T08:25:16.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/973: dwrite d8/d1d/d1f/d3e/d44/d90/da2/d10c/f117 [0,4194304] 0 2024-04-24T08:25:16.767 INFO:tasks.workunit.client.0.smithi090.stdout:1/915: dwrite d3/d5/da/d97/d9c/dc4/ffb [0,4194304] 0 2024-04-24T08:25:16.768 INFO:tasks.workunit.client.0.smithi090.stdout:2/987: write d0/d7/d28/d124/dcd/f121 [170132,31975] 0 2024-04-24T08:25:16.781 INFO:tasks.workunit.client.0.smithi090.stdout:3/932: link d2/df/d5b/ld5 d2/d15/d36/d6f/d9d/ded/l134 0 2024-04-24T08:25:16.784 INFO:tasks.workunit.client.0.smithi090.stdout:9/917: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 [0,4194304] 0 2024-04-24T08:25:16.787 INFO:tasks.workunit.client.0.smithi090.stdout:3/933: chown d2/df/d1a/d37/d59/d98/cef 586290429 1 2024-04-24T08:25:16.787 INFO:tasks.workunit.client.0.smithi090.stdout:3/934: truncate d2/df/f10d 1984464 0 2024-04-24T08:25:16.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/935: chown d2/d15/dc4 3147 1 2024-04-24T08:25:16.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/916: mkdir d3/d1d/d34/d5a/d141 0 2024-04-24T08:25:16.794 INFO:tasks.workunit.client.0.smithi090.stdout:8/886: symlink d2/d62/l12a 0 2024-04-24T08:25:16.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/870: mkdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/d114 0 2024-04-24T08:25:16.801 INFO:tasks.workunit.client.0.smithi090.stdout:3/936: mkdir d2/d90/d135 0 2024-04-24T08:25:16.806 INFO:tasks.workunit.client.0.smithi090.stdout:6/974: creat d8/d1d/d29/f131 x:0 0 0 2024-04-24T08:25:16.808 INFO:tasks.workunit.client.0.smithi090.stdout:6/975: fsync d8/d1d/d1f/d3e/d44/d90/da2/fdd 0 2024-04-24T08:25:16.809 INFO:tasks.workunit.client.0.smithi090.stdout:1/917: rename d3/d5/d3e to d3/d5/da/d97/d9c/dc4/dcc/d140/d142 0 2024-04-24T08:25:16.812 INFO:tasks.workunit.client.0.smithi090.stdout:8/887: creat d2/da/d13/d64/d24/db3/db4/dcd/de8/f12b x:0 0 0 2024-04-24T08:25:16.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/988: link d0/d7/d57/d79/fd6 d0/d7/d67/d7f/d31/ddb/f14d 0 2024-04-24T08:25:16.816 INFO:tasks.workunit.client.0.smithi090.stdout:3/937: mkdir d2/df/d1a/d37/d59/d87/db0/d136 0 2024-04-24T08:25:16.817 INFO:tasks.workunit.client.0.smithi090.stdout:3/938: fdatasync d2/df/d1a/f2b 0 2024-04-24T08:25:16.820 INFO:tasks.workunit.client.0.smithi090.stdout:6/976: mknod d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/c132 0 2024-04-24T08:25:16.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/918: dwrite d0/dc/d24/d7c/d4d/d32/d9f/f9b [0,4194304] 0 2024-04-24T08:25:16.829 INFO:tasks.workunit.client.0.smithi090.stdout:8/888: dwrite d2/f9d [0,4194304] 0 2024-04-24T08:25:16.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/918: rename d3/d5/da/d97/d9c/dc4/dcc to d3/d1d/d34/d5a/d143 0 2024-04-24T08:25:16.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/939: creat d2/df/d57/d4f/f137 x:0 0 0 2024-04-24T08:25:16.831 INFO:tasks.workunit.client.0.smithi090.stdout:8/889: chown d2/da/d13/d64 2367 1 2024-04-24T08:25:16.835 INFO:tasks.workunit.client.0.smithi090.stdout:6/977: creat d8/d1d/d2f/f133 x:0 0 0 2024-04-24T08:25:16.839 INFO:tasks.workunit.client.0.smithi090.stdout:2/989: link d0/d7/d28/daa/def/d50/lb2 d0/d14/d17/d11f/l14e 0 2024-04-24T08:25:16.842 INFO:tasks.workunit.client.0.smithi090.stdout:2/990: fdatasync d0/d14/f15 0 2024-04-24T08:25:16.843 INFO:tasks.workunit.client.0.smithi090.stdout:8/890: creat d2/da/d13/d60/d91/d117/f12c x:0 0 0 2024-04-24T08:25:16.845 INFO:tasks.workunit.client.0.smithi090.stdout:2/991: readlink d0/d7/d57/d79/le7 0 2024-04-24T08:25:16.846 INFO:tasks.workunit.client.0.smithi090.stdout:8/891: read d2/d62/d6b/f90 [1847971,14415] 0 2024-04-24T08:25:16.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/978: mknod d8/d1d/d1f/d3e/d44/d90/d106/c134 0 2024-04-24T08:25:16.849 INFO:tasks.workunit.client.0.smithi090.stdout:1/919: rename d3/d1d/ff1 to d3/d5/da/d97/f144 0 2024-04-24T08:25:16.852 INFO:tasks.workunit.client.0.smithi090.stdout:3/940: getdents d2/df/d1a/d37/d53 0 2024-04-24T08:25:16.852 INFO:tasks.workunit.client.0.smithi090.stdout:2/992: symlink d0/d14/d17/l14f 0 2024-04-24T08:25:16.853 INFO:tasks.workunit.client.0.smithi090.stdout:8/892: mknod d2/da/d13/d60/d91/c12d 0 2024-04-24T08:25:16.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/941: fdatasync d2/df/d57/d4f/f86 0 2024-04-24T08:25:16.855 INFO:tasks.workunit.client.0.smithi090.stdout:9/919: dwrite d0/dc/f29 [0,4194304] 0 2024-04-24T08:25:16.860 INFO:tasks.workunit.client.0.smithi090.stdout:2/993: creat d0/d14/d17/dc2/f150 x:0 0 0 2024-04-24T08:25:16.864 INFO:tasks.workunit.client.0.smithi090.stdout:9/920: write d0/f5 [6450109,107381] 0 2024-04-24T08:25:16.865 INFO:tasks.workunit.client.0.smithi090.stdout:8/893: mkdir d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e 0 2024-04-24T08:25:16.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/920: rename d3/d5/da/c2d to d3/d1d/d34/d5a/c145 0 2024-04-24T08:25:16.870 INFO:tasks.workunit.client.0.smithi090.stdout:2/994: unlink d0/d7/d67/d7f/d31/d46/c92 0 2024-04-24T08:25:16.871 INFO:tasks.workunit.client.0.smithi090.stdout:9/921: creat d0/d19/f127 x:0 0 0 2024-04-24T08:25:16.873 INFO:tasks.workunit.client.0.smithi090.stdout:8/894: chown d2/da/d13/d64/d24/dad/f115 27 1 2024-04-24T08:25:16.874 INFO:tasks.workunit.client.0.smithi090.stdout:1/921: mkdir d3/d1d/d54/d146 0 2024-04-24T08:25:16.876 INFO:tasks.workunit.client.0.smithi090.stdout:2/995: dread - d0/d7/d28/d124/dcd/fd1 zero size 2024-04-24T08:25:16.876 INFO:tasks.workunit.client.0.smithi090.stdout:3/942: write d2/df/d1a/f6d [1095573,126993] 0 2024-04-24T08:25:16.879 INFO:tasks.workunit.client.0.smithi090.stdout:3/943: dread - d2/d15/d36/d6f/d92/fae zero size 2024-04-24T08:25:16.880 INFO:tasks.workunit.client.0.smithi090.stdout:1/922: symlink d3/d5/da/d94/d10b/l147 0 2024-04-24T08:25:16.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/923: chown d3/d68/def 3 1 2024-04-24T08:25:16.882 INFO:tasks.workunit.client.0.smithi090.stdout:1/924: readlink d3/d68/def/lf3 0 2024-04-24T08:25:16.883 INFO:tasks.workunit.client.0.smithi090.stdout:8/895: creat d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e/f12f x:0 0 0 2024-04-24T08:25:16.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/996: truncate d0/d7/d28/fbf 1276437 0 2024-04-24T08:25:16.888 INFO:tasks.workunit.client.0.smithi090.stdout:9/922: rename d0/d19/fa0 to d0/dc/d24/d2d/d49/d62/f128 0 2024-04-24T08:25:16.892 INFO:tasks.workunit.client.0.smithi090.stdout:8/896: symlink d2/d97/l130 0 2024-04-24T08:25:16.893 INFO:tasks.workunit.client.0.smithi090.stdout:3/944: dwrite d2/df/f19 [4194304,4194304] 0 2024-04-24T08:25:16.895 INFO:tasks.workunit.client.0.smithi090.stdout:2/997: rename d0/d7/d28/d124/dcd to d0/d7/d28/daa/def/d50/d96/d151 0 2024-04-24T08:25:16.897 INFO:tasks.workunit.client.0.smithi090.stdout:8/897: unlink d2/da/db0/ccb 0 2024-04-24T08:25:16.899 INFO:tasks.workunit.client.0.smithi090.stdout:2/998: readlink d0/d2e/l44 0 2024-04-24T08:25:16.899 INFO:tasks.workunit.client.0.smithi090.stdout:8/898: fsync d2/d62/d83/f100 0 2024-04-24T08:25:16.902 INFO:tasks.workunit.client.0.smithi090.stdout:3/945: rename d2/df/d1a/d37/d59 to d2/d120/d138 0 2024-04-24T08:25:16.911 INFO:tasks.workunit.client.0.smithi090.stdout:3/946: symlink d2/df/d1a/d37/d53/d60/l139 0 2024-04-24T08:25:16.911 INFO:tasks.workunit.client.0.smithi090.stdout:9/923: dread d0/d2a/f8e [0,4194304] 0 2024-04-24T08:25:16.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/947: fsync d2/df/d1a/f2b 0 2024-04-24T08:25:16.915 INFO:tasks.workunit.client.0.smithi090.stdout:8/899: creat d2/d62/f131 x:0 0 0 2024-04-24T08:25:16.916 INFO:tasks.workunit.client.0.smithi090.stdout:9/924: creat d0/d19/f129 x:0 0 0 2024-04-24T08:25:16.917 INFO:tasks.workunit.client.0.smithi090.stdout:1/925: dwrite d3/d5/fa5 [0,4194304] 0 2024-04-24T08:25:16.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/999: rename d0/d7/d28/d124 to d0/d7/d57/d152 0 2024-04-24T08:25:16.920 INFO:tasks.workunit.client.0.smithi090.stdout:8/900: stat d2/f69 0 2024-04-24T08:25:16.925 INFO:tasks.workunit.client.0.smithi090.stdout:9/925: unlink d0/f5 0 2024-04-24T08:25:16.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/926: write d0/dc/d24/d2d/d49/d62/fad [2003016,83778] 0 2024-04-24T08:25:16.936 INFO:tasks.workunit.client.0.smithi090.stdout:0/916: sync 2024-04-24T08:25:16.936 INFO:tasks.workunit.client.0.smithi090.stdout:5/993: sync 2024-04-24T08:25:16.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/901: mkdir d2/da/d13/d64/d24/dad/d10c/d125/d132 0 2024-04-24T08:25:16.939 INFO:tasks.workunit.client.0.smithi090.stdout:0/917: chown d1a/d4b/d53/d79/d58/de7 4 1 2024-04-24T08:25:16.939 INFO:tasks.workunit.client.0.smithi090.stdout:3/948: write d2/d25/d33/f43 [698815,63960] 0 2024-04-24T08:25:16.940 INFO:tasks.workunit.client.0.smithi090.stdout:8/902: chown d2/da/d13/d64/cc8 810286040 1 2024-04-24T08:25:16.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/903: chown d2/da/l39 794129442 1 2024-04-24T08:25:16.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/994: creat d8/d76/d119/f142 x:0 0 0 2024-04-24T08:25:16.944 INFO:tasks.workunit.client.0.smithi090.stdout:3/949: chown d2/d15/d36/d6f/d92/l97 51884 1 2024-04-24T08:25:16.945 INFO:tasks.workunit.client.0.smithi090.stdout:0/918: symlink d1a/d4b/l136 0 2024-04-24T08:25:16.946 INFO:tasks.workunit.client.0.smithi090.stdout:9/927: dread d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 [0,4194304] 0 2024-04-24T08:25:16.951 INFO:tasks.workunit.client.0.smithi090.stdout:1/926: dread d3/d5/da/d94/da6/d115/d30/d9d/da2/fde [0,4194304] 0 2024-04-24T08:25:16.953 INFO:tasks.workunit.client.0.smithi090.stdout:9/928: rmdir d0 39 2024-04-24T08:25:16.953 INFO:tasks.workunit.client.0.smithi090.stdout:1/927: read - d3/d1d/d34/d5a/f8b zero size 2024-04-24T08:25:16.957 INFO:tasks.workunit.client.0.smithi090.stdout:3/950: creat d2/df/d1a/d37/dc0/de6/dfa/f13a x:0 0 0 2024-04-24T08:25:16.960 INFO:tasks.workunit.client.0.smithi090.stdout:9/929: chown d0/dc/d24/d7c/f11d 2 1 2024-04-24T08:25:16.961 INFO:tasks.workunit.client.0.smithi090.stdout:0/919: dread d1a/d4b/d53/d79/fa9 [0,4194304] 0 2024-04-24T08:25:16.967 INFO:tasks.workunit.client.0.smithi090.stdout:0/920: fsync d1a/d4b/d53/d79/d35/f42 0 2024-04-24T08:25:16.967 INFO:tasks.workunit.client.0.smithi090.stdout:8/904: dwrite d2/da/d13/d64/d50/f5d [0,4194304] 0 2024-04-24T08:25:16.969 INFO:tasks.workunit.client.0.smithi090.stdout:3/951: dread d2/d84/fa3 [0,4194304] 0 2024-04-24T08:25:16.970 INFO:tasks.workunit.client.0.smithi090.stdout:9/930: creat d0/dc/d24/d7c/d4d/d8c/de2/f12a x:0 0 0 2024-04-24T08:25:16.971 INFO:tasks.workunit.client.0.smithi090.stdout:0/921: unlink d1a/d4b/d53/lf6 0 2024-04-24T08:25:16.975 INFO:tasks.workunit.client.0.smithi090.stdout:3/952: fdatasync d2/df/d57/d114/f123 0 2024-04-24T08:25:16.977 INFO:tasks.workunit.client.0.smithi090.stdout:9/931: creat d0/dc/d24/d7c/d4d/d32/d9f/d78/f12b x:0 0 0 2024-04-24T08:25:16.977 INFO:tasks.workunit.client.0.smithi090.stdout:5/995: dread d8/d7c/da9/d118/fb8 [0,4194304] 0 2024-04-24T08:25:16.979 INFO:tasks.workunit.client.0.smithi090.stdout:3/953: write d2/d120/d138/d98/dfd/f130 [597547,51980] 0 2024-04-24T08:25:16.981 INFO:tasks.workunit.client.0.smithi090.stdout:9/932: stat d0/dc/d24/d2d/d49 0 2024-04-24T08:25:16.982 INFO:tasks.workunit.client.0.smithi090.stdout:1/928: dwrite d3/d5/da/d97/d9c/f130 [0,4194304] 0 2024-04-24T08:25:16.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/954: fsync d2/df/d1a/d37/d4b/f112 0 2024-04-24T08:25:16.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/996: mkdir d8/da/d14/d2f/d44/d143 0 2024-04-24T08:25:16.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/929: read - d3/d1d/d54/f74 zero size 2024-04-24T08:25:16.990 INFO:tasks.workunit.client.0.smithi090.stdout:0/922: rename d1a/d4b/d53/d79/d35/db8/c111 to d1a/d4b/d53/d79/d35/dea/dfe/c137 0 2024-04-24T08:25:16.997 INFO:tasks.workunit.client.0.smithi090.stdout:8/905: write d2/da/d13/f28 [3337424,7159] 0 2024-04-24T08:25:17.002 INFO:tasks.workunit.client.0.smithi090.stdout:3/955: rename d2/df/d1a/d37/dc0/f10c to d2/df/d57/f13b 0 2024-04-24T08:25:17.002 INFO:tasks.workunit.client.0.smithi090.stdout:9/933: dwrite d0/dc/d24/d7c/d4d/d8c/fdd [0,4194304] 0 2024-04-24T08:25:17.002 INFO:tasks.workunit.client.0.smithi090.stdout:0/923: rmdir d1a/d4b/d53/d79/d69 39 2024-04-24T08:25:17.005 INFO:tasks.workunit.client.0.smithi090.stdout:0/924: readlink d1a/d4b/d53/d79/d35/dea/dfe/l72 0 2024-04-24T08:25:17.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/930: creat d3/d1d/f148 x:0 0 0 2024-04-24T08:25:17.011 INFO:tasks.workunit.client.0.smithi090.stdout:4/871: sync 2024-04-24T08:25:17.017 INFO:tasks.workunit.client.0.smithi090.stdout:3/956: getdents d2/d15/d36/d6f/d9d/ded 0 2024-04-24T08:25:17.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/925: mkdir d1a/d4b/d53/d79/d69/d78/d138 0 2024-04-24T08:25:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:8/906: dwrite d2/da/d13/d60/d91/f121 [0,4194304] 0 2024-04-24T08:25:17.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/997: dwrite d8/d7c/da9/d118/f98 [0,4194304] 0 2024-04-24T08:25:17.024 INFO:tasks.workunit.client.0.smithi090.stdout:5/998: write d8/f31 [4851139,124926] 0 2024-04-24T08:25:17.025 INFO:tasks.workunit.client.0.smithi090.stdout:1/931: rename d3/d5/da/d97/fd7 to d3/d5/da/dc/d80/deb/d111/d12c/f149 0 2024-04-24T08:25:17.030 INFO:tasks.workunit.client.0.smithi090.stdout:1/932: write d3/d5/f8d [2607324,127629] 0 2024-04-24T08:25:17.031 INFO:tasks.workunit.client.0.smithi090.stdout:8/907: mknod d2/d97/def/c133 0 2024-04-24T08:25:17.037 INFO:tasks.workunit.client.0.smithi090.stdout:1/933: readlink d3/d5/da/d94/da6/d115/l5f 0 2024-04-24T08:25:17.041 INFO:tasks.workunit.client.0.smithi090.stdout:4/872: dwrite d3/d8e/ddf/fd6 [0,4194304] 0 2024-04-24T08:25:17.043 INFO:tasks.workunit.client.0.smithi090.stdout:9/934: link d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/c100 d0/dc/d24/d2d/d49/d81/d87/c12c 0 2024-04-24T08:25:17.045 INFO:tasks.workunit.client.0.smithi090.stdout:8/908: readlink d2/da/d13/d64/db2/ldb 0 2024-04-24T08:25:17.047 INFO:tasks.workunit.client.0.smithi090.stdout:1/934: creat d3/d68/df2/f14a x:0 0 0 2024-04-24T08:25:17.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/999: creat d8/da/d14/d2f/dc2/f144 x:0 0 0 2024-04-24T08:25:17.049 INFO:tasks.workunit.client.0.smithi090.stdout:3/957: dwrite d2/df/d57/d4f/f137 [0,4194304] 0 2024-04-24T08:25:17.050 INFO:tasks.workunit.client.0.smithi090.stdout:0/926: rename d1a/l37 to d1a/d4b/d53/d79/d35/dea/dfe/l139 0 2024-04-24T08:25:17.051 INFO:tasks.workunit.client.0.smithi090.stdout:4/873: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/f8d [1226575,82042] 0 2024-04-24T08:25:17.057 INFO:tasks.workunit.client.0.smithi090.stdout:8/909: mkdir d2/da/d13/d64/d134 0 2024-04-24T08:25:17.065 INFO:tasks.workunit.client.0.smithi090.stdout:3/958: rename d2/d120/d138/d87/l91 to d2/d25/d124/l13c 0 2024-04-24T08:25:17.067 INFO:tasks.workunit.client.0.smithi090.stdout:8/910: mkdir d2/da/d13/d64/db2/dfe/d135 0 2024-04-24T08:25:17.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/911: dread - d2/da/d13/d64/f8f zero size 2024-04-24T08:25:17.073 INFO:tasks.workunit.client.0.smithi090.stdout:3/959: symlink d2/df/l13d 0 2024-04-24T08:25:17.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/912: fsync d2/da/d13/d64/d50/f10b 0 2024-04-24T08:25:17.080 INFO:tasks.workunit.client.0.smithi090.stdout:3/960: mkdir d2/d120/d138/d87/db0/d136/d13e 0 2024-04-24T08:25:17.085 INFO:tasks.workunit.client.0.smithi090.stdout:9/935: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/f118 [0,4194304] 0 2024-04-24T08:25:17.091 INFO:tasks.workunit.client.0.smithi090.stdout:8/913: rename d2/da/l31 to d2/da/d13/l136 0 2024-04-24T08:25:17.092 INFO:tasks.workunit.client.0.smithi090.stdout:3/961: read d2/d15/d36/d6f/d92/fa9 [2517295,18828] 0 2024-04-24T08:25:17.096 INFO:tasks.workunit.client.0.smithi090.stdout:1/935: dwrite d3/d5/da/d97/d9c/dc4/fd2 [0,4194304] 0 2024-04-24T08:25:17.100 INFO:tasks.workunit.client.0.smithi090.stdout:9/936: mkdir d0/dc/d24/d2d/d49/d62/d12d 0 2024-04-24T08:25:17.102 INFO:tasks.workunit.client.0.smithi090.stdout:8/914: mkdir d2/da/db0/d111/d137 0 2024-04-24T08:25:17.102 INFO:tasks.workunit.client.0.smithi090.stdout:3/962: creat d2/d120/d138/d87/f13f x:0 0 0 2024-04-24T08:25:17.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/927: dread d1a/d4b/f10f [0,4194304] 0 2024-04-24T08:25:17.109 INFO:tasks.workunit.client.0.smithi090.stdout:8/915: mknod d2/da/d13/d60/d76/c138 0 2024-04-24T08:25:17.111 INFO:tasks.workunit.client.0.smithi090.stdout:3/963: readlink d2/d15/d36/d6f/d9d/ded/l134 0 2024-04-24T08:25:17.112 INFO:tasks.workunit.client.0.smithi090.stdout:0/928: mknod d1a/d4b/d53/d79/d69/d78/c13a 0 2024-04-24T08:25:17.113 INFO:tasks.workunit.client.0.smithi090.stdout:3/964: chown d2/df/d57/d4f/fe8 2128784329 1 2024-04-24T08:25:17.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/916: creat d2/d62/d83/da3/deb/f139 x:0 0 0 2024-04-24T08:25:17.115 INFO:tasks.workunit.client.0.smithi090.stdout:1/936: creat d3/d5/da/d94/f14b x:0 0 0 2024-04-24T08:25:17.118 INFO:tasks.workunit.client.0.smithi090.stdout:1/937: read d3/d68/d7f/fda [7643208,26424] 0 2024-04-24T08:25:17.119 INFO:tasks.workunit.client.0.smithi090.stdout:8/917: symlink d2/d62/d83/l13a 0 2024-04-24T08:25:17.121 INFO:tasks.workunit.client.0.smithi090.stdout:1/938: dread - d3/d1d/d34/f12b zero size 2024-04-24T08:25:17.121 INFO:tasks.workunit.client.0.smithi090.stdout:3/965: creat d2/d120/d138/d98/dfd/f140 x:0 0 0 2024-04-24T08:25:17.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/937: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f5c [1684319,35211] 0 2024-04-24T08:25:17.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/929: creat d1a/d4b/d53/d79/f13b x:0 0 0 2024-04-24T08:25:17.126 INFO:tasks.workunit.client.0.smithi090.stdout:0/930: chown d1a/d4b/d53/d79/lfb 22999 1 2024-04-24T08:25:17.127 INFO:tasks.workunit.client.0.smithi090.stdout:3/966: rmdir d2/d15/d36/d6f/d7a 39 2024-04-24T08:25:17.129 INFO:tasks.workunit.client.0.smithi090.stdout:8/918: rename d2/da/d13/d34/f3d to d2/da/d13/d64/db2/dfe/f13b 0 2024-04-24T08:25:17.131 INFO:tasks.workunit.client.0.smithi090.stdout:9/938: fdatasync d0/dc/d24/d2d/d49/d62/f128 0 2024-04-24T08:25:17.132 INFO:tasks.workunit.client.0.smithi090.stdout:0/931: read - d1a/fd0 zero size 2024-04-24T08:25:17.133 INFO:tasks.workunit.client.0.smithi090.stdout:8/919: symlink d2/da/d13/d64/db2/d109/l13c 0 2024-04-24T08:25:17.134 INFO:tasks.workunit.client.0.smithi090.stdout:0/932: readlink d1a/d4b/d53/d79/d69/d8a/leb 0 2024-04-24T08:25:17.136 INFO:tasks.workunit.client.0.smithi090.stdout:9/939: dread - d0/dc/d24/d7c/d4d/d32/d9f/d72/f124 zero size 2024-04-24T08:25:17.142 INFO:tasks.workunit.client.0.smithi090.stdout:9/940: rmdir d0/d2a 39 2024-04-24T08:25:17.143 INFO:tasks.workunit.client.0.smithi090.stdout:6/979: sync 2024-04-24T08:25:17.145 INFO:tasks.workunit.client.0.smithi090.stdout:6/980: dread - d8/d1d/d2f/d85/fbc zero size 2024-04-24T08:25:17.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/933: mkdir d1a/d13c 0 2024-04-24T08:25:17.147 INFO:tasks.workunit.client.0.smithi090.stdout:0/934: stat f8 0 2024-04-24T08:25:17.147 INFO:tasks.workunit.client.0.smithi090.stdout:1/939: dread d3/d5/f8d [0,4194304] 0 2024-04-24T08:25:17.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/967: link d2/df/d5b/l117 d2/df/d57/d4f/l141 0 2024-04-24T08:25:17.149 INFO:tasks.workunit.client.0.smithi090.stdout:9/941: mknod d0/d19/df7/c12e 0 2024-04-24T08:25:17.150 INFO:tasks.workunit.client.0.smithi090.stdout:6/981: truncate d8/d1d/d1f/d3e/fca 3322997 0 2024-04-24T08:25:17.151 INFO:tasks.workunit.client.0.smithi090.stdout:0/935: creat d1a/d13c/f13d x:0 0 0 2024-04-24T08:25:17.153 INFO:tasks.workunit.client.0.smithi090.stdout:1/940: truncate d3/d5/da/d94/d10b/f11a 500378 0 2024-04-24T08:25:17.153 INFO:tasks.workunit.client.0.smithi090.stdout:0/936: truncate d1a/d13c/f13d 932852 0 2024-04-24T08:25:17.154 INFO:tasks.workunit.client.0.smithi090.stdout:6/982: mknod d8/d1d/d1f/d3e/d44/c135 0 2024-04-24T08:25:17.160 INFO:tasks.workunit.client.0.smithi090.stdout:8/920: dread d2/d62/d71/fc9 [0,4194304] 0 2024-04-24T08:25:17.163 INFO:tasks.workunit.client.0.smithi090.stdout:6/983: symlink d8/d1d/d1f/d3e/d44/l136 0 2024-04-24T08:25:17.167 INFO:tasks.workunit.client.0.smithi090.stdout:3/968: truncate d2/df/d1a/d37/d4b/fa8 1244113 0 2024-04-24T08:25:17.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/941: rename d3/d5/da/d94/da6/d115/l5f to d3/d5/da/dc/d80/deb/l14c 0 2024-04-24T08:25:17.171 INFO:tasks.workunit.client.0.smithi090.stdout:4/874: sync 2024-04-24T08:25:17.175 INFO:tasks.workunit.client.0.smithi090.stdout:4/875: fdatasync d3/d8e/ddf/d5c/db5/d69/d8b/db2/f8d 0 2024-04-24T08:25:17.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/942: link d0/da4/cdb d0/dc/d24/d2d/d49/dfa/c12f 0 2024-04-24T08:25:17.178 INFO:tasks.workunit.client.0.smithi090.stdout:3/969: rmdir d2/df/db5/deb 39 2024-04-24T08:25:17.178 INFO:tasks.workunit.client.0.smithi090.stdout:8/921: dread d2/da/d13/d64/d24/db3/fc2 [0,4194304] 0 2024-04-24T08:25:17.180 INFO:tasks.workunit.client.0.smithi090.stdout:6/984: dwrite d8/d1d/d1f/d3e/d44/fde [0,4194304] 0 2024-04-24T08:25:17.181 INFO:tasks.workunit.client.0.smithi090.stdout:3/970: chown d2/df/d57/d4f/fe8 651167076 1 2024-04-24T08:25:17.183 INFO:tasks.workunit.client.0.smithi090.stdout:3/971: dread - d2/df/d5b/f107 zero size 2024-04-24T08:25:17.184 INFO:tasks.workunit.client.0.smithi090.stdout:0/937: creat d1a/d4b/d53/d79/d35/dea/f13e x:0 0 0 2024-04-24T08:25:17.191 INFO:tasks.workunit.client.0.smithi090.stdout:9/943: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d96/d130 0 2024-04-24T08:25:17.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/942: symlink d3/d1d/dfe/l14d 0 2024-04-24T08:25:17.194 INFO:tasks.workunit.client.0.smithi090.stdout:8/922: symlink d2/da/d13/d64/d50/l13d 0 2024-04-24T08:25:17.195 INFO:tasks.workunit.client.0.smithi090.stdout:6/985: rmdir d8/d1d/d1f/d3e/d10a/d19/dd2 39 2024-04-24T08:25:17.198 INFO:tasks.workunit.client.0.smithi090.stdout:3/972: unlink d2/df/d1a/d37/d53/d60/cbc 0 2024-04-24T08:25:17.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/973: write d2/df/d57/d4f/f86 [804582,1170] 0 2024-04-24T08:25:17.200 INFO:tasks.workunit.client.0.smithi090.stdout:9/944: fdatasync d0/dc/d24/d7c/d4d/f52 0 2024-04-24T08:25:17.202 INFO:tasks.workunit.client.0.smithi090.stdout:8/923: dread d2/da/d13/d64/f6d [0,4194304] 0 2024-04-24T08:25:17.204 INFO:tasks.workunit.client.0.smithi090.stdout:4/876: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f4f [3471623,48135] 0 2024-04-24T08:25:17.204 INFO:tasks.workunit.client.0.smithi090.stdout:0/938: truncate d1a/f52 954414 0 2024-04-24T08:25:17.206 INFO:tasks.workunit.client.0.smithi090.stdout:0/939: fdatasync d1a/d4b/d53/d79/d69/d8a/fbe 0 2024-04-24T08:25:17.208 INFO:tasks.workunit.client.0.smithi090.stdout:6/986: symlink d8/d1d/d1f/d3e/d44/d90/d106/d38/l137 0 2024-04-24T08:25:17.213 INFO:tasks.workunit.client.0.smithi090.stdout:8/924: dread - d2/da/d13/f94 zero size 2024-04-24T08:25:17.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/943: rename d3/d5/da/d94/d10b/c139 to d3/d5/da/d97/d9c/dc4/c14e 0 2024-04-24T08:25:17.222 INFO:tasks.workunit.client.0.smithi090.stdout:4/877: mkdir d3/d8e/ddf/d115 0 2024-04-24T08:25:17.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/878: readlink d3/d8e/ddf/d5c/db5/d7c/le6 0 2024-04-24T08:25:17.224 INFO:tasks.workunit.client.0.smithi090.stdout:8/925: symlink d2/da/d13/d64/db2/dfe/d135/l13e 0 2024-04-24T08:25:17.226 INFO:tasks.workunit.client.0.smithi090.stdout:4/879: stat d3/d8e/ddf/d109/le0 0 2024-04-24T08:25:17.226 INFO:tasks.workunit.client.0.smithi090.stdout:9/945: rename d0/dc/d24/d7c/d4d/l2b to d0/d19/l131 0 2024-04-24T08:25:17.227 INFO:tasks.workunit.client.0.smithi090.stdout:4/880: readlink d3/dd0/lee 0 2024-04-24T08:25:17.227 INFO:tasks.workunit.client.0.smithi090.stdout:8/926: dread - d2/da/d13/d60/d91/ff8 zero size 2024-04-24T08:25:17.227 INFO:tasks.workunit.client.0.smithi090.stdout:0/940: link d1a/d4b/d53/d79/d69/d8a/ccb d1a/d4b/d53/d79/d35/d62/c13f 0 2024-04-24T08:25:17.232 INFO:tasks.workunit.client.0.smithi090.stdout:4/881: chown d3/d8e/ddf/c56 206356 1 2024-04-24T08:25:17.233 INFO:tasks.workunit.client.0.smithi090.stdout:6/987: dread d8/fa [0,4194304] 0 2024-04-24T08:25:17.234 INFO:tasks.workunit.client.0.smithi090.stdout:0/941: dread - d1a/d4b/d53/d79/d69/d90/dcc/fe9 zero size 2024-04-24T08:25:17.236 INFO:tasks.workunit.client.0.smithi090.stdout:9/946: dread - d0/dc/d24/d2d/d49/d62/dcb/dcd/ffb zero size 2024-04-24T08:25:17.237 INFO:tasks.workunit.client.0.smithi090.stdout:3/974: write d2/df/d1a/d37/d4b/fa8 [1871727,109209] 0 2024-04-24T08:25:17.241 INFO:tasks.workunit.client.0.smithi090.stdout:6/988: symlink d8/d1d/d1f/d3e/d44/d90/d106/d81/l138 0 2024-04-24T08:25:17.251 INFO:tasks.workunit.client.0.smithi090.stdout:4/882: creat d3/d8e/ddf/d5c/db5/d69/d8b/f116 x:0 0 0 2024-04-24T08:25:17.255 INFO:tasks.workunit.client.0.smithi090.stdout:4/883: truncate d3/d8e/ddf/d1d/f46 8804228 0 2024-04-24T08:25:17.256 INFO:tasks.workunit.client.0.smithi090.stdout:8/927: dwrite d2/da/d13/d64/d24/f21 [0,4194304] 0 2024-04-24T08:25:17.259 INFO:tasks.workunit.client.0.smithi090.stdout:0/942: getdents d1a/d4b/d53/d79/d69/d90/dcc 0 2024-04-24T08:25:17.263 INFO:tasks.workunit.client.0.smithi090.stdout:3/975: truncate d2/d25/d33/f80 1495882 0 2024-04-24T08:25:17.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/884: mknod d3/d8e/ddf/d1d/c117 0 2024-04-24T08:25:17.268 INFO:tasks.workunit.client.0.smithi090.stdout:6/989: dwrite d8/d1d/d1f/d3e/d10a/fad [4194304,4194304] 0 2024-04-24T08:25:17.269 INFO:tasks.workunit.client.0.smithi090.stdout:3/976: read d2/df/d1a/f4c [3857990,11204] 0 2024-04-24T08:25:17.269 INFO:tasks.workunit.client.0.smithi090.stdout:0/943: creat d1a/d11d/f140 x:0 0 0 2024-04-24T08:25:17.271 INFO:tasks.workunit.client.0.smithi090.stdout:3/977: chown d2/df/d1a/l69 0 1 2024-04-24T08:25:17.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/885: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/f118 x:0 0 0 2024-04-24T08:25:17.278 INFO:tasks.workunit.client.0.smithi090.stdout:0/944: rename d1a/d4b/d53/d57/lbf to d1a/d4b/d53/d57/d12e/l141 0 2024-04-24T08:25:17.280 INFO:tasks.workunit.client.0.smithi090.stdout:3/978: creat d2/d90/d135/f142 x:0 0 0 2024-04-24T08:25:17.282 INFO:tasks.workunit.client.0.smithi090.stdout:0/945: rmdir d1a/d4b/d53/d79/d58/d96 39 2024-04-24T08:25:17.285 INFO:tasks.workunit.client.0.smithi090.stdout:4/886: creat d3/d8e/ddf/d5c/db5/d77/dec/d102/f119 x:0 0 0 2024-04-24T08:25:17.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/979: mkdir d2/d120/d138/d87/db0/d136/d13e/d143 0 2024-04-24T08:25:17.289 INFO:tasks.workunit.client.0.smithi090.stdout:8/928: fdatasync d2/da/d13/d64/d24/f21 0 2024-04-24T08:25:17.291 INFO:tasks.workunit.client.0.smithi090.stdout:0/946: truncate d1a/d4b/d53/d79/d35/dea/dfe/f76 3735212 0 2024-04-24T08:25:17.301 INFO:tasks.workunit.client.0.smithi090.stdout:8/929: mkdir d2/d62/d83/d13f 0 2024-04-24T08:25:17.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/930: fdatasync d2/da/d13/d64/d24/f21 0 2024-04-24T08:25:17.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/980: link d2/df/d57/d4f/la0 d2/d25/d109/l144 0 2024-04-24T08:25:17.308 INFO:tasks.workunit.client.0.smithi090.stdout:8/931: dread - d2/da/d13/d64/db2/dbb/f127 zero size 2024-04-24T08:25:17.309 INFO:tasks.workunit.client.0.smithi090.stdout:4/887: dwrite d3/d8e/ddf/d5c/db5/d77/ff0 [0,4194304] 0 2024-04-24T08:25:17.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/932: chown d2/da/d13/fc5 0 1 2024-04-24T08:25:17.313 INFO:tasks.workunit.client.0.smithi090.stdout:6/990: dread d8/d1d/d1f/d3e/d10a/f5f [0,4194304] 0 2024-04-24T08:25:17.315 INFO:tasks.workunit.client.0.smithi090.stdout:0/947: dwrite d1a/d4b/d53/d79/d69/d90/dcc/ff7 [0,4194304] 0 2024-04-24T08:25:17.316 INFO:tasks.workunit.client.0.smithi090.stdout:3/981: getdents d2 0 2024-04-24T08:25:17.323 INFO:tasks.workunit.client.0.smithi090.stdout:8/933: chown d2/d62/f93 35 1 2024-04-24T08:25:17.323 INFO:tasks.workunit.client.0.smithi090.stdout:8/934: readlink d2/da/d13/d34/l58 0 2024-04-24T08:25:17.326 INFO:tasks.workunit.client.0.smithi090.stdout:1/944: sync 2024-04-24T08:25:17.329 INFO:tasks.workunit.client.0.smithi090.stdout:6/991: truncate d8/d1d/d29/fe3 378395 0 2024-04-24T08:25:17.332 INFO:tasks.workunit.client.0.smithi090.stdout:1/945: chown d3/d5/da/d94/lbe 127662450 1 2024-04-24T08:25:17.335 INFO:tasks.workunit.client.0.smithi090.stdout:3/982: truncate d2/df/d1a/d37/d4b/f112 2405802 0 2024-04-24T08:25:17.337 INFO:tasks.workunit.client.0.smithi090.stdout:1/946: mkdir d3/d68/df2/d129/d14f 0 2024-04-24T08:25:17.344 INFO:tasks.workunit.client.0.smithi090.stdout:3/983: write d2/df/db5/deb/f127 [254024,177] 0 2024-04-24T08:25:17.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/888: dwrite d3/fd1 [0,4194304] 0 2024-04-24T08:25:17.352 INFO:tasks.workunit.client.0.smithi090.stdout:3/984: chown d2/df/d1a/d37/d4b/l9e 6535461 1 2024-04-24T08:25:17.357 INFO:tasks.workunit.client.0.smithi090.stdout:0/948: dwrite d1a/d4b/d53/d79/d69/d90/dcc/fe9 [0,4194304] 0 2024-04-24T08:25:17.362 INFO:tasks.workunit.client.0.smithi090.stdout:4/889: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f11a x:0 0 0 2024-04-24T08:25:17.364 INFO:tasks.workunit.client.0.smithi090.stdout:9/947: sync 2024-04-24T08:25:17.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/992: dread d8/d1d/d1f/d3e/d44/f82 [0,4194304] 0 2024-04-24T08:25:17.366 INFO:tasks.workunit.client.0.smithi090.stdout:1/947: dread d3/d1d/d34/d5a/d143/d140/d142/da3/fd0 [0,4194304] 0 2024-04-24T08:25:17.366 INFO:tasks.workunit.client.0.smithi090.stdout:3/985: mknod d2/d120/d138/d87/db0/d136/d13e/d143/c145 0 2024-04-24T08:25:17.367 INFO:tasks.workunit.client.0.smithi090.stdout:9/948: readlink d0/dc/d24/d7c/d4d/le4 0 2024-04-24T08:25:17.368 INFO:tasks.workunit.client.0.smithi090.stdout:3/986: fdatasync d2/d25/f108 0 2024-04-24T08:25:17.372 INFO:tasks.workunit.client.0.smithi090.stdout:0/949: write d1a/d4b/d53/d79/d35/dea/f135 [1538843,29646] 0 2024-04-24T08:25:17.374 INFO:tasks.workunit.client.0.smithi090.stdout:4/890: rename d3/d24/c37 to d3/d87/c11b 0 2024-04-24T08:25:17.376 INFO:tasks.workunit.client.0.smithi090.stdout:9/949: chown d0/dc/d24/d2d/d49/d62/f68 87778 1 2024-04-24T08:25:17.376 INFO:tasks.workunit.client.0.smithi090.stdout:3/987: write d2/d25/d2c/f10a [778756,120414] 0 2024-04-24T08:25:17.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/993: symlink d8/d1d/d1f/d3e/d44/d90/d106/l139 0 2024-04-24T08:25:17.381 INFO:tasks.workunit.client.0.smithi090.stdout:8/935: dread d2/f59 [0,4194304] 0 2024-04-24T08:25:17.386 INFO:tasks.workunit.client.0.smithi090.stdout:3/988: rename d2/d15/fda to d2/d120/d138/d87/f146 0 2024-04-24T08:25:17.386 INFO:tasks.workunit.client.0.smithi090.stdout:6/994: chown d8/d1d/d1f/d3e/d44/d90/d106/d81/lee 143 1 2024-04-24T08:25:17.393 INFO:tasks.workunit.client.0.smithi090.stdout:8/936: rename d2/da/d13/d64/f8e to d2/d62/d71/f140 0 2024-04-24T08:25:17.396 INFO:tasks.workunit.client.0.smithi090.stdout:0/950: rename d1a/d4b/d53/d79/d35/dea/dfe/ca7 to d1a/d4b/d53/d79/d35/dea/dfe/c142 0 2024-04-24T08:25:17.404 INFO:tasks.workunit.client.0.smithi090.stdout:3/989: rename d2/d84/cff to d2/df/d1a/d37/d4b/c147 0 2024-04-24T08:25:17.405 INFO:tasks.workunit.client.0.smithi090.stdout:9/950: write d0/f59 [485357,78666] 0 2024-04-24T08:25:17.408 INFO:tasks.workunit.client.0.smithi090.stdout:4/891: getdents d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49 0 2024-04-24T08:25:17.409 INFO:tasks.workunit.client.0.smithi090.stdout:0/951: creat d1a/d4b/d53/d79/d35/f143 x:0 0 0 2024-04-24T08:25:17.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/990: read d2/df/f10 [13034,18731] 0 2024-04-24T08:25:17.409 INFO:tasks.workunit.client.0.smithi090.stdout:8/937: stat d2/d62/d83/f100 0 2024-04-24T08:25:17.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/995: rename d8/d1d/d1f/d3e/d10a/d19/dd2/c10e to d8/d1d/d1f/d3e/d44/c13a 0 2024-04-24T08:25:17.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/948: dwrite d3/d5/da/d94/da6/d115/d30/d9d/fe0 [0,4194304] 0 2024-04-24T08:25:17.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/938: truncate d2/f10 5839292 0 2024-04-24T08:25:17.429 INFO:tasks.workunit.client.0.smithi090.stdout:8/939: chown d2/l2e 36 1 2024-04-24T08:25:17.429 INFO:tasks.workunit.client.0.smithi090.stdout:6/996: unlink d8/d1d/d1f/d3e/d44/d90/cb9 0 2024-04-24T08:25:17.430 INFO:tasks.workunit.client.0.smithi090.stdout:3/991: creat d2/d15/d36/d6f/d126/d128/f148 x:0 0 0 2024-04-24T08:25:17.432 INFO:tasks.workunit.client.0.smithi090.stdout:4/892: dread d3/ff6 [0,4194304] 0 2024-04-24T08:25:17.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/992: read d2/df/d1a/d37/d53/fe1 [709837,87635] 0 2024-04-24T08:25:17.436 INFO:tasks.workunit.client.0.smithi090.stdout:3/993: dread - d2/df/d1a/d37/dc0/de6/f111 zero size 2024-04-24T08:25:17.459 INFO:tasks.workunit.client.0.smithi090.stdout:1/949: dread d3/d5/da/dc/d80/fff [0,4194304] 0 2024-04-24T08:25:17.461 INFO:tasks.workunit.client.0.smithi090.stdout:0/952: dread d1a/d4b/d53/d79/d82/f98 [0,4194304] 0 2024-04-24T08:25:17.474 INFO:tasks.workunit.client.0.smithi090.stdout:9/951: creat d0/dc/f132 x:0 0 0 2024-04-24T08:25:17.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/893: creat d3/d87/f11c x:0 0 0 2024-04-24T08:25:17.477 INFO:tasks.workunit.client.0.smithi090.stdout:9/952: write d0/d19/df7/ffd [770878,64399] 0 2024-04-24T08:25:17.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/997: unlink d8/d1d/d1f/d3e/d44/d90/d106/d81/d96/d97/ffe 0 2024-04-24T08:25:17.479 INFO:tasks.workunit.client.0.smithi090.stdout:3/994: unlink d2/df/d1a/c5c 0 2024-04-24T08:25:17.481 INFO:tasks.workunit.client.0.smithi090.stdout:3/995: stat d2/df/d1a/d37/d53/d60 0 2024-04-24T08:25:17.483 INFO:tasks.workunit.client.0.smithi090.stdout:9/953: truncate d0/dc/d24/d2d/d49/d81/fea 1028711 0 2024-04-24T08:25:17.489 INFO:tasks.workunit.client.0.smithi090.stdout:6/998: rmdir d8/d1d/d1f/d3e/d4f 39 2024-04-24T08:25:17.490 INFO:tasks.workunit.client.0.smithi090.stdout:3/996: creat d2/d25/d2c/f149 x:0 0 0 2024-04-24T08:25:17.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/997: chown d2/df/d1a/d37/d53/d60/d9b/d12b 206017 1 2024-04-24T08:25:17.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/999: readlink d8/d1d/d1f/d3e/d44/d90/d106/d81/lee 0 2024-04-24T08:25:17.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/894: getdents d3/d8e/ddf/d1d 0 2024-04-24T08:25:17.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/998: write d2/df/d5b/d105/f11a [1007130,100260] 0 2024-04-24T08:25:17.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/895: symlink d3/d8e/ddf/d109/l11d 0 2024-04-24T08:25:17.503 INFO:tasks.workunit.client.0.smithi090.stdout:3/999: creat d2/d15/f14a x:0 0 0 2024-04-24T08:25:17.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/896: rename d3/d8e/ddf/d5c/db5/d69/d8b/db2/lc0 to d3/d87/da6/l11e 0 2024-04-24T08:25:17.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/954: write d0/d2a/fb8 [1745954,10844] 0 2024-04-24T08:25:17.527 INFO:tasks.workunit.client.0.smithi090.stdout:1/950: dwrite d3/d5/da/dc/f40 [0,4194304] 0 2024-04-24T08:25:17.528 INFO:tasks.workunit.client.0.smithi090.stdout:9/955: unlink d0/dc/d24/d7c/d4d/d8c/de2/ccc 0 2024-04-24T08:25:17.537 INFO:tasks.workunit.client.0.smithi090.stdout:1/951: getdents d3/d5/d58 0 2024-04-24T08:25:17.543 INFO:tasks.workunit.client.0.smithi090.stdout:9/956: rename d0/dc/d24/d2d/d49/d62/d12d to d0/dc/d24/d2d/dda/d133 0 2024-04-24T08:25:17.544 INFO:tasks.workunit.client.0.smithi090.stdout:4/897: dread d3/d8e/ddf/d5c/db5/d7c/fa9 [0,4194304] 0 2024-04-24T08:25:17.544 INFO:tasks.workunit.client.0.smithi090.stdout:1/952: rmdir d3/d5/da/d94/da6 39 2024-04-24T08:25:17.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/898: symlink d3/d8e/ddf/d5c/db5/l11f 0 2024-04-24T08:25:17.561 INFO:tasks.workunit.client.0.smithi090.stdout:4/899: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/f39 [8255966,57533] 0 2024-04-24T08:25:17.585 INFO:tasks.workunit.client.0.smithi090.stdout:4/900: rename d3/d8e/ddf/d5c/db5/f98 to d3/d8e/ddf/d5c/db5/d77/f120 0 2024-04-24T08:25:17.587 INFO:tasks.workunit.client.0.smithi090.stdout:1/953: dwrite d3/d5/da/d94/da6/d115/d30/d9d/da2/fd6 [0,4194304] 0 2024-04-24T08:25:17.590 INFO:tasks.workunit.client.0.smithi090.stdout:4/901: read - d3/d8e/ddf/d5c/db5/d69/ffc zero size 2024-04-24T08:25:17.600 INFO:tasks.workunit.client.0.smithi090.stdout:1/954: rename d3/d1d/d34/d73/ff6 to d3/d5/da/dc/f150 0 2024-04-24T08:25:17.603 INFO:tasks.workunit.client.0.smithi090.stdout:4/902: symlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/l121 0 2024-04-24T08:25:17.607 INFO:tasks.workunit.client.0.smithi090.stdout:1/955: mknod d3/d5/d10e/c151 0 2024-04-24T08:25:17.611 INFO:tasks.workunit.client.0.smithi090.stdout:1/956: chown d3/d5/da/d94/da6/fb9 4598 1 2024-04-24T08:25:17.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/957: dread d3/d5/da/dc/d80/deb/d111/d12c/f149 [0,4194304] 0 2024-04-24T08:25:17.626 INFO:tasks.workunit.client.0.smithi090.stdout:4/903: truncate d3/d8e/ddf/fef 4118664 0 2024-04-24T08:25:17.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/940: sync 2024-04-24T08:25:17.636 INFO:tasks.workunit.client.0.smithi090.stdout:1/958: write d3/d5/da/dc/f87 [4185061,22083] 0 2024-04-24T08:25:17.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/904: write d3/d8e/ddf/d5c/db5/d7c/fa9 [446640,100261] 0 2024-04-24T08:25:17.641 INFO:tasks.workunit.client.0.smithi090.stdout:1/959: chown d3/d5/d58/fe5 57292624 1 2024-04-24T08:25:17.646 INFO:tasks.workunit.client.0.smithi090.stdout:4/905: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/c76 495 1 2024-04-24T08:25:17.649 INFO:tasks.workunit.client.0.smithi090.stdout:1/960: truncate d3/d68/df2/ffd 941737 0 2024-04-24T08:25:17.657 INFO:tasks.workunit.client.0.smithi090.stdout:1/961: read d3/d5/da/d94/da6/d115/d30/d9d/fe0 [1227672,25766] 0 2024-04-24T08:25:17.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/962: creat d3/d5/d58/f152 x:0 0 0 2024-04-24T08:25:17.675 INFO:tasks.workunit.client.0.smithi090.stdout:8/941: dwrite d2/d62/f6e [0,4194304] 0 2024-04-24T08:25:17.677 INFO:tasks.workunit.client.0.smithi090.stdout:1/963: write d3/d5/da/dc/fe3 [529068,22898] 0 2024-04-24T08:25:17.678 INFO:tasks.workunit.client.0.smithi090.stdout:4/906: dread d3/d8e/ddf/d5c/db5/d77/ff0 [0,4194304] 0 2024-04-24T08:25:17.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/964: fsync d3/d1d/f86 0 2024-04-24T08:25:17.687 INFO:tasks.workunit.client.0.smithi090.stdout:1/965: creat d3/d1d/d54/f153 x:0 0 0 2024-04-24T08:25:17.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/957: sync 2024-04-24T08:25:17.688 INFO:tasks.workunit.client.0.smithi090.stdout:0/953: sync 2024-04-24T08:25:17.695 INFO:tasks.workunit.client.0.smithi090.stdout:0/954: unlink d1a/d4b/d53/d79/d3c/c95 0 2024-04-24T08:25:17.697 INFO:tasks.workunit.client.0.smithi090.stdout:9/958: creat d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/f134 x:0 0 0 2024-04-24T08:25:17.698 INFO:tasks.workunit.client.0.smithi090.stdout:1/966: chown d3/d5/da/d94/da6/d115/d30/d9d/da2/cb6 447001235 1 2024-04-24T08:25:17.700 INFO:tasks.workunit.client.0.smithi090.stdout:0/955: dread - d1a/d4b/d53/d79/d35/f143 zero size 2024-04-24T08:25:17.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/959: write d0/dc/d24/d7c/f11c [58933,90914] 0 2024-04-24T08:25:17.704 INFO:tasks.workunit.client.0.smithi090.stdout:1/967: symlink d3/d5/d58/d11f/l154 0 2024-04-24T08:25:17.712 INFO:tasks.workunit.client.0.smithi090.stdout:1/968: mkdir d3/d5/da/d94/d10b/d155 0 2024-04-24T08:25:17.717 INFO:tasks.workunit.client.0.smithi090.stdout:9/960: mkdir d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110/d135 0 2024-04-24T08:25:17.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/907: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f62 [0,4194304] 0 2024-04-24T08:25:17.718 INFO:tasks.workunit.client.0.smithi090.stdout:0/956: write d1a/d4b/d53/d79/d3c/f54 [421293,4720] 0 2024-04-24T08:25:17.719 INFO:tasks.workunit.client.0.smithi090.stdout:1/969: mknod d3/d68/d7f/d101/c156 0 2024-04-24T08:25:17.720 INFO:tasks.workunit.client.0.smithi090.stdout:9/961: stat d0/dc/d24 0 2024-04-24T08:25:17.723 INFO:tasks.workunit.client.0.smithi090.stdout:4/908: mknod d3/d87/da6/c122 0 2024-04-24T08:25:17.725 INFO:tasks.workunit.client.0.smithi090.stdout:9/962: chown d0/dc/d24/d7c/d4d/c67 3500 1 2024-04-24T08:25:17.728 INFO:tasks.workunit.client.0.smithi090.stdout:0/957: dread d1a/d4b/d53/d79/d58/fd3 [0,4194304] 0 2024-04-24T08:25:17.728 INFO:tasks.workunit.client.0.smithi090.stdout:1/970: creat d3/d5/f157 x:0 0 0 2024-04-24T08:25:17.733 INFO:tasks.workunit.client.0.smithi090.stdout:0/958: fsync d1a/d4b/d53/fbb 0 2024-04-24T08:25:17.734 INFO:tasks.workunit.client.0.smithi090.stdout:1/971: mknod d3/d1d/d34/d73/dd8/c158 0 2024-04-24T08:25:17.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/963: truncate d0/dc/d24/d7c/d4d/d32/d9f/d78/f12b 770146 0 2024-04-24T08:25:17.737 INFO:tasks.workunit.client.0.smithi090.stdout:0/959: read - d1a/d4b/d53/d79/d82/f129 zero size 2024-04-24T08:25:17.737 INFO:tasks.workunit.client.0.smithi090.stdout:1/972: chown d3/d1d/d54/f153 2 1 2024-04-24T08:25:17.740 INFO:tasks.workunit.client.0.smithi090.stdout:9/964: read d0/dc/d24/d2d/d49/d81/fea [349200,52507] 0 2024-04-24T08:25:17.741 INFO:tasks.workunit.client.0.smithi090.stdout:0/960: chown d1a/d4b/d53/d79/d69/d78/fd7 135 1 2024-04-24T08:25:17.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/961: chown d1a/f1b 1 1 2024-04-24T08:25:17.744 INFO:tasks.workunit.client.0.smithi090.stdout:9/965: rmdir d0/dc/d24/d7c/d4d 39 2024-04-24T08:25:17.749 INFO:tasks.workunit.client.0.smithi090.stdout:0/962: truncate d1a/d4b/d53/d79/d69/d90/fdc 980432 0 2024-04-24T08:25:17.749 INFO:tasks.workunit.client.0.smithi090.stdout:9/966: mknod d0/dc/d24/d2d/d49/d62/dcb/dcd/c136 0 2024-04-24T08:25:17.754 INFO:tasks.workunit.client.0.smithi090.stdout:9/967: readlink d0/dc/d24/d7c/d4d/d32/d9f/d72/de9/lf9 0 2024-04-24T08:25:17.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/968: readlink d0/d2a/l61 0 2024-04-24T08:25:17.760 INFO:tasks.workunit.client.0.smithi090.stdout:0/963: getdents d1a/d4b/d53/d79/d69/d78 0 2024-04-24T08:25:17.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/973: write d3/d5/da/dc/d80/f7d [641517,43668] 0 2024-04-24T08:25:17.765 INFO:tasks.workunit.client.0.smithi090.stdout:0/964: chown d1a/d4b/d53/d79/d35/d8e/d119 0 1 2024-04-24T08:25:17.766 INFO:tasks.workunit.client.0.smithi090.stdout:8/942: sync 2024-04-24T08:25:17.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/965: stat d1a/d4b/d53/d79/f32 0 2024-04-24T08:25:17.767 INFO:tasks.workunit.client.0.smithi090.stdout:4/909: dread d3/d8e/ddf/d5c/db5/f36 [0,4194304] 0 2024-04-24T08:25:17.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/969: getdents d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d 0 2024-04-24T08:25:17.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/974: rename d3/d1d/d34/d5a/l9a to d3/d5/da/dc/d80/deb/l159 0 2024-04-24T08:25:17.772 INFO:tasks.workunit.client.0.smithi090.stdout:0/966: chown d1a/d4b/d53/d79/d58/db4/lc2 27637983 1 2024-04-24T08:25:17.773 INFO:tasks.workunit.client.0.smithi090.stdout:8/943: mknod d2/d62/d83/c141 0 2024-04-24T08:25:17.774 INFO:tasks.workunit.client.0.smithi090.stdout:4/910: truncate d3/d8e/ddf/d1d/f1f 4058785 0 2024-04-24T08:25:17.775 INFO:tasks.workunit.client.0.smithi090.stdout:9/970: creat d0/dc/d24/d2d/d49/d62/def/d109/f137 x:0 0 0 2024-04-24T08:25:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:17 smithi149 ceph-mon[31282]: pgmap v31: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 187 MiB/s wr, 425 op/s 2024-04-24T08:25:17.781 INFO:tasks.workunit.client.0.smithi090.stdout:9/971: chown d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/l120 283 1 2024-04-24T08:25:17.781 INFO:tasks.workunit.client.0.smithi090.stdout:0/967: dread d1a/d4b/d53/d79/d69/d78/fd7 [0,4194304] 0 2024-04-24T08:25:17.782 INFO:tasks.workunit.client.0.smithi090.stdout:4/911: getdents d3/d105 0 2024-04-24T08:25:17.785 INFO:tasks.workunit.client.0.smithi090.stdout:9/972: chown d0/dc/d24/d7c/d4d/d32/d9f/d72/fcf 7 1 2024-04-24T08:25:17.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/968: symlink d1a/d11d/l144 0 2024-04-24T08:25:17.788 INFO:tasks.workunit.client.0.smithi090.stdout:8/944: write d2/da/d13/d64/d24/f5b [717728,89143] 0 2024-04-24T08:25:17.790 INFO:tasks.workunit.client.0.smithi090.stdout:0/969: rmdir d1a/d4b/d53/d79/d35 39 2024-04-24T08:25:17.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:17 smithi090 ceph-mon[25016]: pgmap v31: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 187 MiB/s wr, 425 op/s 2024-04-24T08:25:17.793 INFO:tasks.workunit.client.0.smithi090.stdout:9/973: creat d0/dc/d24/d2d/d49/d81/d87/f138 x:0 0 0 2024-04-24T08:25:17.794 INFO:tasks.workunit.client.0.smithi090.stdout:8/945: readlink d2/da/d13/d60/d76/l124 0 2024-04-24T08:25:17.797 INFO:tasks.workunit.client.0.smithi090.stdout:8/946: chown d2/da/d13/d60/l6a 40603500 1 2024-04-24T08:25:17.802 INFO:tasks.workunit.client.0.smithi090.stdout:0/970: getdents d1a/d4b/d53/d79/d82 0 2024-04-24T08:25:17.803 INFO:tasks.workunit.client.0.smithi090.stdout:9/974: creat d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/dbc/dd9/d110/d11b/f139 x:0 0 0 2024-04-24T08:25:17.805 INFO:tasks.workunit.client.0.smithi090.stdout:8/947: mkdir d2/da/d13/d64/db2/d142 0 2024-04-24T08:25:17.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/912: dwrite d3/d8e/fd5 [0,4194304] 0 2024-04-24T08:25:17.809 INFO:tasks.workunit.client.0.smithi090.stdout:8/948: chown d2/da/d13/d64/d50/f5d 0 1 2024-04-24T08:25:17.810 INFO:tasks.workunit.client.0.smithi090.stdout:4/913: stat d3/d8e/ddf/d1d/cce 0 2024-04-24T08:25:17.812 INFO:tasks.workunit.client.0.smithi090.stdout:9/975: getdents d0/dc 0 2024-04-24T08:25:17.815 INFO:tasks.workunit.client.0.smithi090.stdout:8/949: truncate d2/da/d13/f4b 2170322 0 2024-04-24T08:25:17.816 INFO:tasks.workunit.client.0.smithi090.stdout:4/914: readlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/lc7 0 2024-04-24T08:25:17.819 INFO:tasks.workunit.client.0.smithi090.stdout:8/950: readlink d2/da/d13/d64/d50/l13d 0 2024-04-24T08:25:17.822 INFO:tasks.workunit.client.0.smithi090.stdout:8/951: write d2/d62/d83/da3/deb/f139 [451640,18194] 0 2024-04-24T08:25:17.822 INFO:tasks.workunit.client.0.smithi090.stdout:4/915: symlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/l123 0 2024-04-24T08:25:17.826 INFO:tasks.workunit.client.0.smithi090.stdout:8/952: symlink d2/d62/d71/l143 0 2024-04-24T08:25:17.829 INFO:tasks.workunit.client.0.smithi090.stdout:0/971: truncate d1a/d4b/d53/d79/d82/d9d/f11f 3973100 0 2024-04-24T08:25:17.833 INFO:tasks.workunit.client.0.smithi090.stdout:8/953: link d2/da/c1b d2/da/d13/d64/d24/dad/d10c/d125/d132/c144 0 2024-04-24T08:25:17.837 INFO:tasks.workunit.client.0.smithi090.stdout:0/972: rename f8 to d1a/d4b/d53/d79/d35/dea/dfe/f145 0 2024-04-24T08:25:17.838 INFO:tasks.workunit.client.0.smithi090.stdout:9/976: dwrite d0/dc/d24/d7c/d4d/f84 [0,4194304] 0 2024-04-24T08:25:17.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/977: creat d0/d19/dfc/f13a x:0 0 0 2024-04-24T08:25:17.847 INFO:tasks.workunit.client.0.smithi090.stdout:8/954: write d2/da/f82 [1525233,15683] 0 2024-04-24T08:25:17.848 INFO:tasks.workunit.client.0.smithi090.stdout:9/978: rmdir d0/d2a/d106 39 2024-04-24T08:25:17.854 INFO:tasks.workunit.client.0.smithi090.stdout:8/955: mkdir d2/da/d13/d60/d76/d145 0 2024-04-24T08:25:17.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/979: truncate d0/dc/d24/d2d/f47 3485595 0 2024-04-24T08:25:17.854 INFO:tasks.workunit.client.0.smithi090.stdout:4/916: dwrite d3/d8e/ddf/d5c/db5/f84 [0,4194304] 0 2024-04-24T08:25:17.859 INFO:tasks.workunit.client.0.smithi090.stdout:9/980: write d0/d19/f1a [1036864,120775] 0 2024-04-24T08:25:17.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/917: symlink d3/d8e/ddf/d5c/db5/d7c/l124 0 2024-04-24T08:25:17.871 INFO:tasks.workunit.client.0.smithi090.stdout:4/918: rename d3/d8e/ddf/d5c/db5/d77/dec/d102 to d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/d125 0 2024-04-24T08:25:17.877 INFO:tasks.workunit.client.0.smithi090.stdout:9/981: dwrite d0/dc/d24/d2d/d49/d62/dcb/dcd/ffb [0,4194304] 0 2024-04-24T08:25:17.885 INFO:tasks.workunit.client.0.smithi090.stdout:8/956: truncate d2/da/d13/d64/d24/f79 4587612 0 2024-04-24T08:25:17.890 INFO:tasks.workunit.client.0.smithi090.stdout:8/957: write d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e/f12f [174406,128403] 0 2024-04-24T08:25:17.893 INFO:tasks.workunit.client.0.smithi090.stdout:9/982: dwrite d0/fb [0,4194304] 0 2024-04-24T08:25:17.895 INFO:tasks.workunit.client.0.smithi090.stdout:9/983: stat d0/dc/d24/d2d/d49/d62/def 0 2024-04-24T08:25:17.897 INFO:tasks.workunit.client.0.smithi090.stdout:8/958: symlink d2/da/d13/d60/d91/dfb/l146 0 2024-04-24T08:25:17.899 INFO:tasks.workunit.client.0.smithi090.stdout:8/959: chown d2/da/d13/fae 1496 1 2024-04-24T08:25:17.902 INFO:tasks.workunit.client.0.smithi090.stdout:8/960: mknod d2/d62/d83/d13f/c147 0 2024-04-24T08:25:17.909 INFO:tasks.workunit.client.0.smithi090.stdout:8/961: dread d2/da/d13/d64/db2/dfe/f13b [0,4194304] 0 2024-04-24T08:25:17.916 INFO:tasks.workunit.client.0.smithi090.stdout:8/962: mkdir d2/da/d13/d64/d24/db3/db4/dcd/d11a/d148 0 2024-04-24T08:25:17.920 INFO:tasks.workunit.client.0.smithi090.stdout:4/919: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f93 [0,4194304] 0 2024-04-24T08:25:17.921 INFO:tasks.workunit.client.0.smithi090.stdout:8/963: mkdir d2/da/d13/d64/d24/dad/d10c/d149 0 2024-04-24T08:25:17.927 INFO:tasks.workunit.client.0.smithi090.stdout:8/964: mknod d2/d62/d83/da3/c14a 0 2024-04-24T08:25:17.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/984: dread d0/dc/d24/d7c/d4d/d32/f3c [0,4194304] 0 2024-04-24T08:25:17.932 INFO:tasks.workunit.client.0.smithi090.stdout:4/920: mknod d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/c126 0 2024-04-24T08:25:17.937 INFO:tasks.workunit.client.0.smithi090.stdout:9/985: unlink d0/dc/d24/d7c/d4d/d32/d9f/d72/d77/fb2 0 2024-04-24T08:25:17.938 INFO:tasks.workunit.client.0.smithi090.stdout:4/921: dread - d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/fad zero size 2024-04-24T08:25:17.940 INFO:tasks.workunit.client.0.smithi090.stdout:4/922: write d3/fd1 [892348,124216] 0 2024-04-24T08:25:17.944 INFO:tasks.workunit.client.0.smithi090.stdout:4/923: mkdir d3/d8e/ddf/d109/d127 0 2024-04-24T08:25:17.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/965: dwrite d2/d62/d71/fc9 [0,4194304] 0 2024-04-24T08:25:17.948 INFO:tasks.workunit.client.0.smithi090.stdout:9/986: getdents d0/dc/d24 0 2024-04-24T08:25:17.952 INFO:tasks.workunit.client.0.smithi090.stdout:9/987: mknod d0/d19/df7/c13b 0 2024-04-24T08:25:17.955 INFO:tasks.workunit.client.0.smithi090.stdout:1/975: sync 2024-04-24T08:25:17.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/988: chown d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/ld6 508284 1 2024-04-24T08:25:17.958 INFO:tasks.workunit.client.0.smithi090.stdout:8/966: rename d2/da/d13/d34/c5e to d2/da/d13/d64/db2/dbb/c14b 0 2024-04-24T08:25:17.970 INFO:tasks.workunit.client.0.smithi090.stdout:8/967: dread d2/d62/f6e [0,4194304] 0 2024-04-24T08:25:17.979 INFO:tasks.workunit.client.0.smithi090.stdout:8/968: mknod d2/da/d13/d64/d50/c14c 0 2024-04-24T08:25:17.983 INFO:tasks.workunit.client.0.smithi090.stdout:4/924: getdents d3/d8e/ddf/d5c/db5/db6 0 2024-04-24T08:25:17.984 INFO:tasks.workunit.client.0.smithi090.stdout:0/973: sync 2024-04-24T08:25:17.993 INFO:tasks.workunit.client.0.smithi090.stdout:8/969: write d2/f10 [986512,73416] 0 2024-04-24T08:25:17.995 INFO:tasks.workunit.client.0.smithi090.stdout:9/989: link d0/d2a/fb8 d0/dc/d24/d2d/d49/f13c 0 2024-04-24T08:25:18.003 INFO:tasks.workunit.client.0.smithi090.stdout:1/976: getdents d3/d68/df2/d129 0 2024-04-24T08:25:18.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/974: unlink d1a/d4b/d53/f7f 0 2024-04-24T08:25:18.013 INFO:tasks.workunit.client.0.smithi090.stdout:1/977: mknod d3/d5/c15a 0 2024-04-24T08:25:18.014 INFO:tasks.workunit.client.0.smithi090.stdout:0/975: creat d1a/d4b/d53/d79/d35/d62/f146 x:0 0 0 2024-04-24T08:25:18.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/976: unlink d1a/d4b/d53/d79/d3c/l43 0 2024-04-24T08:25:18.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/977: readlink d1a/l60 0 2024-04-24T08:25:18.028 INFO:tasks.workunit.client.0.smithi090.stdout:1/978: getdents d3/d5/d10e 0 2024-04-24T08:25:18.037 INFO:tasks.workunit.client.0.smithi090.stdout:0/978: mkdir d1a/d4b/d53/d79/d35/d8e/d147 0 2024-04-24T08:25:18.044 INFO:tasks.workunit.client.0.smithi090.stdout:0/979: mknod d1a/d4b/d53/d57/c148 0 2024-04-24T08:25:18.049 INFO:tasks.workunit.client.0.smithi090.stdout:9/990: dwrite d0/d2a/faa [0,4194304] 0 2024-04-24T08:25:18.050 INFO:tasks.workunit.client.0.smithi090.stdout:4/925: dwrite d3/d8e/ddf/fef [0,4194304] 0 2024-04-24T08:25:18.055 INFO:tasks.workunit.client.0.smithi090.stdout:0/980: write d1a/d4b/d53/d79/d35/f131 [3945664,83943] 0 2024-04-24T08:25:18.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/991: symlink d0/da4/l13d 0 2024-04-24T08:25:18.060 INFO:tasks.workunit.client.0.smithi090.stdout:0/981: readlink d1a/d4b/d53/d79/d69/d90/dcc/l107 0 2024-04-24T08:25:18.060 INFO:tasks.workunit.client.0.smithi090.stdout:9/992: symlink d0/dc/l13e 0 2024-04-24T08:25:18.061 INFO:tasks.workunit.client.0.smithi090.stdout:1/979: dread d3/d5/da/d97/d9c/dc4/ffb [0,4194304] 0 2024-04-24T08:25:18.066 INFO:tasks.workunit.client.0.smithi090.stdout:0/982: write d1a/d4b/d53/d79/d82/f123 [513523,41712] 0 2024-04-24T08:25:18.067 INFO:tasks.workunit.client.0.smithi090.stdout:9/993: dread - d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/fb7 zero size 2024-04-24T08:25:18.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/994: chown d0/dc/d24/d7c/d4d/l66 26916345 1 2024-04-24T08:25:18.071 INFO:tasks.workunit.client.0.smithi090.stdout:1/980: mkdir d3/d5/da/d94/d10b/d155/d15b 0 2024-04-24T08:25:18.073 INFO:tasks.workunit.client.0.smithi090.stdout:4/926: dread d3/d8e/ddf/d1d/f40 [0,4194304] 0 2024-04-24T08:25:18.076 INFO:tasks.workunit.client.0.smithi090.stdout:0/983: symlink d1a/d4b/d53/d79/d82/d9d/d116/d12a/l149 0 2024-04-24T08:25:18.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/984: stat d1a/d4b/d53/d79/d35/db8/fdd 0 2024-04-24T08:25:18.087 INFO:tasks.workunit.client.0.smithi090.stdout:0/985: read d1a/d4b/d53/d79/d58/db4/fb5 [1666264,45514] 0 2024-04-24T08:25:18.088 INFO:tasks.workunit.client.0.smithi090.stdout:4/927: rename d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/cf9 to d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/c128 0 2024-04-24T08:25:18.091 INFO:tasks.workunit.client.0.smithi090.stdout:4/928: chown d3/d8e/ddf/d109/le0 117741 1 2024-04-24T08:25:18.091 INFO:tasks.workunit.client.0.smithi090.stdout:0/986: dread - d1a/d4b/d53/d79/d58/db4/fde zero size 2024-04-24T08:25:18.094 INFO:tasks.workunit.client.0.smithi090.stdout:0/987: dread - d1a/d4b/d53/d79/d35/d62/f146 zero size 2024-04-24T08:25:18.096 INFO:tasks.workunit.client.0.smithi090.stdout:4/929: rmdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb 39 2024-04-24T08:25:18.097 INFO:tasks.workunit.client.0.smithi090.stdout:0/988: rmdir d1a/d4b/d53/d79/d69/d90 39 2024-04-24T08:25:18.098 INFO:tasks.workunit.client.0.smithi090.stdout:4/930: readlink d3/d8e/ddf/d5c/db5/lb1 0 2024-04-24T08:25:18.098 INFO:tasks.workunit.client.0.smithi090.stdout:1/981: dwrite d3/d5/da/d94/da6/d115/d30/d9d/da2/fde [0,4194304] 0 2024-04-24T08:25:18.100 INFO:tasks.workunit.client.0.smithi090.stdout:4/931: chown d3/d8e/ddf/d5c/f7f 2203 1 2024-04-24T08:25:18.105 INFO:tasks.workunit.client.0.smithi090.stdout:4/932: creat d3/d8e/ddf/d115/f129 x:0 0 0 2024-04-24T08:25:18.107 INFO:tasks.workunit.client.0.smithi090.stdout:0/989: getdents d1a/d4b/d53/d79/d82/d9d/d116 0 2024-04-24T08:25:18.109 INFO:tasks.workunit.client.0.smithi090.stdout:8/970: sync 2024-04-24T08:25:18.124 INFO:tasks.workunit.client.0.smithi090.stdout:9/995: write d0/dc/d24/d2d/d49/d62/dcb/dcd/dd1/d7d/d5b/d92/fc1 [1179206,25189] 0 2024-04-24T08:25:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:9/996: dread d0/dc/d24/d7c/d4d/f52 [0,4194304] 0 2024-04-24T08:25:18.153 INFO:tasks.workunit.client.0.smithi090.stdout:4/933: truncate d3/d8e/ddf/d5c/f7f 1220648 0 2024-04-24T08:25:18.159 INFO:tasks.workunit.client.0.smithi090.stdout:1/982: dwrite d3/d5/da/fc1 [0,4194304] 0 2024-04-24T08:25:18.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/997: mkdir d0/d13f 0 2024-04-24T08:25:18.177 INFO:tasks.workunit.client.0.smithi090.stdout:1/983: rename d3/d5/da/dc/d80/deb/d111/d12c/f149 to d3/d68/d7f/d11b/f15c 0 2024-04-24T08:25:18.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/984: chown d3/d5/d10e/c151 705833 1 2024-04-24T08:25:18.182 INFO:tasks.workunit.client.0.smithi090.stdout:4/934: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f107 [0,4194304] 0 2024-04-24T08:25:18.188 INFO:tasks.workunit.client.0.smithi090.stdout:1/985: symlink d3/d5/da/d94/da6/d115/l15d 0 2024-04-24T08:25:18.191 INFO:tasks.workunit.client.0.smithi090.stdout:1/986: chown d3/d5/da/dc/lfa 45 1 2024-04-24T08:25:18.199 INFO:tasks.workunit.client.0.smithi090.stdout:1/987: write d3/d1d/d34/d73/dd8/ff8 [2634755,122384] 0 2024-04-24T08:25:18.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/988: fsync d3/d1d/d34/d5a/d143/f124 0 2024-04-24T08:25:18.210 INFO:tasks.workunit.client.0.smithi090.stdout:1/989: unlink d3/d5/da/d97/d9c/fdd 0 2024-04-24T08:25:18.214 INFO:tasks.workunit.client.0.smithi090.stdout:1/990: chown d3/d1d/d34/d5a/f5e 38563479 1 2024-04-24T08:25:18.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/991: dread - d3/d5/d58/fe5 zero size 2024-04-24T08:25:18.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/992: write d3/d5/d10e/f12d [234817,8566] 0 2024-04-24T08:25:18.228 INFO:tasks.workunit.client.0.smithi090.stdout:4/935: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f82 [1043850,70822] 0 2024-04-24T08:25:18.229 INFO:tasks.workunit.client.0.smithi090.stdout:1/993: truncate d3/d1d/d54/fc2 718877 0 2024-04-24T08:25:18.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/994: write d3/d1d/d54/f153 [876497,129160] 0 2024-04-24T08:25:18.236 INFO:tasks.workunit.client.0.smithi090.stdout:1/995: mknod d3/d5/da/d97/d9c/c15e 0 2024-04-24T08:25:18.238 INFO:tasks.workunit.client.0.smithi090.stdout:4/936: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/f12a x:0 0 0 2024-04-24T08:25:18.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/996: mkdir d3/d5/da/dc/d80/deb/d111/d15f 0 2024-04-24T08:25:18.245 INFO:tasks.workunit.client.0.smithi090.stdout:1/997: mknod d3/d5/da/d97/c160 0 2024-04-24T08:25:18.256 INFO:tasks.workunit.client.0.smithi090.stdout:4/937: dread d3/d87/da6/fca [0,4194304] 0 2024-04-24T08:25:18.261 INFO:tasks.workunit.client.0.smithi090.stdout:4/938: fdatasync d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f83 0 2024-04-24T08:25:18.261 INFO:tasks.workunit.client.0.smithi090.stdout:1/998: dwrite d3/d5/da/dc/df4/f131 [0,4194304] 0 2024-04-24T08:25:18.269 INFO:tasks.workunit.client.0.smithi090.stdout:4/939: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/f12b x:0 0 0 2024-04-24T08:25:18.272 INFO:tasks.workunit.client.0.smithi090.stdout:4/940: chown d3/d8e/ddf/d5c/db5/d69/d8b/db2/f7e 24437093 1 2024-04-24T08:25:18.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/941: dread - d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/f12a zero size 2024-04-24T08:25:18.276 INFO:tasks.workunit.client.0.smithi090.stdout:0/990: sync 2024-04-24T08:25:18.281 INFO:tasks.workunit.client.0.smithi090.stdout:0/991: unlink d1a/d4b/d53/d79/d35/c48 0 2024-04-24T08:25:18.283 INFO:tasks.workunit.client.0.smithi090.stdout:4/942: write d3/d8e/fed [1627021,85868] 0 2024-04-24T08:25:18.288 INFO:tasks.workunit.client.0.smithi090.stdout:1/999: dread d3/d5/da/dc/f87 [0,4194304] 0 2024-04-24T08:25:18.293 INFO:tasks.workunit.client.0.smithi090.stdout:9/998: sync 2024-04-24T08:25:18.293 INFO:tasks.workunit.client.0.smithi090.stdout:8/971: sync 2024-04-24T08:25:18.294 INFO:tasks.workunit.client.0.smithi090.stdout:4/943: write d3/f13 [2667438,125177] 0 2024-04-24T08:25:18.296 INFO:tasks.workunit.client.0.smithi090.stdout:9/999: fdatasync d0/fd0 0 2024-04-24T08:25:18.301 INFO:tasks.workunit.client.0.smithi090.stdout:0/992: dwrite d1a/d4b/d53/d79/d69/d90/dbc/ff1 [0,4194304] 0 2024-04-24T08:25:18.306 INFO:tasks.workunit.client.0.smithi090.stdout:4/944: mkdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/d114/d12c 0 2024-04-24T08:25:18.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/972: dread d2/d62/d71/fc9 [0,4194304] 0 2024-04-24T08:25:18.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/973: mkdir d2/da/d13/d34/d14d 0 2024-04-24T08:25:18.320 INFO:tasks.workunit.client.0.smithi090.stdout:0/993: dwrite d1a/d4b/d53/d79/d69/d90/dbc/f103 [4194304,4194304] 0 2024-04-24T08:25:18.321 INFO:tasks.workunit.client.0.smithi090.stdout:8/974: truncate d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e/f12f 1153957 0 2024-04-24T08:25:18.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/945: dread d3/d8e/ddf/d5c/db5/f84 [0,4194304] 0 2024-04-24T08:25:18.322 INFO:tasks.workunit.client.0.smithi090.stdout:0/994: read - d1a/d11d/f140 zero size 2024-04-24T08:25:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:8/975: write d2/d62/d6b/d9c/f120 [4692699,13258] 0 2024-04-24T08:25:18.340 INFO:tasks.workunit.client.0.smithi090.stdout:0/995: getdents d1a/d4b/d53 0 2024-04-24T08:25:18.344 INFO:tasks.workunit.client.0.smithi090.stdout:8/976: dwrite d2/da/d13/f5c [0,4194304] 0 2024-04-24T08:25:18.345 INFO:tasks.workunit.client.0.smithi090.stdout:4/946: dread d3/f32 [0,4194304] 0 2024-04-24T08:25:18.347 INFO:tasks.workunit.client.0.smithi090.stdout:8/977: write d2/da/d13/d64/d50/f9f [1826767,39259] 0 2024-04-24T08:25:18.349 INFO:tasks.workunit.client.0.smithi090.stdout:4/947: rmdir d3/dd9 39 2024-04-24T08:25:18.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/948: fsync d3/d87/f11c 0 2024-04-24T08:25:18.355 INFO:tasks.workunit.client.0.smithi090.stdout:8/978: rmdir d2/d62/d83 39 2024-04-24T08:25:18.359 INFO:tasks.workunit.client.0.smithi090.stdout:4/949: unlink d3/fd1 0 2024-04-24T08:25:18.363 INFO:tasks.workunit.client.0.smithi090.stdout:8/979: mkdir d2/da/d13/d60/d91/dfb/d14e 0 2024-04-24T08:25:18.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/950: mkdir d3/d8e/ddf/d5c/db5/d7c/d12d 0 2024-04-24T08:25:18.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/951: fsync d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/d6f/f107 0 2024-04-24T08:25:18.369 INFO:tasks.workunit.client.0.smithi090.stdout:8/980: dread d2/da/d13/d34/fc3 [0,4194304] 0 2024-04-24T08:25:18.370 INFO:tasks.workunit.client.0.smithi090.stdout:0/996: truncate d1a/d4b/d53/d79/d69/d90/dbc/f103 6790416 0 2024-04-24T08:25:18.372 INFO:tasks.workunit.client.0.smithi090.stdout:8/981: truncate d2/d62/d71/f123 564929 0 2024-04-24T08:25:18.374 INFO:tasks.workunit.client.0.smithi090.stdout:0/997: mkdir d1a/d4b/d53/d79/d58/db4/d14a 0 2024-04-24T08:25:18.376 INFO:tasks.workunit.client.0.smithi090.stdout:0/998: readlink d1a/d4b/d53/d79/d35/dbd/l106 0 2024-04-24T08:25:18.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/982: unlink d2/da/d13/d60/d91/l122 0 2024-04-24T08:25:18.382 INFO:tasks.workunit.client.0.smithi090.stdout:8/983: mkdir d2/d62/d6b/d9c/d14f 0 2024-04-24T08:25:18.382 INFO:tasks.workunit.client.0.smithi090.stdout:4/952: dwrite d3/d8e/ddf/d5c/db5/f9a [0,4194304] 0 2024-04-24T08:25:18.387 INFO:tasks.workunit.client.0.smithi090.stdout:8/984: dread - d2/d62/fe4 zero size 2024-04-24T08:25:18.390 INFO:tasks.workunit.client.0.smithi090.stdout:4/953: mkdir d3/d87/da6/d12e 0 2024-04-24T08:25:18.394 INFO:tasks.workunit.client.0.smithi090.stdout:4/954: fdatasync d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/fde 0 2024-04-24T08:25:18.397 INFO:tasks.workunit.client.0.smithi090.stdout:4/955: mkdir d3/d8e/ddf/d1d/d12f 0 2024-04-24T08:25:18.399 INFO:tasks.workunit.client.0.smithi090.stdout:0/999: dwrite d1a/d4b/d53/d79/d58/f93 [0,4194304] 0 2024-04-24T08:25:18.414 INFO:tasks.workunit.client.0.smithi090.stdout:8/985: dwrite d2/d62/d6b/fba [0,4194304] 0 2024-04-24T08:25:18.421 INFO:tasks.workunit.client.0.smithi090.stdout:8/986: mkdir d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e/d150 0 2024-04-24T08:25:18.424 INFO:tasks.workunit.client.0.smithi090.stdout:4/956: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f6d [4194304,4194304] 0 2024-04-24T08:25:18.448 INFO:tasks.workunit.client.0.smithi090.stdout:4/957: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f93 [0,4194304] 0 2024-04-24T08:25:18.448 INFO:tasks.workunit.client.0.smithi090.stdout:8/987: write d2/da/d13/d64/d24/f2f [4958853,114031] 0 2024-04-24T08:25:18.451 INFO:tasks.workunit.client.0.smithi090.stdout:4/958: chown d3/d8e/ddf/d5c/f74 257613 1 2024-04-24T08:25:18.512 INFO:tasks.workunit.client.0.smithi090.stdout:4/959: getdents d3/d8e/ddf 0 2024-04-24T08:25:18.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/960: fdatasync d3/d8e/ddf/d5c/db5/db6/f10e 0 2024-04-24T08:25:18.520 INFO:tasks.workunit.client.0.smithi090.stdout:4/961: mkdir d3/d8e/ddf/d1d/dcb/d130 0 2024-04-24T08:25:18.524 INFO:tasks.workunit.client.0.smithi090.stdout:4/962: read - d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/fad zero size 2024-04-24T08:25:18.597 INFO:tasks.workunit.client.0.smithi090.stdout:8/988: dwrite d2/da/d13/d64/d24/f92 [0,4194304] 0 2024-04-24T08:25:18.605 INFO:tasks.workunit.client.0.smithi090.stdout:8/989: mknod d2/d97/def/c151 0 2024-04-24T08:25:18.614 INFO:tasks.workunit.client.0.smithi090.stdout:8/990: creat d2/da/d13/d64/d24/db3/db4/dcd/de8/d12e/f152 x:0 0 0 2024-04-24T08:25:18.623 INFO:tasks.workunit.client.0.smithi090.stdout:8/991: creat d2/da/d13/d64/d24/db3/f153 x:0 0 0 2024-04-24T08:25:18.627 INFO:tasks.workunit.client.0.smithi090.stdout:8/992: dread - d2/da/d13/d64/d24/db3/db4/dcd/de8/f12b zero size 2024-04-24T08:25:18.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/993: creat d2/da/d13/d64/db2/db6/dd0/f154 x:0 0 0 2024-04-24T08:25:18.640 INFO:tasks.workunit.client.0.smithi090.stdout:8/994: chown d2/da/d13/d64/db2/db6/cf4 350 1 2024-04-24T08:25:18.646 INFO:tasks.workunit.client.0.smithi090.stdout:8/995: write d2/da/d13/d64/d50/f10b [924092,75063] 0 2024-04-24T08:25:18.655 INFO:tasks.workunit.client.0.smithi090.stdout:8/996: symlink d2/d62/d6b/d9c/d14f/l155 0 2024-04-24T08:25:18.695 INFO:tasks.workunit.client.0.smithi090.stdout:4/963: sync 2024-04-24T08:25:18.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/997: dwrite d2/d62/d83/da3/deb/fec [0,4194304] 0 2024-04-24T08:25:18.704 INFO:tasks.workunit.client.0.smithi090.stdout:4/964: write d3/d8e/ddf/d5c/db5/d7c/f100 [766838,77473] 0 2024-04-24T08:25:18.713 INFO:tasks.workunit.client.0.smithi090.stdout:8/998: mknod d2/da/d13/d64/d24/db3/db4/de0/c156 0 2024-04-24T08:25:18.719 INFO:tasks.workunit.client.0.smithi090.stdout:4/965: symlink d3/d8e/ddf/d5c/db5/d69/d8b/l131 0 2024-04-24T08:25:18.724 INFO:tasks.workunit.client.0.smithi090.stdout:8/999: fdatasync d2/da/d13/d64/d24/dad/ff6 0 2024-04-24T08:25:18.729 INFO:tasks.workunit.client.0.smithi090.stdout:4/966: symlink d3/d8e/ddf/d109/d127/l132 0 2024-04-24T08:25:18.743 INFO:tasks.workunit.client.0.smithi090.stdout:4/967: rename d3/d8e/ddf/d5c/db5/d69/d8b/db2/c3c to d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/d114/d12c/c133 0 2024-04-24T08:25:18.747 INFO:tasks.workunit.client.0.smithi090.stdout:4/968: chown d3/d24/cc6 14011860 1 2024-04-24T08:25:18.758 INFO:tasks.workunit.client.0.smithi090.stdout:4/969: read d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/f108 [2671039,97548] 0 2024-04-24T08:25:18.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/970: dwrite d3/d8e/ddf/d5c/db5/d69/d8b/db2/f118 [0,4194304] 0 2024-04-24T08:25:18.806 INFO:tasks.workunit.client.0.smithi090.stdout:4/971: stat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d10b/f12a 0 2024-04-24T08:25:18.814 INFO:tasks.workunit.client.0.smithi090.stdout:4/972: mknod d3/d8e/ddf/c134 0 2024-04-24T08:25:18.833 INFO:tasks.workunit.client.0.smithi090.stdout:4/973: fsync d3/d8e/ddf/d5c/db5/d69/fe7 0 2024-04-24T08:25:18.844 INFO:tasks.workunit.client.0.smithi090.stdout:4/974: write d3/d8e/ddf/d5c/db5/d69/d8b/f116 [465285,12277] 0 2024-04-24T08:25:18.854 INFO:tasks.workunit.client.0.smithi090.stdout:4/975: symlink d3/d8e/ddf/d5c/db5/d7c/d12d/l135 0 2024-04-24T08:25:18.865 INFO:tasks.workunit.client.0.smithi090.stdout:4/976: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/d114/d12c/f136 x:0 0 0 2024-04-24T08:25:18.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/977: creat d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f137 x:0 0 0 2024-04-24T08:25:18.889 INFO:tasks.workunit.client.0.smithi090.stdout:4/978: creat d3/d8e/ddf/d5c/db5/d69/f138 x:0 0 0 2024-04-24T08:25:18.891 INFO:tasks.workunit.client.0.smithi090.stdout:4/979: readlink d3/dd0/lee 0 2024-04-24T08:25:18.897 INFO:tasks.workunit.client.0.smithi090.stdout:4/980: chown d3/d8e/ddf/d5c/db5/d77/fa7 720 1 2024-04-24T08:25:18.902 INFO:tasks.workunit.client.0.smithi090.stdout:4/981: read - d3/d8e/ddf/d5c/db5/d69/d8b/fbd zero size 2024-04-24T08:25:18.912 INFO:tasks.workunit.client.0.smithi090.stdout:4/982: symlink d3/d8e/ddf/d5c/db5/d7c/l139 0 2024-04-24T08:25:18.918 INFO:tasks.workunit.client.0.smithi090.stdout:4/983: mkdir d3/d8e/ddf/d5c/db5/d13a 0 2024-04-24T08:25:18.933 INFO:tasks.workunit.client.0.smithi090.stdout:4/984: link d3/c21 d3/d8e/ddf/d109/c13b 0 2024-04-24T08:25:18.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/985: write d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f137 [125048,60388] 0 2024-04-24T08:25:18.951 INFO:tasks.workunit.client.0.smithi090.stdout:4/986: getdents d3/d8e/ddf/d5c/db5/db6 0 2024-04-24T08:25:18.959 INFO:tasks.workunit.client.0.smithi090.stdout:4/987: unlink d3/d8e/ddf/d5c/db5/d69/d8b/l131 0 2024-04-24T08:25:18.962 INFO:tasks.workunit.client.0.smithi090.stdout:4/988: readlink d3/d8e/ddf/d109/l11d 0 2024-04-24T08:25:18.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/989: getdents d3/d8e 0 2024-04-24T08:25:18.993 INFO:tasks.workunit.client.0.smithi090.stdout:4/990: read d3/d8e/f97 [3602944,61939] 0 2024-04-24T08:25:19.002 INFO:tasks.workunit.client.0.smithi090.stdout:4/991: rename d3/d105 to d3/d8e/ddf/d5c/db5/d7c/d13c 0 2024-04-24T08:25:19.047 INFO:tasks.workunit.client.0.smithi090.stdout:4/992: dread d3/d8e/ddf/d1d/f1f [0,4194304] 0 2024-04-24T08:25:19.054 INFO:tasks.workunit.client.0.smithi090.stdout:4/993: mknod d3/d24/c13d 0 2024-04-24T08:25:19.094 INFO:tasks.workunit.client.0.smithi090.stdout:4/994: dwrite d3/d8e/ddf/d1d/f40 [0,4194304] 0 2024-04-24T08:25:19.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/995: mkdir d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/ddb/d13e 0 2024-04-24T08:25:19.112 INFO:tasks.workunit.client.0.smithi090.stdout:4/996: write d3/d8e/ddf/d5c/db5/db6/f10e [197492,35890] 0 2024-04-24T08:25:19.125 INFO:tasks.workunit.client.0.smithi090.stdout:4/997: unlink d3/d8e/ddf/d5c/db5/d69/d8b/db2/d49/f4f 0 2024-04-24T08:25:19.130 INFO:tasks.workunit.client.0.smithi090.stdout:4/998: fsync d3/d8e/ddf/d5c/db5/d69/d8b/db2/d6b/f93 0 2024-04-24T08:25:19.148 INFO:tasks.workunit.client.0.smithi090.stdout:4/999: mknod d3/d8e/ddf/d1d/dcb/d130/c13f 0 2024-04-24T08:25:19.164 INFO:tasks.workunit.client.0.smithi090.stderr:+ rm -rf -- ./tmp.hsCCAW66Ry 2024-04-24T08:25:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:19 smithi149 ceph-mon[31282]: pgmap v32: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 83 MiB/s rd, 206 MiB/s wr, 469 op/s 2024-04-24T08:25:19.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:19 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:19.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:19 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:19.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:19 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:19 smithi090 ceph-mon[25016]: pgmap v32: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 83 MiB/s rd, 206 MiB/s wr, 469 op/s 2024-04-24T08:25:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:19 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:19 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:19 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:21.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:21 smithi090 ceph-mon[25016]: pgmap v33: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 153 MiB/s wr, 349 op/s 2024-04-24T08:25:21.710 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:21 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:21 smithi149 ceph-mon[31282]: pgmap v33: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 153 MiB/s wr, 349 op/s 2024-04-24T08:25:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:21 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:23.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:22 smithi149 ceph-mon[31282]: pgmap v34: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 74 MiB/s rd, 177 MiB/s wr, 442 op/s 2024-04-24T08:25:23.088 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:22 smithi090 ceph-mon[25016]: pgmap v34: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 74 MiB/s rd, 177 MiB/s wr, 442 op/s 2024-04-24T08:25:25.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:25 smithi149 ceph-mon[31282]: pgmap v35: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 98 MiB/s wr, 262 op/s 2024-04-24T08:25:25.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:25 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:25.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:25 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:25.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:25 smithi090 ceph-mon[25016]: pgmap v35: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 98 MiB/s wr, 262 op/s 2024-04-24T08:25:25.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:25 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:25.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:25 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.710 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:27 smithi149 ceph-mon[31282]: pgmap v36: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 41 MiB/s rd, 99 MiB/s wr, 301 op/s 2024-04-24T08:25:27.710 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:27 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.711 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:27 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.711 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:27 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.711 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:27 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:27 smithi090 ceph-mon[25016]: pgmap v36: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 41 MiB/s rd, 99 MiB/s wr, 301 op/s 2024-04-24T08:25:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:27 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:27 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:27 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:27.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:27 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 -- 172.21.15.90:0/3109113279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 msgr2=0x7f8c4c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 --2- 172.21.15.90:0/3109113279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c105790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009910 tx=0x7f8c34009c20 comp rx=0 tx=0).stop 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 -- 172.21.15.90:0/3109113279 shutdown_connections 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 --2- 172.21.15.90:0/3109113279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c105790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 --2- 172.21.15.90:0/3109113279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c069270 0x7f8c4c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 -- 172.21.15.90:0/3109113279 >> 172.21.15.90:0/3109113279 conn(0x7f8c4c0fa780 msgr2=0x7f8c4c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:28.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 -- 172.21.15.90:0/3109113279 shutdown_connections 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.253+0000 7f8c51c00700 1 -- 172.21.15.90:0/3109113279 wait complete. 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 Processor -- start 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 -- start start 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c196010 0x7f8c4c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c196510 con 0x7f8c4c103310 2024-04-24T08:25:28.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.254+0000 7f8c51c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c196680 con 0x7f8c4c196010 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55874/0 (socket says 172.21.15.90:55874) 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 -- 172.21.15.90:0/780820796 learned_addr learned my addr 172.21.15.90:0/780820796 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c43e00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c196010 0x7f8c4c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 -- 172.21.15.90:0/780820796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c196010 msgr2=0x7f8c4c19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c196010 0x7f8c4c19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.255+0000 7f8c4aa00700 1 -- 172.21.15.90:0/780820796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340095f0 con 0x7f8c4c103310 2024-04-24T08:25:28.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c4aa00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e8f0 tx=0x7f8c3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:28.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c009790 con 0x7f8c4c103310 2024-04-24T08:25:28.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c004ce0 con 0x7f8c4c103310 2024-04-24T08:25:28.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c19b620 con 0x7f8c4c103310 2024-04-24T08:25:28.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c005690 con 0x7f8c4c103310 2024-04-24T08:25:28.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.256+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c19bb40 con 0x7f8c4c103310 2024-04-24T08:25:28.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.257+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c18fbf0 con 0x7f8c4c103310 2024-04-24T08:25:28.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.258+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f8c3c004750 con 0x7f8c4c103310 2024-04-24T08:25:28.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.259+0000 7f8c43400700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 0x7f8c30070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.260+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8c3c014070 con 0x7f8c4c103310 2024-04-24T08:25:28.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.260+0000 7f8c43e00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 0x7f8c30070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.261+0000 7f8c43e00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 0x7f8c30070f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009fd0 tx=0x7f8c34005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:28.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.264+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f8c3c05bb10 con 0x7f8c4c103310 2024-04-24T08:25:28.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.539+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c4c02ced0 con 0x7f8c3006ea70 2024-04-24T08:25:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.542+0000 7f8c43400700 1 -- 172.21.15.90:0/780820796 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f8c4c02ced0 con 0x7f8c3006ea70 2024-04-24T08:25:28.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 msgr2=0x7f8c30070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 0x7f8c30070f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009fd0 tx=0x7f8c34005fd0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 msgr2=0x7f8c4c195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e8f0 tx=0x7f8c3c00ecb0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 shutdown_connections 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8c3006ea70 0x7f8c30070f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4c103310 0x7f8c4c195ad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 --2- 172.21.15.90:0/780820796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c4c196010 0x7f8c4c19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.545+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 >> 172.21.15.90:0/780820796 conn(0x7f8c4c0fa780 msgr2=0x7f8c4c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.546+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 shutdown_connections 2024-04-24T08:25:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.546+0000 7f8c51c00700 1 -- 172.21.15.90:0/780820796 wait complete. 2024-04-24T08:25:28.564 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 -- 172.21.15.90:0/1312960268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 msgr2=0x7f5144073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 --2- 172.21.15.90:0/1312960268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f5144073180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5138009910 tx=0x7f5138009c20 comp rx=0 tx=0).stop 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 -- 172.21.15.90:0/1312960268 shutdown_connections 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 --2- 172.21.15.90:0/1312960268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 0x7f5144073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 --2- 172.21.15.90:0/1312960268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f5144073180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.712+0000 7f514b200700 1 -- 172.21.15.90:0/1312960268 >> 172.21.15.90:0/1312960268 conn(0x7f51440fb860 msgr2=0x7f51440fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.713+0000 7f514b200700 1 -- 172.21.15.90:0/1312960268 shutdown_connections 2024-04-24T08:25:28.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.713+0000 7f514b200700 1 -- 172.21.15.90:0/1312960268 wait complete. 2024-04-24T08:25:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.714+0000 7f514b200700 1 Processor -- start 2024-04-24T08:25:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.714+0000 7f514b200700 1 -- start start 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.714+0000 7f514b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 0x7f5144197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f514b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f514b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441987d0 con 0x7f5144074d20 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f514b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514419d1e0 con 0x7f5144073750 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 0x7f5144197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55906/0 (socket says 172.21.15.90:55906) 2024-04-24T08:25:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 -- 172.21.15.90:0/3812565063 learned_addr learned my addr 172.21.15.90:0/3812565063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:25:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 -- 172.21.15.90:0/3812565063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 msgr2=0x7f5144197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 0x7f5144197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.715+0000 7f5149400700 1 -- 172.21.15.90:0/3812565063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380095f0 con 0x7f5144074d20 2024-04-24T08:25:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f5149400700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f513400d8d0 tx=0x7f513400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51340098a0 con 0x7f5144074d20 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5134010460 con 0x7f5144074d20 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513400f5f0 con 0x7f5144074d20 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514419d3e0 con 0x7f5144074d20 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.716+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514419d8b0 con 0x7f5144074d20 2024-04-24T08:25:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.717+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514406a550 con 0x7f5144074d20 2024-04-24T08:25:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.719+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f5134009a00 con 0x7f5144074d20 2024-04-24T08:25:28.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.719+0000 7f513ea00700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 0x7f512c070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:28.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.720+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f513408eab0 con 0x7f5144074d20 2024-04-24T08:25:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.720+0000 7f5149e00700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 0x7f512c070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.720+0000 7f5149e00700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 0x7f512c070f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5138006010 tx=0x7f513800b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.724+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f5134090e60 con 0x7f5144074d20 2024-04-24T08:25:28.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.979+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51441077f0 con 0x7f512c06ea70 2024-04-24T08:25:28.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.981+0000 7f513ea00700 1 -- 172.21.15.90:0/3812565063 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f51441077f0 con 0x7f512c06ea70 2024-04-24T08:25:28.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 msgr2=0x7f512c070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 0x7f512c070f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5138006010 tx=0x7f513800b560 comp rx=0 tx=0).stop 2024-04-24T08:25:28.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 msgr2=0x7f51441981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:28.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f513400d8d0 tx=0x7f513400dbe0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 shutdown_connections 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f512c06ea70 0x7f512c070f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144073750 0x7f5144197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 --2- 172.21.15.90:0/3812565063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144074d20 0x7f51441981b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 >> 172.21.15.90:0/3812565063 conn(0x7f51440fb860 msgr2=0x7f51441060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 shutdown_connections 2024-04-24T08:25:28.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:28.984+0000 7f514b200700 1 -- 172.21.15.90:0/3812565063 wait complete. 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 -- 172.21.15.90:0/1520709164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134100d60 msgr2=0x7f7134103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 --2- 172.21.15.90:0/1520709164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134100d60 0x7f7134103140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7128009910 tx=0x7f7128009c20 comp rx=0 tx=0).stop 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 -- 172.21.15.90:0/1520709164 shutdown_connections 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 --2- 172.21.15.90:0/1520709164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134103680 0x7f7134105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 --2- 172.21.15.90:0/1520709164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134100d60 0x7f7134103140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 -- 172.21.15.90:0/1520709164 >> 172.21.15.90:0/1520709164 conn(0x7f71340fa780 msgr2=0x7f71340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.165+0000 7f713bc00700 1 -- 172.21.15.90:0/1520709164 shutdown_connections 2024-04-24T08:25:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.166+0000 7f713bc00700 1 -- 172.21.15.90:0/1520709164 wait complete. 2024-04-24T08:25:29.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.167+0000 7f713bc00700 1 Processor -- start 2024-04-24T08:25:29.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.167+0000 7f713bc00700 1 -- start start 2024-04-24T08:25:29.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f713bc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 0x7f7134191d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f713bc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f713bc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713418e880 con 0x7f7134103680 2024-04-24T08:25:29.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f713bc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713418e9f0 con 0x7f7134100d60 2024-04-24T08:25:29.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f7133e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f7133e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55914/0 (socket says 172.21.15.90:55914) 2024-04-24T08:25:29.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.168+0000 7f7133e00700 1 -- 172.21.15.90:0/3183169311 learned_addr learned my addr 172.21.15.90:0/3183169311 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:25:29.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.169+0000 7f7133e00700 1 -- 172.21.15.90:0/3183169311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 msgr2=0x7f7134191d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.169+0000 7f7139200700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 0x7f7134191d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.170+0000 7f7133e00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 0x7f7134191d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.170+0000 7f7133e00700 1 -- 172.21.15.90:0/3183169311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280095f0 con 0x7f7134103680 2024-04-24T08:25:29.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.170+0000 7f7139200700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 0x7f7134191d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:25:29.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.170+0000 7f7133e00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f712000e920 tx=0x7f712000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:29.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.171+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7120009790 con 0x7f7134103680 2024-04-24T08:25:29.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.171+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f712000cba0 con 0x7f7134103680 2024-04-24T08:25:29.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.171+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7120018630 con 0x7f7134103680 2024-04-24T08:25:29.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.171+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713418ecd0 con 0x7f7134103680 2024-04-24T08:25:29.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.172+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713418f220 con 0x7f7134103680 2024-04-24T08:25:29.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.174+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114005320 con 0x7f7134103680 2024-04-24T08:25:29.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.177+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f712000cd10 con 0x7f7134103680 2024-04-24T08:25:29.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.178+0000 7f7131600700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 0x7f711c070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.178+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7120014070 con 0x7f7134103680 2024-04-24T08:25:29.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.178+0000 7f7139200700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 0x7f711c070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.179+0000 7f7139200700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 0x7f711c070f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7128009fd0 tx=0x7f7128005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:29.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.186+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f7120053ca0 con 0x7f7134103680 2024-04-24T08:25:29.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.471+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7114000bf0 con 0x7f711c06eac0 2024-04-24T08:25:29.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.484+0000 7f7131600700 1 -- 172.21.15.90:0/3183169311 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7114000bf0 con 0x7f711c06eac0 2024-04-24T08:25:29.485 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:25:29.485 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (10m) 2s ago 12m 14.3M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (12m) 2s ago 12m 7918k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (11m) 5s ago 11m 7843k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (12m) 2s ago 12m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (11m) 5s ago 11m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (10m) 2s ago 12m 75.9M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (5m) 2s ago 5m 270M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (5m) 2s ago 5m 14.4M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (5m) 5s ago 5m 253M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (5m) 5s ago 5m 14.4M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (86s) 2s ago 14m 494M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (10s) 5s ago 11m 56.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (14m) 2s ago 14m 53.8M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:25:29.486 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (10m) 5s ago 10m 44.0M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (12m) 2s ago 12m 8112k - 1.5.0 0da6a335fe13 3dd5ba987ff4 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (11m) 5s ago 11m 7810k - 1.5.0 0da6a335fe13 49a0d163d479 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 2s ago 10m 210M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (9m) 2s ago 9m 208M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (8m) 2s ago 8m 169M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (8m) 5s ago 8m 227M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (7m) 5s ago 7m 193M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (7m) 5s ago 7m 212M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (27s) 2s ago 11m 53.0M - 2.43.0 a07b618ecd1d 07acd8b5fe0d 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 msgr2=0x7f711c070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 0x7f711c070f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7128009fd0 tx=0x7f7128005fb0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 msgr2=0x7f71341922d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f712000e920 tx=0x7f712000ece0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 shutdown_connections 2024-04-24T08:25:29.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f711c06eac0 0x7f711c070f70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7134100d60 0x7f7134191d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 --2- 172.21.15.90:0/3183169311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134103680 0x7f71341922d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 >> 172.21.15.90:0/3183169311 conn(0x7f71340fa780 msgr2=0x7f71340ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:29.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 shutdown_connections 2024-04-24T08:25:29.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.487+0000 7f713bc00700 1 -- 172.21.15.90:0/3183169311 wait complete. 2024-04-24T08:25:29.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.647+0000 7f32b2000700 1 -- 172.21.15.90:0/2919735622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 msgr2=0x7f32ac105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.647+0000 7f32b2000700 1 --2- 172.21.15.90:0/2919735622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac105890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3298009910 tx=0x7f3298009c20 comp rx=0 tx=0).stop 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 -- 172.21.15.90:0/2919735622 shutdown_connections 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 --2- 172.21.15.90:0/2919735622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac105890 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 --2- 172.21.15.90:0/2919735622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 -- 172.21.15.90:0/2919735622 >> 172.21.15.90:0/2919735622 conn(0x7f32ac0fa5d0 msgr2=0x7f32ac0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 -- 172.21.15.90:0/2919735622 shutdown_connections 2024-04-24T08:25:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.648+0000 7f32b2000700 1 -- 172.21.15.90:0/2919735622 wait complete. 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.649+0000 7f32b2000700 1 Processor -- start 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.649+0000 7f32b2000700 1 -- start start 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32b2000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac1958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32b2000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac196410 con 0x7f32ac1034b0 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32b2000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac196550 con 0x7f32ac100b90 2024-04-24T08:25:29.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32abe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32abe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55934/0 (socket says 172.21.15.90:55934) 2024-04-24T08:25:29.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32abe00700 1 -- 172.21.15.90:0/1733344078 learned_addr learned my addr 172.21.15.90:0/1733344078 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:25:29.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.650+0000 7f32b0c00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac1958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32abe00700 1 -- 172.21.15.90:0/1733344078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 msgr2=0x7f32ac1958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32abe00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac1958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32abe00700 1 -- 172.21.15.90:0/1733344078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980095f0 con 0x7f32ac1034b0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32b0c00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32abe00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3298000c00 tx=0x7f329800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.651+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329801d070 con 0x7f32ac1034b0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.652+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3298005dc0 con 0x7f32ac1034b0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.652+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac19afa0 con 0x7f32ac1034b0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.652+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac19b460 con 0x7f32ac1034b0 2024-04-24T08:25:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.652+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32980226a0 con 0x7f32ac1034b0 2024-04-24T08:25:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.653+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac18fa10 con 0x7f32ac1034b0 2024-04-24T08:25:29.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.656+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f32980038f0 con 0x7f32ac1034b0 2024-04-24T08:25:29.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.657+0000 7f32a9600700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 0x7f3294070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:29.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.657+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f32980901c0 con 0x7f32ac1034b0 2024-04-24T08:25:29.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.657+0000 7f32b0c00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 0x7f3294070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:29.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.658+0000 7f32b0c00700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 0x7f3294070f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32ac069890 tx=0x7f32a0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:29.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:29.663+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f329805ccc0 con 0x7f32ac1034b0 2024-04-24T08:25:29.753 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:29 smithi149 ceph-mon[31282]: pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 29 MiB/s rd, 72 MiB/s wr, 241 op/s 2024-04-24T08:25:29.753 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:29 smithi149 ceph-mon[31282]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:29 smithi090 ceph-mon[25016]: pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 29 MiB/s rd, 72 MiB/s wr, 241 op/s 2024-04-24T08:25:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:29 smithi090 ceph-mon[25016]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.025+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32ac02ce80 con 0x7f32ac1034b0 2024-04-24T08:25:30.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.026+0000 7f32a9600700 1 -- 172.21.15.90:0/1733344078 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f3298003680 con 0x7f32ac1034b0 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:25:30.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:25:30.029 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:25:30.029 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:25:30.029 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:25:30.029 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:25:30.029 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:25:30.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.030+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 msgr2=0x7f3294070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.030+0000 7f32b2000700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 0x7f3294070f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32ac069890 tx=0x7f32a0009260 comp rx=0 tx=0).stop 2024-04-24T08:25:30.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.030+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 msgr2=0x7f32ac195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.030+0000 7f32b2000700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3298000c00 tx=0x7f329800f690 comp rx=0 tx=0).stop 2024-04-24T08:25:30.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 shutdown_connections 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f329406ea70 0x7f3294070f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32ac100b90 0x7f32ac1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 --2- 172.21.15.90:0/1733344078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32ac1034b0 0x7f32ac195df0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 >> 172.21.15.90:0/1733344078 conn(0x7f32ac0fa5d0 msgr2=0x7f32ac0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 shutdown_connections 2024-04-24T08:25:30.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.031+0000 7f32b2000700 1 -- 172.21.15.90:0/1733344078 wait complete. 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.211+0000 7ffb46600700 1 -- 172.21.15.90:0/765699679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40103310 msgr2=0x7ffb40105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.211+0000 7ffb46600700 1 --2- 172.21.15.90:0/765699679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40103310 0x7ffb40105790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009910 tx=0x7ffb2c009c20 comp rx=0 tx=0).stop 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.212+0000 7ffb46600700 1 -- 172.21.15.90:0/765699679 shutdown_connections 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.212+0000 7ffb46600700 1 --2- 172.21.15.90:0/765699679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40103310 0x7ffb40105790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.212+0000 7ffb46600700 1 --2- 172.21.15.90:0/765699679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40069270 0x7ffb40102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.212+0000 7ffb46600700 1 -- 172.21.15.90:0/765699679 >> 172.21.15.90:0/765699679 conn(0x7ffb400fa780 msgr2=0x7ffb400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.213+0000 7ffb46600700 1 -- 172.21.15.90:0/765699679 shutdown_connections 2024-04-24T08:25:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.213+0000 7ffb46600700 1 -- 172.21.15.90:0/765699679 wait complete. 2024-04-24T08:25:30.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.215+0000 7ffb46600700 1 Processor -- start 2024-04-24T08:25:30.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.215+0000 7ffb46600700 1 -- start start 2024-04-24T08:25:30.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.216+0000 7ffb46600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:30.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.216+0000 7ffb46600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb4010f240 0x7ffb401122a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:30.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.216+0000 7ffb46600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4010f740 con 0x7ffb4010f240 2024-04-24T08:25:30.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.216+0000 7ffb46600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4010f8b0 con 0x7ffb40103310 2024-04-24T08:25:30.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:30.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:55480/0 (socket says 172.21.15.90:55480) 2024-04-24T08:25:30.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 -- 172.21.15.90:0/1884206256 learned_addr learned my addr 172.21.15.90:0/1884206256 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:25:30.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3ea00700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb4010f240 0x7ffb401122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:30.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 -- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb4010f240 msgr2=0x7ffb401122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb4010f240 0x7ffb401122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.217+0000 7ffb3f400700 1 -- 172.21.15.90:0/1884206256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0095f0 con 0x7ffb40103310 2024-04-24T08:25:30.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.218+0000 7ffb3f400700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d900 tx=0x7ffb3000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:30.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.220+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb30009900 con 0x7ffb40103310 2024-04-24T08:25:30.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.220+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb30010460 con 0x7ffb40103310 2024-04-24T08:25:30.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.220+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40112840 con 0x7ffb40103310 2024-04-24T08:25:30.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.220+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3000f5f0 con 0x7ffb40103310 2024-04-24T08:25:30.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.220+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb40112d60 con 0x7ffb40103310 2024-04-24T08:25:30.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.222+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40069270 con 0x7ffb40103310 2024-04-24T08:25:30.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.227+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 31) v1 ==== 92700+0+0 (secure 0 0 0) 0x7ffb300105d0 con 0x7ffb40103310 2024-04-24T08:25:30.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.229+0000 7ffb37e00700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 0x7ffb24070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:25:30.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.229+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffb3008ea20 con 0x7ffb40103310 2024-04-24T08:25:30.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.230+0000 7ffb3ea00700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 0x7ffb24070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:25:30.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.231+0000 7ffb3ea00700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 0x7ffb24070fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0098e0 tx=0x7ffb2c005c00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:25:30.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.234+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7ffb3005b620 con 0x7ffb40103310 2024-04-24T08:25:30.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:30 smithi149 ceph-mon[31282]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:30.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:30 smithi149 ceph-mon[31282]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:30.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:30 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1733344078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.545+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb40112fd0 con 0x7ffb2406eb10 2024-04-24T08:25:30.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.547+0000 7ffb37e00700 1 -- 172.21.15.90:0/1884206256 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ffb40112fd0 con 0x7ffb2406eb10 2024-04-24T08:25:30.548 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:25:30.548 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:25:30.548 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:25:30.548 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:25:30.549 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:25:30.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 msgr2=0x7ffb24070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 0x7ffb24070fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0098e0 tx=0x7ffb2c005c00 comp rx=0 tx=0).stop 2024-04-24T08:25:30.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 msgr2=0x7ffb4010ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:25:30.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d900 tx=0x7ffb3000dcc0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 shutdown_connections 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffb2406eb10 0x7ffb24070fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb40103310 0x7ffb4010ed00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 --2- 172.21.15.90:0/1884206256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb4010f240 0x7ffb401122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.550+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 >> 172.21.15.90:0/1884206256 conn(0x7ffb400fa780 msgr2=0x7ffb400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.551+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 shutdown_connections 2024-04-24T08:25:30.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:25:30.551+0000 7ffb46600700 1 -- 172.21.15.90:0/1884206256 wait complete. 2024-04-24T08:25:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:30 smithi090 ceph-mon[25016]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:30 smithi090 ceph-mon[25016]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:30 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1733344078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:31.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:31 smithi090 ceph-mon[25016]: pgmap v38: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 9.9 MiB/s rd, 24 MiB/s wr, 149 op/s 2024-04-24T08:25:31.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:31 smithi090 ceph-mon[25016]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:33.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:25:33.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:32 smithi090 ceph-mon[25016]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 10 MiB/s rd, 24 MiB/s wr, 204 op/s 2024-04-24T08:25:33.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:31 smithi149 ceph-mon[31282]: pgmap v38: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 9.9 MiB/s rd, 24 MiB/s wr, 149 op/s 2024-04-24T08:25:33.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:31 smithi149 ceph-mon[31282]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:25:33.767 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/4111923116' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:25:33.768 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:32 smithi149 ceph-mon[31282]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 10 MiB/s rd, 24 MiB/s wr, 204 op/s 2024-04-24T08:25:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:33 smithi149 ceph-mon[31282]: mgrmap e32: smithi090.jsrfxo(active, since 72s), standbys: smithi149.vavftl 2024-04-24T08:25:34.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:33 smithi090 ceph-mon[25016]: mgrmap e32: smithi090.jsrfxo(active, since 72s), standbys: smithi149.vavftl 2024-04-24T08:25:35.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:34 smithi090 ceph-mon[25016]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 929 KiB/s rd, 836 KiB/s wr, 111 op/s 2024-04-24T08:25:35.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:34 smithi149 ceph-mon[31282]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 929 KiB/s rd, 836 KiB/s wr, 111 op/s 2024-04-24T08:25:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:35 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:36.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:35 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:37.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:36 smithi149 ceph-mon[31282]: pgmap v41: 65 pgs: 65 active+clean; 2.3 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 957 KiB/s rd, 917 KiB/s wr, 123 op/s 2024-04-24T08:25:37.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:36 smithi090 ceph-mon[25016]: pgmap v41: 65 pgs: 65 active+clean; 2.3 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 957 KiB/s rd, 917 KiB/s wr, 123 op/s 2024-04-24T08:25:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:39 smithi149 ceph-mon[31282]: pgmap v42: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 830 KiB/s rd, 734 KiB/s wr, 118 op/s 2024-04-24T08:25:39.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:39 smithi090 ceph-mon[25016]: pgmap v42: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 830 KiB/s rd, 734 KiB/s wr, 118 op/s 2024-04-24T08:25:41.094 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T08:25:41.094 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-24T08:25:41.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:41 smithi149 ceph-mon[31282]: pgmap v43: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 681 KiB/s rd, 713 KiB/s wr, 101 op/s 2024-04-24T08:25:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:41 smithi090 ceph-mon[25016]: pgmap v43: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 681 KiB/s rd, 713 KiB/s wr, 101 op/s 2024-04-24T08:25:43.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:42 smithi149 ceph-mon[31282]: pgmap v44: 65 pgs: 65 active+clean; 1.2 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-04-24T08:25:43.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:42 smithi090 ceph-mon[25016]: pgmap v44: 65 pgs: 65 active+clean; 1.2 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.442 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all mgr 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.jsrfxo"}]: dispatch 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.jsrfxo"}]': finished 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.vavftl"}]: dispatch 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.vavftl"}]': finished 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all rgw 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T08:25:44.443 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all iscsi 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all nfs 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all nvmeof 2024-04-24T08:25:44.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all mgr 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.jsrfxo"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.jsrfxo"}]': finished 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.vavftl"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.vavftl"}]': finished 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all rgw 2024-04-24T08:25:44.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all iscsi 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all nfs 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all nvmeof 2024-04-24T08:25:44.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:45 smithi090 ceph-mon[25016]: pgmap v45: 65 pgs: 65 active+clean; 1.2 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 663 KiB/s rd, 695 KiB/s wr, 91 op/s 2024-04-24T08:25:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:45 smithi149 ceph-mon[31282]: pgmap v45: 65 pgs: 65 active+clean; 1.2 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 663 KiB/s rd, 695 KiB/s wr, 91 op/s 2024-04-24T08:25:47.772 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:47 smithi090 ceph-mon[25016]: Upgrade: Updating node-exporter.smithi090 (1/2) 2024-04-24T08:25:47.772 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:47 smithi090 ceph-mon[25016]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-24T08:25:47.772 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:47 smithi090 ceph-mon[25016]: pgmap v46: 65 pgs: 65 active+clean; 957 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 679 KiB/s rd, 718 KiB/s wr, 101 op/s 2024-04-24T08:25:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:47 smithi149 ceph-mon[31282]: Upgrade: Updating node-exporter.smithi090 (1/2) 2024-04-24T08:25:47.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:47 smithi149 ceph-mon[31282]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-24T08:25:47.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:47 smithi149 ceph-mon[31282]: pgmap v46: 65 pgs: 65 active+clean; 957 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 679 KiB/s rd, 718 KiB/s wr, 101 op/s 2024-04-24T08:25:48.239 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T08:25:48.239 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T08:25:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:49 smithi149 ceph-mon[31282]: pgmap v47: 65 pgs: 65 active+clean; 627 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 947 KiB/s rd, 927 KiB/s wr, 118 op/s 2024-04-24T08:25:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:49 smithi090 ceph-mon[25016]: pgmap v47: 65 pgs: 65 active+clean; 627 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 947 KiB/s rd, 927 KiB/s wr, 118 op/s 2024-04-24T08:25:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:51 smithi149 ceph-mon[31282]: pgmap v48: 65 pgs: 65 active+clean; 627 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 686 KiB/s rd, 662 KiB/s wr, 84 op/s 2024-04-24T08:25:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:51 smithi090 ceph-mon[25016]: pgmap v48: 65 pgs: 65 active+clean; 627 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 686 KiB/s rd, 662 KiB/s wr, 84 op/s 2024-04-24T08:25:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:25:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:25:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:52 smithi090 ceph-mon[25016]: pgmap v49: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-24T08:25:53.091 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:52 smithi149 ceph-mon[31282]: pgmap v49: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-24T08:25:54.223 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:53 smithi149 ceph-mon[31282]: mgrmap e33: smithi090.jsrfxo(active, since 92s), standbys: smithi149.vavftl 2024-04-24T08:25:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:53 smithi090 ceph-mon[25016]: mgrmap e33: smithi090.jsrfxo(active, since 92s), standbys: smithi149.vavftl 2024-04-24T08:25:55.243 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:54 smithi090 ceph-mon[25016]: Upgrade: Updating node-exporter.smithi149 (2/2) 2024-04-24T08:25:55.243 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:54 smithi090 ceph-mon[25016]: Deploying daemon node-exporter.smithi149 on smithi149 2024-04-24T08:25:55.243 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:54 smithi090 ceph-mon[25016]: pgmap v50: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 763 KiB/s wr, 82 op/s 2024-04-24T08:25:55.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:54 smithi149 ceph-mon[31282]: Upgrade: Updating node-exporter.smithi149 (2/2) 2024-04-24T08:25:55.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:54 smithi149 ceph-mon[31282]: Deploying daemon node-exporter.smithi149 on smithi149 2024-04-24T08:25:55.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:54 smithi149 ceph-mon[31282]: pgmap v50: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 763 KiB/s wr, 82 op/s 2024-04-24T08:25:57.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:57 smithi149 ceph-mon[31282]: pgmap v51: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 990 KiB/s rd, 999 KiB/s wr, 98 op/s 2024-04-24T08:25:57.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:57 smithi090 ceph-mon[25016]: pgmap v51: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 990 KiB/s rd, 999 KiB/s wr, 98 op/s 2024-04-24T08:25:59.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:25:59 smithi149 ceph-mon[31282]: pgmap v52: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-04-24T08:25:59.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:25:59 smithi090 ceph-mon[25016]: pgmap v52: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 -- 172.21.15.90:0/845009126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250101440 msgr2=0x7ff250101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 --2- 172.21.15.90:0/845009126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250101440 0x7ff250101890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009910 tx=0x7ff23c009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 -- 172.21.15.90:0/845009126 shutdown_connections 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 --2- 172.21.15.90:0/845009126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250101440 0x7ff250101890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 --2- 172.21.15.90:0/845009126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250100240 0x7ff250100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 -- 172.21.15.90:0/845009126 >> 172.21.15.90:0/845009126 conn(0x7ff2500fb7f0 msgr2=0x7ff2500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 -- 172.21.15.90:0/845009126 shutdown_connections 2024-04-24T08:26:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.762+0000 7ff257800700 1 -- 172.21.15.90:0/845009126 wait complete. 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.763+0000 7ff257800700 1 Processor -- start 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.763+0000 7ff257800700 1 -- start start 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff257800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff257800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250101440 0x7ff250073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff257800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff250073680 con 0x7ff250100240 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff257800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2500737c0 con 0x7ff250101440 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42432/0 (socket says 172.21.15.90:42432) 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 -- 172.21.15.90:0/2857224672 learned_addr learned my addr 172.21.15.90:0/2857224672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:00.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff24fe00700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250101440 0x7ff250073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:00.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 -- 172.21.15.90:0/2857224672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250101440 msgr2=0x7ff250073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:00.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250101440 0x7ff250073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:00.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.764+0000 7ff254e00700 1 -- 172.21.15.90:0/2857224672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0095f0 con 0x7ff250100240 2024-04-24T08:26:00.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff254e00700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e8f0 tx=0x7ff24400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:00.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff244009790 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff250073aa0 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25019db10 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff24400c810 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.765+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff244003d20 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.767+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff230005320 con 0x7ff250100240 2024-04-24T08:26:00.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.767+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7ff244018430 con 0x7ff250100240 2024-04-24T08:26:00.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.768+0000 7ff24d600700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 0x7ff238070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:00.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.768+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff244014070 con 0x7ff250100240 2024-04-24T08:26:00.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.769+0000 7ff24fe00700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 0x7ff238070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:00.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.770+0000 7ff24fe00700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 0x7ff238070df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009fd0 tx=0x7ff23c005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:00.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:00.774+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7ff24405b9a0 con 0x7ff250100240 2024-04-24T08:26:01.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.035+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff230000bf0 con 0x7ff23806e940 2024-04-24T08:26:01.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.037+0000 7ff24d600700 1 -- 172.21.15.90:0/2857224672 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ff230000bf0 con 0x7ff23806e940 2024-04-24T08:26:01.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 msgr2=0x7ff238070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 0x7ff238070df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009fd0 tx=0x7ff23c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 msgr2=0x7ff250074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e8f0 tx=0x7ff24400ec00 comp rx=0 tx=0).stop 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 shutdown_connections 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ff23806e940 0x7ff238070df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff250100240 0x7ff250074af0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 --2- 172.21.15.90:0/2857224672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250101440 0x7ff250073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.040+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 >> 172.21.15.90:0/2857224672 conn(0x7ff2500fb7f0 msgr2=0x7ff250104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.041+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 shutdown_connections 2024-04-24T08:26:01.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.041+0000 7ff257800700 1 -- 172.21.15.90:0/2857224672 wait complete. 2024-04-24T08:26:01.058 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 -- 172.21.15.90:0/900358856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 msgr2=0x7f624c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 --2- 172.21.15.90:0/900358856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c073c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6238009940 tx=0x7f6238009c50 comp rx=0 tx=0).stop 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 -- 172.21.15.90:0/900358856 shutdown_connections 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 --2- 172.21.15.90:0/900358856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c073c20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 --2- 172.21.15.90:0/900358856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 0x7f624c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 -- 172.21.15.90:0/900358856 >> 172.21.15.90:0/900358856 conn(0x7f624c0fb7f0 msgr2=0x7f624c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 -- 172.21.15.90:0/900358856 shutdown_connections 2024-04-24T08:26:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.216+0000 7f6254600700 1 -- 172.21.15.90:0/900358856 wait complete. 2024-04-24T08:26:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.217+0000 7f6254600700 1 Processor -- start 2024-04-24T08:26:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.217+0000 7f6254600700 1 -- start start 2024-04-24T08:26:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6254600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6254600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 0x7f624c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6254600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1987e0 con 0x7f624c0737b0 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6254600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c198950 con 0x7f624c074d80 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6251c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6251c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42446/0 (socket says 172.21.15.90:42446) 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6251c00700 1 -- 172.21.15.90:0/4148851995 learned_addr learned my addr 172.21.15.90:0/4148851995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.218+0000 7f6251200700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 0x7f624c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.219+0000 7f6251c00700 1 -- 172.21.15.90:0/4148851995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 msgr2=0x7f624c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.219+0000 7f6251c00700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 0x7f624c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.219+0000 7f6251c00700 1 -- 172.21.15.90:0/4148851995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380095f0 con 0x7f624c0737b0 2024-04-24T08:26:01.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.219+0000 7f6251c00700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f624000e920 tx=0x7f624000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:01.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.220+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624000c9d0 con 0x7f624c0737b0 2024-04-24T08:26:01.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.220+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6240010430 con 0x7f624c0737b0 2024-04-24T08:26:01.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.220+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6240018650 con 0x7f624c0737b0 2024-04-24T08:26:01.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.220+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c19d3e0 con 0x7f624c0737b0 2024-04-24T08:26:01.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.220+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624c19d900 con 0x7f624c0737b0 2024-04-24T08:26:01.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.221+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c10af80 con 0x7f624c0737b0 2024-04-24T08:26:01.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.223+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f62400105a0 con 0x7f624c0737b0 2024-04-24T08:26:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.224+0000 7f624a000700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 0x7f6234070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.224+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6240014070 con 0x7f624c0737b0 2024-04-24T08:26:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.224+0000 7f6251200700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 0x7f6234070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.225+0000 7f6251200700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 0x7f6234070df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f623800b710 tx=0x7f62380058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.229+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f6240053ca0 con 0x7f624c0737b0 2024-04-24T08:26:01.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.517+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f624c107740 con 0x7f623406e940 2024-04-24T08:26:01.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.520+0000 7f624a000700 1 -- 172.21.15.90:0/4148851995 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f624c107740 con 0x7f623406e940 2024-04-24T08:26:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 msgr2=0x7f6234070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 0x7f6234070df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f623800b710 tx=0x7f62380058e0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 msgr2=0x7f624c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f624000e920 tx=0x7f624000ec30 comp rx=0 tx=0).stop 2024-04-24T08:26:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 shutdown_connections 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f623406e940 0x7f6234070df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f624c0737b0 0x7f624c197ca0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 --2- 172.21.15.90:0/4148851995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f624c074d80 0x7f624c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.524+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 >> 172.21.15.90:0/4148851995 conn(0x7f624c0fb7f0 msgr2=0x7f624c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.525+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 shutdown_connections 2024-04-24T08:26:01.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.525+0000 7f6254600700 1 -- 172.21.15.90:0/4148851995 wait complete. 2024-04-24T08:26:01.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.690+0000 7f8231c00700 1 -- 172.21.15.90:0/2823841357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 msgr2=0x7f822c101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.690+0000 7f8231c00700 1 --2- 172.21.15.90:0/2823841357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c101890 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8214009910 tx=0x7f8214009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:01.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 -- 172.21.15.90:0/2823841357 shutdown_connections 2024-04-24T08:26:01.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 --2- 172.21.15.90:0/2823841357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c101890 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 --2- 172.21.15.90:0/2823841357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 0x7f822c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 -- 172.21.15.90:0/2823841357 >> 172.21.15.90:0/2823841357 conn(0x7f822c0fb7f0 msgr2=0x7f822c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 -- 172.21.15.90:0/2823841357 shutdown_connections 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.691+0000 7f8231c00700 1 -- 172.21.15.90:0/2823841357 wait complete. 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.692+0000 7f8231c00700 1 Processor -- start 2024-04-24T08:26:01.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.692+0000 7f8231c00700 1 -- start start 2024-04-24T08:26:01.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8231c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 0x7f822c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8231c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8231c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c073680 con 0x7f822c101440 2024-04-24T08:26:01.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8231c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0737c0 con 0x7f822c100240 2024-04-24T08:26:01.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8223e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f822aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 0x7f822c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8223e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42454/0 (socket says 172.21.15.90:42454) 2024-04-24T08:26:01.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.693+0000 7f8223e00700 1 -- 172.21.15.90:0/1715155439 learned_addr learned my addr 172.21.15.90:0/1715155439 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:01.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.694+0000 7f8223e00700 1 -- 172.21.15.90:0/1715155439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 msgr2=0x7f822c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:01.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.694+0000 7f8223e00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 0x7f822c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:01.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.694+0000 7f8223e00700 1 -- 172.21.15.90:0/1715155439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140095f0 con 0x7f822c101440 2024-04-24T08:26:01.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.694+0000 7f8223e00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82140092e0 tx=0x7f821400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.695+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821401d070 con 0x7f822c101440 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.695+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c073a40 con 0x7f822c101440 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.695+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c073f30 con 0x7f822c101440 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.696+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c108d90 con 0x7f822c101440 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.697+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8214005dc0 con 0x7f822c101440 2024-04-24T08:26:01.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.697+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821402b440 con 0x7f822c101440 2024-04-24T08:26:01.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.701+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f8214003810 con 0x7f822c101440 2024-04-24T08:26:01.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.702+0000 7f8223400700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 0x7f8210070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:01.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.703+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8214090420 con 0x7f822c101440 2024-04-24T08:26:01.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.705+0000 7f822aa00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 0x7f8210070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:01.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.707+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f821405d2e0 con 0x7f822c101440 2024-04-24T08:26:01.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:01.707+0000 7f822aa00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 0x7f8210070df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f822c1012a0 tx=0x7f821c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:01.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:01 smithi149 ceph-mon[31282]: pgmap v53: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 864 KiB/s rd, 902 KiB/s wr, 79 op/s 2024-04-24T08:26:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:01 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:01 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:01 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:01 smithi090 ceph-mon[25016]: pgmap v53: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 864 KiB/s rd, 902 KiB/s wr, 79 op/s 2024-04-24T08:26:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:01 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:01 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:01.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:01 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:02.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.042+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f822c105d90 con 0x7f821006e940 2024-04-24T08:26:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.056+0000 7f8223400700 1 -- 172.21.15.90:0/1715155439 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f822c105d90 con 0x7f821006e940 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (11m) 35s ago 12m 14.3M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (13m) 35s ago 13m 7918k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (11m) 37s ago 11m 7843k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (13m) 35s ago 13m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:26:02.058 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (11m) 37s ago 11m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (10m) 35s ago 12m 75.9M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (5m) 35s ago 5m 270M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (5m) 35s ago 5m 14.4M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (5m) 37s ago 5m 253M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (5m) 37s ago 5m 14.4M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (118s) 35s ago 14m 494M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (43s) 37s ago 11m 56.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (14m) 35s ago 15m 53.8M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (11m) 37s ago 11m 44.0M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 starting - - - - 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 starting - - - - 2024-04-24T08:26:02.059 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 35s ago 10m 210M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (9m) 35s ago 9m 208M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (9m) 35s ago 9m 169M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (8m) 37s ago 8m 227M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (8m) 37s ago 8m 193M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (7m) 37s ago 7m 212M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:26:02.060 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (60s) 35s ago 12m 53.0M - 2.43.0 a07b618ecd1d 07acd8b5fe0d 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 msgr2=0x7f8210070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 0x7f8210070df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f822c1012a0 tx=0x7f821c009260 comp rx=0 tx=0).stop 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 msgr2=0x7f822c073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82140092e0 tx=0x7f821400f690 comp rx=0 tx=0).stop 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 shutdown_connections 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f821006e940 0x7f8210070df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f822c100240 0x7f822c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 --2- 172.21.15.90:0/1715155439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f822c101440 0x7f822c073140 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.060+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 >> 172.21.15.90:0/1715155439 conn(0x7f822c0fb7f0 msgr2=0x7f822c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:02.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.061+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 shutdown_connections 2024-04-24T08:26:02.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.061+0000 7f8231c00700 1 -- 172.21.15.90:0/1715155439 wait complete. 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 -- 172.21.15.90:0/4265102952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 msgr2=0x7f522c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 --2- 172.21.15.90:0/4265102952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c100650 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5220009910 tx=0x7f5220009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 -- 172.21.15.90:0/4265102952 shutdown_connections 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 --2- 172.21.15.90:0/4265102952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 --2- 172.21.15.90:0/4265102952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c100650 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.231+0000 7f5233a00700 1 -- 172.21.15.90:0/4265102952 >> 172.21.15.90:0/4265102952 conn(0x7f522c0fb7f0 msgr2=0x7f522c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.232+0000 7f5233a00700 1 -- 172.21.15.90:0/4265102952 shutdown_connections 2024-04-24T08:26:02.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.232+0000 7f5233a00700 1 -- 172.21.15.90:0/4265102952 wait complete. 2024-04-24T08:26:02.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 Processor -- start 2024-04-24T08:26:02.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 -- start start 2024-04-24T08:26:02.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c073680 con 0x7f522c100240 2024-04-24T08:26:02.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.233+0000 7f5233a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0737c0 con 0x7f522c101440 2024-04-24T08:26:02.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.234+0000 7f5231000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.234+0000 7f522be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.234+0000 7f5231000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42478/0 (socket says 172.21.15.90:42478) 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.234+0000 7f5231000700 1 -- 172.21.15.90:0/2043781852 learned_addr learned my addr 172.21.15.90:0/2043781852 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.235+0000 7f522be00700 1 -- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 msgr2=0x7f522c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.235+0000 7f522be00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.235+0000 7f522be00700 1 -- 172.21.15.90:0/2043781852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200095f0 con 0x7f522c101440 2024-04-24T08:26:02.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.235+0000 7f5231000700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.235+0000 7f522be00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c073140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f521800d8d0 tx=0x7f521800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.236+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521800f860 con 0x7f522c101440 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.236+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c073aa0 con 0x7f522c101440 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.236+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c19db10 con 0x7f522c101440 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.236+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f521800fe70 con 0x7f522c101440 2024-04-24T08:26:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.236+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521800e3b0 con 0x7f522c101440 2024-04-24T08:26:02.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.237+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c005320 con 0x7f522c101440 2024-04-24T08:26:02.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.239+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f521800f9c0 con 0x7f522c101440 2024-04-24T08:26:02.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.240+0000 7f5229600700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 0x7f5214070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.240+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f521808e3a0 con 0x7f522c101440 2024-04-24T08:26:02.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.240+0000 7f5231000700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 0x7f5214070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.241+0000 7f5231000700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 0x7f5214070df0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5220006010 tx=0x7f522000b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:02.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.256+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f521805aff0 con 0x7f522c101440 2024-04-24T08:26:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.611+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f520c005cc0 con 0x7f522c101440 2024-04-24T08:26:02.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.612+0000 7f5229600700 1 -- 172.21.15.90:0/2043781852 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f521805ae10 con 0x7f522c101440 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:26:02.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:26:02.615 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 msgr2=0x7f5214070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 0x7f5214070df0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5220006010 tx=0x7f522000b560 comp rx=0 tx=0).stop 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 msgr2=0x7f522c073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c073140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f521800d8d0 tx=0x7f521800dbe0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 shutdown_connections 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f521406e940 0x7f5214070df0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f522c100240 0x7f522c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 --2- 172.21.15.90:0/2043781852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f522c101440 0x7f522c073140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 >> 172.21.15.90:0/2043781852 conn(0x7f522c0fb7f0 msgr2=0x7f522c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:02.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 shutdown_connections 2024-04-24T08:26:02.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.616+0000 7f5233a00700 1 -- 172.21.15.90:0/2043781852 wait complete. 2024-04-24T08:26:02.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:02 smithi149 ceph-mon[31282]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:02.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:02 smithi149 ceph-mon[31282]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:02.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.781+0000 7f4809c00700 1 -- 172.21.15.90:0/2312049665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804103640 msgr2=0x7f4804103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.781+0000 7f4809c00700 1 --2- 172.21.15.90:0/2312049665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804103640 0x7f4804103a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009910 tx=0x7f47f0009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:02.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.782+0000 7f4809c00700 1 -- 172.21.15.90:0/2312049665 shutdown_connections 2024-04-24T08:26:02.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.782+0000 7f4809c00700 1 --2- 172.21.15.90:0/2312049665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804103640 0x7f4804103a90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.782+0000 7f4809c00700 1 --2- 172.21.15.90:0/2312049665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804102440 0x7f4804102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.782+0000 7f4809c00700 1 -- 172.21.15.90:0/2312049665 >> 172.21.15.90:0/2312049665 conn(0x7f48040fd9f0 msgr2=0x7f48040ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:02.784 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:02 smithi090 ceph-mon[25016]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:02.784 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:02 smithi090 ceph-mon[25016]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:02.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.783+0000 7f4809c00700 1 -- 172.21.15.90:0/2312049665 shutdown_connections 2024-04-24T08:26:02.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.783+0000 7f4809c00700 1 -- 172.21.15.90:0/2312049665 wait complete. 2024-04-24T08:26:02.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.785+0000 7f4809c00700 1 Processor -- start 2024-04-24T08:26:02.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.785+0000 7f4809c00700 1 -- start start 2024-04-24T08:26:02.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4809c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4809c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 0x7f480419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4809c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480419cd40 con 0x7f4804102440 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4809c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480419ce80 con 0x7f4804103640 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4802a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4802a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42492/0 (socket says 172.21.15.90:42492) 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.786+0000 7f4802a00700 1 -- 172.21.15.90:0/1115857652 learned_addr learned my addr 172.21.15.90:0/1115857652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.787+0000 7f4802000700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 0x7f480419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.787+0000 7f4802a00700 1 -- 172.21.15.90:0/1115857652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 msgr2=0x7f480419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.787+0000 7f4802a00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 0x7f480419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.787+0000 7f4802a00700 1 -- 172.21.15.90:0/1115857652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00095f0 con 0x7f4804102440 2024-04-24T08:26:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.787+0000 7f4802000700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 0x7f480419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:26:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.788+0000 7f4802a00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d900 tx=0x7f47f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.788+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f40041f0 con 0x7f4804102440 2024-04-24T08:26:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.789+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48041a1930 con 0x7f4804102440 2024-04-24T08:26:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.789+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48041a1e80 con 0x7f4804102440 2024-04-24T08:26:02.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.792+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804076930 con 0x7f4804102440 2024-04-24T08:26:02.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.792+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47f4004350 con 0x7f4804102440 2024-04-24T08:26:02.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.792+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f4003da0 con 0x7f4804102440 2024-04-24T08:26:02.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.798+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f47f4004870 con 0x7f4804102440 2024-04-24T08:26:02.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.803+0000 7f47fb400700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 0x7f47e8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:02.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.804+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f47f408ed60 con 0x7f4804102440 2024-04-24T08:26:02.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.805+0000 7f4802000700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 0x7f47e8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:02.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.807+0000 7f4802000700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 0x7f47e8070df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006010 tx=0x7f47f000b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:02.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:02.810+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f47f405b830 con 0x7f4804102440 2024-04-24T08:26:03.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.123+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4804061190 con 0x7f47e806e940 2024-04-24T08:26:03.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.126+0000 7f47fb400700 1 -- 172.21.15.90:0/1115857652 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f4804061190 con 0x7f47e806e940 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:26:03.127 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:26:03.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 msgr2=0x7f47e8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:03.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 0x7f47e8070df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006010 tx=0x7f47f000b560 comp rx=0 tx=0).stop 2024-04-24T08:26:03.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 msgr2=0x7f480419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:03.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d900 tx=0x7f47f400dcc0 comp rx=0 tx=0).stop 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 shutdown_connections 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f47e806e940 0x7f47e8070df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4804102440 0x7f480419c1e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.128+0000 7f4809c00700 1 --2- 172.21.15.90:0/1115857652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4804103640 0x7f480419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.129+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 >> 172.21.15.90:0/1115857652 conn(0x7f48040fd9f0 msgr2=0x7f48040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.129+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 shutdown_connections 2024-04-24T08:26:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:03.129+0000 7f4809c00700 1 -- 172.21.15.90:0/1115857652 wait complete. 2024-04-24T08:26:03.515 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:03 smithi149 ceph-mon[31282]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:03 smithi149 ceph-mon[31282]: pgmap v54: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-24T08:26:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:03 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2043781852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:03.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:03 smithi090 ceph-mon[25016]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:03.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:03 smithi090 ceph-mon[25016]: pgmap v54: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-24T08:26:03.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:03 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2043781852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:04.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:04 smithi149 ceph-mon[31282]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:04 smithi090 ceph-mon[25016]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:05.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:05 smithi149 ceph-mon[31282]: pgmap v55: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 930 KiB/s rd, 993 KiB/s wr, 64 op/s 2024-04-24T08:26:05.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:05 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:05 smithi090 ceph-mon[25016]: pgmap v55: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 930 KiB/s rd, 993 KiB/s wr, 64 op/s 2024-04-24T08:26:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:05 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:07 smithi090 ceph-mon[25016]: pgmap v56: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 930 KiB/s rd, 993 KiB/s wr, 69 op/s 2024-04-24T08:26:07.902 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:07 smithi149 ceph-mon[31282]: pgmap v56: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 930 KiB/s rd, 993 KiB/s wr, 69 op/s 2024-04-24T08:26:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:09 smithi090 ceph-mon[25016]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-04-24T08:26:10.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:09 smithi149 ceph-mon[31282]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-04-24T08:26:10.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:10 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:10.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:10 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:11.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:10 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:11.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:10 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:11 smithi149 ceph-mon[31282]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-24T08:26:12.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:11 smithi090 ceph-mon[25016]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-24T08:26:13.013 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:12 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:13.013 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:12 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:13.013 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:12 smithi090 ceph-mon[25016]: pgmap v59: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-04-24T08:26:13.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:12 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:13.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:12 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:12 smithi149 ceph-mon[31282]: pgmap v59: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-04-24T08:26:14.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:13 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:14.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:13 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:14.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:13 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:14.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:13 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:15.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:14 smithi090 ceph-mon[25016]: pgmap v60: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-04-24T08:26:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:14 smithi149 ceph-mon[31282]: pgmap v60: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-04-24T08:26:16.724 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:16 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:16.724 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:16 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:16.757 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:16 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:16.757 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:16 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:17 smithi149 ceph-mon[31282]: pgmap v61: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-04-24T08:26:17.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:17 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:17.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:17 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:17 smithi090 ceph-mon[25016]: pgmap v61: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-04-24T08:26:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:17 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:17 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:19 smithi149 ceph-mon[31282]: pgmap v62: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-04-24T08:26:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:19 smithi090 ceph-mon[25016]: pgmap v62: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-04-24T08:26:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:21 smithi149 ceph-mon[31282]: pgmap v63: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-04-24T08:26:21.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:21 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:21 smithi090 ceph-mon[25016]: pgmap v63: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-04-24T08:26:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:21 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:22 smithi149 ceph-mon[31282]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-04-24T08:26:23.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:22 smithi090 ceph-mon[25016]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.753 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.754 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.754 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.754 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.754 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.754 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:24 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:24.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:24 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:25.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:25 smithi090 ceph-mon[25016]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-04-24T08:26:25.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:25 smithi149 ceph-mon[31282]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-04-24T08:26:27.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:27 smithi149 ceph-mon[31282]: Upgrade: Updating prometheus.smithi090 2024-04-24T08:26:27.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:27 smithi149 ceph-mon[31282]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-24T08:26:27.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:27 smithi149 ceph-mon[31282]: pgmap v66: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-04-24T08:26:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:27 smithi090 ceph-mon[25016]: Upgrade: Updating prometheus.smithi090 2024-04-24T08:26:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:27 smithi090 ceph-mon[25016]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-24T08:26:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:27 smithi090 ceph-mon[25016]: pgmap v66: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-04-24T08:26:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:29 smithi149 ceph-mon[31282]: pgmap v67: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-24T08:26:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:29 smithi090 ceph-mon[25016]: pgmap v67: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-24T08:26:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:31 smithi149 ceph-mon[31282]: pgmap v68: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-04-24T08:26:31.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:31 smithi090 ceph-mon[25016]: pgmap v68: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-04-24T08:26:33.158 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:32 smithi090 ceph-mon[25016]: pgmap v69: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-04-24T08:26:33.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:32 smithi149 ceph-mon[31282]: pgmap v69: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-04-24T08:26:33.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 -- 172.21.15.90:0/4115890190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc101440 msgr2=0x7f87cc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 --2- 172.21.15.90:0/4115890190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc101440 0x7f87cc101890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009910 tx=0x7f87b8009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 -- 172.21.15.90:0/4115890190 shutdown_connections 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 --2- 172.21.15.90:0/4115890190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc101440 0x7f87cc101890 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 --2- 172.21.15.90:0/4115890190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc100240 0x7f87cc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 -- 172.21.15.90:0/4115890190 >> 172.21.15.90:0/4115890190 conn(0x7f87cc0fb7f0 msgr2=0x7f87cc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 -- 172.21.15.90:0/4115890190 shutdown_connections 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.321+0000 7f87d0c00700 1 -- 172.21.15.90:0/4115890190 wait complete. 2024-04-24T08:26:33.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 Processor -- start 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 -- start start 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc101440 0x7f87cc1931f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc193810 con 0x7f87cc100240 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.322+0000 7f87d0c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc193950 con 0x7f87cc101440 2024-04-24T08:26:33.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58072/0 (socket says 172.21.15.90:58072) 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 -- 172.21.15.90:0/1766666046 learned_addr learned my addr 172.21.15.90:0/1766666046 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9400700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc101440 0x7f87cc1931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 -- 172.21.15.90:0/1766666046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc101440 msgr2=0x7f87cc1931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc101440 0x7f87cc1931f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.323+0000 7f87c9e00700 1 -- 172.21.15.90:0/1766666046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80095f0 con 0x7f87cc100240 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87c9e00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c8f0 tx=0x7f87bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87bc007a70 con 0x7f87cc100240 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87bc007bd0 con 0x7f87cc100240 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc198400 con 0x7f87cc100240 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87bc018630 con 0x7f87cc100240 2024-04-24T08:26:33.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.324+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc198950 con 0x7f87cc100240 2024-04-24T08:26:33.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.325+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac005320 con 0x7f87cc100240 2024-04-24T08:26:33.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.326+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f87bc00fa00 con 0x7f87cc100240 2024-04-24T08:26:33.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.327+0000 7f87c2a00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 0x7f87b0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.327+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f87bc08e7e0 con 0x7f87cc100240 2024-04-24T08:26:33.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.328+0000 7f87c9400700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 0x7f87b0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.328+0000 7f87c9400700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 0x7f87b0070df0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000c00 tx=0x7f87b8005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:33.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.332+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f87bc05b620 con 0x7f87cc100240 2024-04-24T08:26:33.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.664+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ac000bf0 con 0x7f87b006e940 2024-04-24T08:26:33.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.666+0000 7f87c2a00700 1 -- 172.21.15.90:0/1766666046 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f87ac000bf0 con 0x7f87b006e940 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 msgr2=0x7f87b0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 0x7f87b0070df0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000c00 tx=0x7f87b8005fb0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 msgr2=0x7f87cc192cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c8f0 tx=0x7f87bc00ccb0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 shutdown_connections 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f87b006e940 0x7f87b0070df0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc100240 0x7f87cc192cb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1766666046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87cc101440 0x7f87cc1931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 >> 172.21.15.90:0/1766666046 conn(0x7f87cc0fb7f0 msgr2=0x7f87cc102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:33.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 shutdown_connections 2024-04-24T08:26:33.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.669+0000 7f87d0c00700 1 -- 172.21.15.90:0/1766666046 wait complete. 2024-04-24T08:26:33.688 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.847+0000 7f9a6b200700 1 -- 172.21.15.90:0/167985051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 msgr2=0x7f9a6c105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 --2- 172.21.15.90:0/167985051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c105860 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009910 tx=0x7f9a58009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 -- 172.21.15.90:0/167985051 shutdown_connections 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 --2- 172.21.15.90:0/167985051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c105860 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 --2- 172.21.15.90:0/167985051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 0x7f9a6c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 -- 172.21.15.90:0/167985051 >> 172.21.15.90:0/167985051 conn(0x7f9a6c0fa520 msgr2=0x7f9a6c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.848+0000 7f9a6b200700 1 -- 172.21.15.90:0/167985051 shutdown_connections 2024-04-24T08:26:33.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.849+0000 7f9a6b200700 1 -- 172.21.15.90:0/167985051 wait complete. 2024-04-24T08:26:33.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.850+0000 7f9a6b200700 1 Processor -- start 2024-04-24T08:26:33.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.850+0000 7f9a6b200700 1 -- start start 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a6b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 0x7f9a6c100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a6b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a6b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c1994f0 con 0x7f9a6c103480 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a6b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c199660 con 0x7f9a6c100b60 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a69400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a69400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58098/0 (socket says 172.21.15.90:58098) 2024-04-24T08:26:33.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a69400700 1 -- 172.21.15.90:0/109891373 learned_addr learned my addr 172.21.15.90:0/109891373 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:33.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.851+0000 7f9a69e00700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 0x7f9a6c100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a69400700 1 -- 172.21.15.90:0/109891373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 msgr2=0x7f9a6c100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:33.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a69400700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 0x7f9a6c100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a69400700 1 -- 172.21.15.90:0/109891373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a580095f0 con 0x7f9a6c103480 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a69400700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9a58006010 tx=0x7f9a5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5801d070 con 0x7f9a6c103480 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a5800f460 con 0x7f9a6c103480 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a58017630 con 0x7f9a6c103480 2024-04-24T08:26:33.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c0ff050 con 0x7f9a6c103480 2024-04-24T08:26:33.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.852+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c0ff540 con 0x7f9a6c103480 2024-04-24T08:26:33.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.854+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c18fa90 con 0x7f9a6c103480 2024-04-24T08:26:33.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.855+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f9a58003a60 con 0x7f9a6c103480 2024-04-24T08:26:33.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.856+0000 7f9a62a00700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 0x7f9a50070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:33.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.856+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9a5808fdf0 con 0x7f9a6c103480 2024-04-24T08:26:33.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.856+0000 7f9a69e00700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 0x7f9a50070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:33.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.857+0000 7f9a69e00700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 0x7f9a50070df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005950 tx=0x7f9a5c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:33.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:33.861+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f9a58059060 con 0x7f9a6c103480 2024-04-24T08:26:34.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.154+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a6c061190 con 0x7f9a5006e940 2024-04-24T08:26:34.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.156+0000 7f9a62a00700 1 -- 172.21.15.90:0/109891373 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f9a6c061190 con 0x7f9a5006e940 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 msgr2=0x7f9a50070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 0x7f9a50070df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005950 tx=0x7f9a5c0058e0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 msgr2=0x7f9a6c0feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9a58006010 tx=0x7f9a5800be20 comp rx=0 tx=0).stop 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 shutdown_connections 2024-04-24T08:26:34.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9a5006e940 0x7f9a50070df0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a6c100b60 0x7f9a6c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 --2- 172.21.15.90:0/109891373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a6c103480 0x7f9a6c0feb10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 >> 172.21.15.90:0/109891373 conn(0x7f9a6c0fa520 msgr2=0x7f9a6c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:34.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.160+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 shutdown_connections 2024-04-24T08:26:34.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.161+0000 7f9a6b200700 1 -- 172.21.15.90:0/109891373 wait complete. 2024-04-24T08:26:34.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 -- 172.21.15.90:0/3101094700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 msgr2=0x7fe2f0100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 --2- 172.21.15.90:0/3101094700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0100840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009910 tx=0x7fe2e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 -- 172.21.15.90:0/3101094700 shutdown_connections 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 --2- 172.21.15.90:0/3101094700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0100840 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 --2- 172.21.15.90:0/3101094700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 0x7fe2f00ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 -- 172.21.15.90:0/3101094700 >> 172.21.15.90:0/3101094700 conn(0x7fe2f00fa780 msgr2=0x7fe2f00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.340+0000 7fe2f8800700 1 -- 172.21.15.90:0/3101094700 shutdown_connections 2024-04-24T08:26:34.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.341+0000 7fe2f8800700 1 -- 172.21.15.90:0/3101094700 wait complete. 2024-04-24T08:26:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 Processor -- start 2024-04-24T08:26:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 -- start start 2024-04-24T08:26:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 0x7fe2f0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f01965c0 con 0x7fe2f01003d0 2024-04-24T08:26:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.343+0000 7fe2f8800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f0196700 con 0x7fe2f00ff130 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58110/0 (socket says 172.21.15.90:58110) 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 -- 172.21.15.90:0/4044884336 learned_addr learned my addr 172.21.15.90:0/4044884336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5e00700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 0x7fe2f0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 -- 172.21.15.90:0/4044884336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 msgr2=0x7fe2f0195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 0x7fe2f0195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 -- 172.21.15.90:0/4044884336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00095f0 con 0x7fe2f01003d0 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.344+0000 7fe2f5400700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00092e0 tx=0x7fe2e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.345+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e001d070 con 0x7fe2f01003d0 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.345+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe2e0005dc0 con 0x7fe2f01003d0 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.345+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e0017630 con 0x7fe2f01003d0 2024-04-24T08:26:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.345+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f019b150 con 0x7fe2f01003d0 2024-04-24T08:26:34.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.345+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f019b5f0 con 0x7fe2f01003d0 2024-04-24T08:26:34.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.346+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0107c10 con 0x7fe2f01003d0 2024-04-24T08:26:34.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.347+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fe2e000fba0 con 0x7fe2f01003d0 2024-04-24T08:26:34.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.348+0000 7fe2eaa00700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 0x7fe2d8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.348+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe2e008fda0 con 0x7fe2f01003d0 2024-04-24T08:26:34.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.349+0000 7fe2f5e00700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 0x7fe2d8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.349+0000 7fe2f5e00700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 0x7fe2d8070df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009bf0 tx=0x7fe2e4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:34.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.354+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fe2e005cb60 con 0x7fe2f01003d0 2024-04-24T08:26:34.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.723+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2f0068c50 con 0x7fe2d806e940 2024-04-24T08:26:34.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.735+0000 7fe2eaa00700 1 -- 172.21.15.90:0/4044884336 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe2f0068c50 con 0x7fe2d806e940 2024-04-24T08:26:34.736 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (11m) 25s ago 13m 14.5M - 0.25.0 c8568f914cd2 6daa795eea5c 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (13m) 25s ago 13m 7972k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (12m) 21s ago 12m 7902k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (13m) 25s ago 13m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (12m) 21s ago 12m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (11m) 25s ago 13m 76.0M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (6m) 25s ago 6m 283M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (6m) 25s ago 6m 14.8M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (6m) 21s ago 6m 255M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (6m) 21s ago 6m 14.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:26:34.737 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (2m) 25s ago 15m 499M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (76s) 21s ago 12m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (15m) 25s ago 15m 56.5M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (12m) 21s ago 12m 47.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (44s) 25s ago 13m 6459k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (34s) 21s ago 12m 2919k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (11m) 25s ago 11m 181M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (10m) 25s ago 10m 189M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (9m) 25s ago 9m 157M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (9m) 21s ago 9m 208M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (8m) 21s ago 8m 181M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (8m) 21s ago 8m 172M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:26:34.738 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (93s) 25s ago 12m 59.7M - 2.43.0 a07b618ecd1d 07acd8b5fe0d 2024-04-24T08:26:34.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 msgr2=0x7fe2d8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 0x7fe2d8070df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009bf0 tx=0x7fe2e4009260 comp rx=0 tx=0).stop 2024-04-24T08:26:34.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 msgr2=0x7fe2f0195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00092e0 tx=0x7fe2e000f740 comp rx=0 tx=0).stop 2024-04-24T08:26:34.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 shutdown_connections 2024-04-24T08:26:34.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fe2d806e940 0x7fe2d8070df0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00ff130 0x7fe2f0195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 --2- 172.21.15.90:0/4044884336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f01003d0 0x7fe2f0195fa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.739+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 >> 172.21.15.90:0/4044884336 conn(0x7fe2f00fa780 msgr2=0x7fe2f0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:34.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.740+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 shutdown_connections 2024-04-24T08:26:34.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.740+0000 7fe2f8800700 1 -- 172.21.15.90:0/4044884336 wait complete. 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 -- 172.21.15.90:0/2045994349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c103680 msgr2=0x7f143c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 --2- 172.21.15.90:0/2045994349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c103680 0x7f143c105a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1428009910 tx=0x7f1428009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 -- 172.21.15.90:0/2045994349 shutdown_connections 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 --2- 172.21.15.90:0/2045994349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c103680 0x7f143c105a60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 --2- 172.21.15.90:0/2045994349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c100d60 0x7f143c103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 -- 172.21.15.90:0/2045994349 >> 172.21.15.90:0/2045994349 conn(0x7f143c0fa780 msgr2=0x7f143c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 -- 172.21.15.90:0/2045994349 shutdown_connections 2024-04-24T08:26:34.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.907+0000 7f1441000700 1 -- 172.21.15.90:0/2045994349 wait complete. 2024-04-24T08:26:34.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 Processor -- start 2024-04-24T08:26:34.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 -- start start 2024-04-24T08:26:34.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c103680 0x7f143c196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c196640 con 0x7f143c100d60 2024-04-24T08:26:34.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.909+0000 7f1441000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c196780 con 0x7f143c103680 2024-04-24T08:26:34.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58130/0 (socket says 172.21.15.90:58130) 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 -- 172.21.15.90:0/3809961587 learned_addr learned my addr 172.21.15.90:0/3809961587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439400700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c103680 0x7f143c196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 -- 172.21.15.90:0/3809961587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c103680 msgr2=0x7f143c196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c103680 0x7f143c196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.910+0000 7f1439e00700 1 -- 172.21.15.90:0/3809961587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280095f0 con 0x7f143c100d60 2024-04-24T08:26:34.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1439e00700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f142c00d900 tx=0x7f142c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:34.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f142c009900 con 0x7f143c100d60 2024-04-24T08:26:34.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c19b230 con 0x7f143c100d60 2024-04-24T08:26:34.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f142c010460 con 0x7f143c100d60 2024-04-24T08:26:34.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f142c00f5f0 con 0x7f143c100d60 2024-04-24T08:26:34.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.911+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c19b780 con 0x7f143c100d60 2024-04-24T08:26:34.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.912+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c18fbe0 con 0x7f143c100d60 2024-04-24T08:26:34.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.914+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f142c0105d0 con 0x7f143c100d60 2024-04-24T08:26:34.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.914+0000 7f1432a00700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 0x7f1420070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:34.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.915+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f142c08e4c0 con 0x7f143c100d60 2024-04-24T08:26:34.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.915+0000 7f1439400700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 0x7f1420070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:34.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.916+0000 7f1439400700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 0x7f1420070df0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1428006010 tx=0x7f142800b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:34.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:34.920+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f142c05b280 con 0x7f143c100d60 2024-04-24T08:26:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.294+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f143c061a00 con 0x7f143c100d60 2024-04-24T08:26:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.295+0000 7f1432a00700 1 -- 172.21.15.90:0/3809961587 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f142c05ae10 con 0x7f143c100d60 2024-04-24T08:26:35.296 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:26:35.296 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:26:35.297 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:26:35.298 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 msgr2=0x7f1420070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 0x7f1420070df0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1428006010 tx=0x7f142800b560 comp rx=0 tx=0).stop 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 msgr2=0x7f143c195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f142c00d900 tx=0x7f142c00dc10 comp rx=0 tx=0).stop 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 shutdown_connections 2024-04-24T08:26:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f142006e940 0x7f1420070df0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f143c100d60 0x7f143c195ae0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 --2- 172.21.15.90:0/3809961587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c103680 0x7f143c196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 >> 172.21.15.90:0/3809961587 conn(0x7f143c0fa780 msgr2=0x7f143c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 shutdown_connections 2024-04-24T08:26:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.298+0000 7f1441000700 1 -- 172.21.15.90:0/3809961587 wait complete. 2024-04-24T08:26:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 -- 172.21.15.90:0/1000220494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 msgr2=0x7f3578101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 --2- 172.21.15.90:0/1000220494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578101730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3564009910 tx=0x7f3564009c20 comp rx=0 tx=0).stop 2024-04-24T08:26:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 -- 172.21.15.90:0/1000220494 shutdown_connections 2024-04-24T08:26:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 --2- 172.21.15.90:0/1000220494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578101730 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 --2- 172.21.15.90:0/1000220494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 0x7f3578100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 -- 172.21.15.90:0/1000220494 >> 172.21.15.90:0/1000220494 conn(0x7f35780fb650 msgr2=0x7f35780fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:35.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 -- 172.21.15.90:0/1000220494 shutdown_connections 2024-04-24T08:26:35.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.488+0000 7f357ca00700 1 -- 172.21.15.90:0/1000220494 wait complete. 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 Processor -- start 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 -- start start 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 0x7f3578197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3578198560 con 0x7f35781012c0 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.489+0000 7f357ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35781986a0 con 0x7f3578100020 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58146/0 (socket says 172.21.15.90:58146) 2024-04-24T08:26:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 -- 172.21.15.90:0/2841736673 learned_addr learned my addr 172.21.15.90:0/2841736673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:26:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3576800700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 0x7f3578197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 -- 172.21.15.90:0/2841736673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 msgr2=0x7f3578197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 0x7f3578197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 -- 172.21.15.90:0/2841736673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640095f0 con 0x7f35781012c0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.490+0000 7f3575e00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3564005b40 tx=0x7f356400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.491+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356401d070 con 0x7f35781012c0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.491+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f356400f460 con 0x7f35781012c0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.491+0000 7f357ca00700 1 -- 172.21.15.90:0/2841736673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357819d0f0 con 0x7f35781012c0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.491+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3564021620 con 0x7f35781012c0 2024-04-24T08:26:35.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.491+0000 7f357ca00700 1 -- 172.21.15.90:0/2841736673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357819d5e0 con 0x7f35781012c0 2024-04-24T08:26:35.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.492+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35500052f0 con 0x7f35781012c0 2024-04-24T08:26:35.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.493+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f356400fa40 con 0x7f35781012c0 2024-04-24T08:26:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.494+0000 7f356f400700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 0x7f355c070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:26:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.494+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f356408fa90 con 0x7f35781012c0 2024-04-24T08:26:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.496+0000 7f3576800700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 0x7f355c070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:26:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.496+0000 7f3576800700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 0x7f355c070df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3568007c60 tx=0x7f35680073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:26:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.500+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f356405c8d0 con 0x7f35781012c0 2024-04-24T08:26:35.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.770+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3550000bc0 con 0x7f355c06e940 2024-04-24T08:26:35.771 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:35 smithi090 ceph-mon[25016]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:35.771 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:35 smithi090 ceph-mon[25016]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:35.771 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:35 smithi090 ceph-mon[25016]: pgmap v70: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-04-24T08:26:35.772 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:35 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/3809961587' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.772+0000 7f356f400700 1 -- 172.21.15.90:0/2841736673 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f3550000bc0 con 0x7f355c06e940 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:26:35.773 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:26:35.774 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:26:35.774 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:26:35.774 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-24T08:26:35.774 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:26:35.774 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 msgr2=0x7f355c070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 0x7f355c070df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3568007c60 tx=0x7f35680073d0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 msgr2=0x7f3578197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3564005b40 tx=0x7f356400bd70 comp rx=0 tx=0).stop 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 shutdown_connections 2024-04-24T08:26:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f355c06e940 0x7f355c070df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3578100020 0x7f3578197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 --2- 172.21.15.90:0/2841736673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781012c0 0x7f3578197f40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:26:35.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 >> 172.21.15.90:0/2841736673 conn(0x7f35780fb650 msgr2=0x7f35780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:26:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:35 smithi149 ceph-mon[31282]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:35 smithi149 ceph-mon[31282]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:35 smithi149 ceph-mon[31282]: pgmap v70: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-04-24T08:26:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:35 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3809961587' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:35.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 shutdown_connections 2024-04-24T08:26:35.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:26:35.775+0000 7f356cc00700 1 -- 172.21.15.90:0/2841736673 wait complete. 2024-04-24T08:26:36.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:36 smithi149 ceph-mon[31282]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:36.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:36 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:36.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:36 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:36.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:36 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:36.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:36 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:36.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:36 smithi090 ceph-mon[25016]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:36.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:36 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:36.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:36 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:36.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:36 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:36.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:36 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:37 smithi149 ceph-mon[31282]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:37 smithi149 ceph-mon[31282]: pgmap v71: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-04-24T08:26:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:37 smithi090 ceph-mon[25016]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:26:37.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:37 smithi090 ceph-mon[25016]: pgmap v71: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-04-24T08:26:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:39 smithi149 ceph-mon[31282]: pgmap v72: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-04-24T08:26:39.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:39 smithi090 ceph-mon[25016]: pgmap v72: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-04-24T08:26:41.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:41 smithi090 ceph-mon[25016]: pgmap v73: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-24T08:26:42.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:41 smithi149 ceph-mon[31282]: pgmap v73: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-24T08:26:43.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:42 smithi149 ceph-mon[31282]: pgmap v74: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-04-24T08:26:43.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:42 smithi090 ceph-mon[25016]: pgmap v74: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-04-24T08:26:45.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:45.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:44 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:45.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:44 smithi149 ceph-mon[31282]: pgmap v75: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-04-24T08:26:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:44 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:45.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:44 smithi090 ceph-mon[25016]: pgmap v75: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-04-24T08:26:47.394 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:47 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:47.395 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:47 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:47.395 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:47 smithi090 ceph-mon[25016]: pgmap v76: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 279 op/s 2024-04-24T08:26:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:47 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:47 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:47.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:47 smithi149 ceph-mon[31282]: pgmap v76: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 279 op/s 2024-04-24T08:26:49.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:49 smithi149 ceph-mon[31282]: pgmap v77: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 336 op/s 2024-04-24T08:26:49.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:49 smithi090 ceph-mon[25016]: pgmap v77: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 336 op/s 2024-04-24T08:26:51.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: pgmap v78: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-24T08:26:51.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:51.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: pgmap v78: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:26:51.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:51.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:26:52.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:52 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:26:52.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:52 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T08:26:53.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:53 smithi149 ceph-mon[31282]: pgmap v79: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 387 op/s 2024-04-24T08:26:53.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:53 smithi090 ceph-mon[25016]: pgmap v79: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 387 op/s 2024-04-24T08:26:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:54 smithi090 ceph-mon[25016]: Upgrade: Updating alertmanager.smithi090 2024-04-24T08:26:54.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:54 smithi090 ceph-mon[25016]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-24T08:26:55.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:54 smithi149 ceph-mon[31282]: Upgrade: Updating alertmanager.smithi090 2024-04-24T08:26:55.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:54 smithi149 ceph-mon[31282]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-24T08:26:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:55 smithi090 ceph-mon[25016]: pgmap v80: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 1.1 MiB/s wr, 234 op/s 2024-04-24T08:26:56.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:55 smithi149 ceph-mon[31282]: pgmap v80: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 1.1 MiB/s wr, 234 op/s 2024-04-24T08:26:57.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:57 smithi090 ceph-mon[25016]: pgmap v81: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 285 op/s 2024-04-24T08:26:58.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:57 smithi149 ceph-mon[31282]: pgmap v81: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 285 op/s 2024-04-24T08:26:59.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:26:59 smithi090 ceph-mon[25016]: pgmap v82: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-04-24T08:27:00.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:26:59 smithi149 ceph-mon[31282]: pgmap v82: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-04-24T08:27:01.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:01 smithi090 ceph-mon[25016]: pgmap v83: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 901 KiB/s rd, 950 KiB/s wr, 150 op/s 2024-04-24T08:27:02.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:01 smithi149 ceph-mon[31282]: pgmap v83: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 901 KiB/s rd, 950 KiB/s wr, 150 op/s 2024-04-24T08:27:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:03 smithi149 ceph-mon[31282]: pgmap v84: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-04-24T08:27:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:03 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:03 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:03 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:03 smithi090 ceph-mon[25016]: pgmap v84: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-04-24T08:27:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:03 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:03 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:03 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:05 smithi149 ceph-mon[31282]: pgmap v85: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 820 KiB/s rd, 864 KiB/s wr, 136 op/s 2024-04-24T08:27:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:05 smithi090 ceph-mon[25016]: pgmap v85: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 820 KiB/s rd, 864 KiB/s wr, 136 op/s 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 -- 172.21.15.90:0/2677564515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628103640 msgr2=0x7f3628103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 --2- 172.21.15.90:0/2677564515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628103640 0x7f3628103a90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3614009910 tx=0x7f3614009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 -- 172.21.15.90:0/2677564515 shutdown_connections 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 --2- 172.21.15.90:0/2677564515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628103640 0x7f3628103a90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 --2- 172.21.15.90:0/2677564515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628102440 0x7f3628102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:05.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 -- 172.21.15.90:0/2677564515 >> 172.21.15.90:0/2677564515 conn(0x7f36280fd9f0 msgr2=0x7f36280ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 -- 172.21.15.90:0/2677564515 shutdown_connections 2024-04-24T08:27:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.959+0000 7f3630000700 1 -- 172.21.15.90:0/2677564515 wait complete. 2024-04-24T08:27:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 Processor -- start 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 -- start start 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628103640 0x7f36281981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281987d0 con 0x7f3628102440 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.961+0000 7f3630000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3628198940 con 0x7f3628103640 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43486/0 (socket says 172.21.15.90:43486) 2024-04-24T08:27:05.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 -- 172.21.15.90:0/20874200 learned_addr learned my addr 172.21.15.90:0/20874200 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362cc00700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628103640 0x7f36281981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 -- 172.21.15.90:0/20874200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628103640 msgr2=0x7f36281981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628103640 0x7f36281981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.962+0000 7f362d600700 1 -- 172.21.15.90:0/20874200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140095f0 con 0x7f3628102440 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f362d600700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f361c00d900 tx=0x7f361c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c0041f0 con 0x7f3628102440 2024-04-24T08:27:05.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f361c004350 con 0x7f3628102440 2024-04-24T08:27:05.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362819d3d0 con 0x7f3628102440 2024-04-24T08:27:05.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c003da0 con 0x7f3628102440 2024-04-24T08:27:05.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.963+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362819d920 con 0x7f3628102440 2024-04-24T08:27:05.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.965+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362810af90 con 0x7f3628102440 2024-04-24T08:27:05.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.966+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f361c009730 con 0x7f3628102440 2024-04-24T08:27:05.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.967+0000 7f3626000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 0x7f3610070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:05.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.967+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f361c08ea40 con 0x7f3628102440 2024-04-24T08:27:05.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.967+0000 7f362cc00700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 0x7f3610070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:05.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.968+0000 7f362cc00700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 0x7f3610070df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f36281991a0 tx=0x7f3614005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:05.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:05.972+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f361c057cd0 con 0x7f3628102440 2024-04-24T08:27:06.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.330+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3628107f90 con 0x7f361006e940 2024-04-24T08:27:06.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.331+0000 7f3626000700 1 -- 172.21.15.90:0/20874200 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f3628107f90 con 0x7f361006e940 2024-04-24T08:27:06.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.334+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 msgr2=0x7f3610070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.334+0000 7f3630000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 0x7f3610070df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f36281991a0 tx=0x7f3614005a90 comp rx=0 tx=0).stop 2024-04-24T08:27:06.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.334+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 msgr2=0x7f3628197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.334+0000 7f3630000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f361c00d900 tx=0x7f361c00dcc0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 shutdown_connections 2024-04-24T08:27:06.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f361006e940 0x7f3610070df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3628102440 0x7f3628197c90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 --2- 172.21.15.90:0/20874200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3628103640 0x7f36281981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 >> 172.21.15.90:0/20874200 conn(0x7f36280fd9f0 msgr2=0x7f3628106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:06.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 shutdown_connections 2024-04-24T08:27:06.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.335+0000 7f3630000700 1 -- 172.21.15.90:0/20874200 wait complete. 2024-04-24T08:27:06.355 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 -- 172.21.15.90:0/2433427095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8100d60 msgr2=0x7f12d8103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 --2- 172.21.15.90:0/2433427095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8100d60 0x7f12d8103140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009910 tx=0x7f12cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 -- 172.21.15.90:0/2433427095 shutdown_connections 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 --2- 172.21.15.90:0/2433427095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8103680 0x7f12d8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 --2- 172.21.15.90:0/2433427095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8100d60 0x7f12d8103140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 -- 172.21.15.90:0/2433427095 >> 172.21.15.90:0/2433427095 conn(0x7f12d80fa780 msgr2=0x7f12d80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:06.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 -- 172.21.15.90:0/2433427095 shutdown_connections 2024-04-24T08:27:06.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.512+0000 7f12dfc00700 1 -- 172.21.15.90:0/2433427095 wait complete. 2024-04-24T08:27:06.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.513+0000 7f12dfc00700 1 Processor -- start 2024-04-24T08:27:06.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.513+0000 7f12dfc00700 1 -- start start 2024-04-24T08:27:06.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12dfc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8100d60 0x7f12d8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12dfc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12dfc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8198860 con 0x7f12d8103680 2024-04-24T08:27:06.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12dfc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d819d270 con 0x7f12d8100d60 2024-04-24T08:27:06.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12d7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12d7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43508/0 (socket says 172.21.15.90:43508) 2024-04-24T08:27:06.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12d7e00700 1 -- 172.21.15.90:0/3262882138 learned_addr learned my addr 172.21.15.90:0/3262882138 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:06.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.514+0000 7f12dd200700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8100d60 0x7f12d8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d7e00700 1 -- 172.21.15.90:0/3262882138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8100d60 msgr2=0x7f12d8197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d7e00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8100d60 0x7f12d8197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d7e00700 1 -- 172.21.15.90:0/3262882138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0095f0 con 0x7f12d8103680 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d7e00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f12c400d8d0 tx=0x7f12c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c40098a0 con 0x7f12d8103680 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f12c4010460 con 0x7f12d8103680 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c400f5f0 con 0x7f12d8103680 2024-04-24T08:27:06.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d819d470 con 0x7f12d8103680 2024-04-24T08:27:06.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.515+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d819d940 con 0x7f12d8103680 2024-04-24T08:27:06.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.517+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d80fc340 con 0x7f12d8103680 2024-04-24T08:27:06.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.518+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f12c4009a00 con 0x7f12d8103680 2024-04-24T08:27:06.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.519+0000 7f12d5600700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 0x7f12c0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.519+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f12c408e830 con 0x7f12d8103680 2024-04-24T08:27:06.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.519+0000 7f12dd200700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 0x7f12c0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.520+0000 7f12dd200700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 0x7f12c0070df0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009fd0 tx=0x7f12cc00b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.524+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f12c4057a20 con 0x7f12d8103680 2024-04-24T08:27:06.684 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:06 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:06.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:06 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:06.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.807+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12d8061190 con 0x7f12c006e940 2024-04-24T08:27:06.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.810+0000 7f12d5600700 1 -- 172.21.15.90:0/3262882138 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f12d8061190 con 0x7f12c006e940 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 msgr2=0x7f12c0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 0x7f12c0070df0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009fd0 tx=0x7f12cc00b560 comp rx=0 tx=0).stop 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 msgr2=0x7f12d8198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f12c400d8d0 tx=0x7f12c400dbe0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 shutdown_connections 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f12c006e940 0x7f12c0070df0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12d8100d60 0x7f12d8197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 --2- 172.21.15.90:0/3262882138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12d8103680 0x7f12d8198240 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 >> 172.21.15.90:0/3262882138 conn(0x7f12d80fa780 msgr2=0x7f12d8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:06.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 shutdown_connections 2024-04-24T08:27:06.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.813+0000 7f12dfc00700 1 -- 172.21.15.90:0/3262882138 wait complete. 2024-04-24T08:27:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 -- 172.21.15.90:0/4019384438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 msgr2=0x7fd30c073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 --2- 172.21.15.90:0/4019384438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c073ba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f4009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 -- 172.21.15.90:0/4019384438 shutdown_connections 2024-04-24T08:27:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 --2- 172.21.15.90:0/4019384438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c073ba0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 --2- 172.21.15.90:0/4019384438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 0x7fd30c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 -- 172.21.15.90:0/4019384438 >> 172.21.15.90:0/4019384438 conn(0x7fd30c0fb7f0 msgr2=0x7fd30c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 -- 172.21.15.90:0/4019384438 shutdown_connections 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.973+0000 7fd312200700 1 -- 172.21.15.90:0/4019384438 wait complete. 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.974+0000 7fd312200700 1 Processor -- start 2024-04-24T08:27:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd312200700 1 -- start start 2024-04-24T08:27:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd312200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd312200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 0x7fd30c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd312200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1987b0 con 0x7fd30c073730 2024-04-24T08:27:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd312200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c198920 con 0x7fd30c074d00 2024-04-24T08:27:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd30b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd30b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43532/0 (socket says 172.21.15.90:43532) 2024-04-24T08:27:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.975+0000 7fd30b400700 1 -- 172.21.15.90:0/2172083729 learned_addr learned my addr 172.21.15.90:0/2172083729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.976+0000 7fd302a00700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 0x7fd30c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.976+0000 7fd30b400700 1 -- 172.21.15.90:0/2172083729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 msgr2=0x7fd30c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.976+0000 7fd30b400700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 0x7fd30c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.976+0000 7fd30b400700 1 -- 172.21.15.90:0/2172083729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40095f0 con 0x7fd30c073730 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.976+0000 7fd30b400700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0060b0 tx=0x7fd2fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.977+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc00dda0 con 0x7fd30c073730 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.977+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc00f040 con 0x7fd30c073730 2024-04-24T08:27:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.977+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c19d3b0 con 0x7fd30c073730 2024-04-24T08:27:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.977+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc014790 con 0x7fd30c073730 2024-04-24T08:27:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.977+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c19d900 con 0x7fd30c073730 2024-04-24T08:27:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.978+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c10af70 con 0x7fd30c073730 2024-04-24T08:27:06.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.979+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fd2fc0148f0 con 0x7fd30c073730 2024-04-24T08:27:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.981+0000 7fd308c00700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 0x7fd2f0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.981+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd2fc08e3a0 con 0x7fd30c073730 2024-04-24T08:27:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.981+0000 7fd302a00700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 0x7fd2f0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:06.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.982+0000 7fd302a00700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 0x7fd2f0070df0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd30c199180 tx=0x7fd2f400b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:06.986+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fd2fc0576c0 con 0x7fd30c073730 2024-04-24T08:27:07.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.300+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd30c1077c0 con 0x7fd2f006e940 2024-04-24T08:27:07.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.317+0000 7fd308c00700 1 -- 172.21.15.90:0/2172083729 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd30c1077c0 con 0x7fd2f006e940 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 starting - - - - 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (14m) 23s ago 14m 8019k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (13m) 54s ago 12m 7902k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (14m) 23s ago 14m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (12m) 54s ago 12m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (11m) 23s ago 13m 75.6M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (6m) 23s ago 6m 238M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:27:07.320 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (6m) 23s ago 6m 15.0M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (6m) 54s ago 6m 255M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (6m) 54s ago 6m 14.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (3m) 23s ago 15m 502M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (108s) 54s ago 12m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (15m) 23s ago 16m 56.9M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (12m) 54s ago 12m 47.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (77s) 23s ago 14m 7554k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (66s) 54s ago 12m 2919k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:27:07.321 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (11m) 23s ago 11m 187M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (11m) 23s ago 11m 201M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (10m) 23s ago 10m 165M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (9m) 54s ago 9m 208M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (9m) 54s ago 9m 181M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (8m) 54s ago 8m 172M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (32s) 23s ago 13m 45.8M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:27:07.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.320+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 msgr2=0x7fd2f0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.321+0000 7fd312200700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 0x7fd2f0070df0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd30c199180 tx=0x7fd2f400b540 comp rx=0 tx=0).stop 2024-04-24T08:27:07.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.321+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 msgr2=0x7fd30c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.321+0000 7fd312200700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0060b0 tx=0x7fd2fc00d3b0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 shutdown_connections 2024-04-24T08:27:07.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd2f006e940 0x7fd2f0070df0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd30c073730 0x7fd30c197c70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 --2- 172.21.15.90:0/2172083729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd30c074d00 0x7fd30c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 >> 172.21.15.90:0/2172083729 conn(0x7fd30c0fb7f0 msgr2=0x7fd30c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:07.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 shutdown_connections 2024-04-24T08:27:07.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.325+0000 7fd312200700 1 -- 172.21.15.90:0/2172083729 wait complete. 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 -- 172.21.15.90:0/3564328099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 msgr2=0x7f0f201038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 --2- 172.21.15.90:0/3564328099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f201038d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009910 tx=0x7f0f0c009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 -- 172.21.15.90:0/3564328099 shutdown_connections 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 --2- 172.21.15.90:0/3564328099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f201038d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 --2- 172.21.15.90:0/3564328099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 0x7f0f20102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 -- 172.21.15.90:0/3564328099 >> 172.21.15.90:0/3564328099 conn(0x7f0f200fd8d0 msgr2=0x7f0f200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:07.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 -- 172.21.15.90:0/3564328099 shutdown_connections 2024-04-24T08:27:07.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.490+0000 7f0f1f200700 1 -- 172.21.15.90:0/3564328099 wait complete. 2024-04-24T08:27:07.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.491+0000 7f0f1f200700 1 Processor -- start 2024-04-24T08:27:07.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.491+0000 7f0f1f200700 1 -- start start 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 0x7f0f20197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201985e0 con 0x7f0f20103480 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20198720 con 0x7f0f20102280 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43554/0 (socket says 172.21.15.90:43554) 2024-04-24T08:27:07.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1d400700 1 -- 172.21.15.90:0/1259373370 learned_addr learned my addr 172.21.15.90:0/1259373370 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:07.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.492+0000 7f0f1de00700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 0x7f0f20197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:07.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.493+0000 7f0f1d400700 1 -- 172.21.15.90:0/1259373370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 msgr2=0x7f0f20197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.493+0000 7f0f1d400700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 0x7f0f20197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.493+0000 7f0f1d400700 1 -- 172.21.15.90:0/1259373370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0095f0 con 0x7f0f20103480 2024-04-24T08:27:07.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.493+0000 7f0f1d400700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b5c0 tx=0x7f0f0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.494+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f0c01d070 con 0x7f0f20103480 2024-04-24T08:27:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.494+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f0c022470 con 0x7f0f20103480 2024-04-24T08:27:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.494+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f0c00f690 con 0x7f0f20103480 2024-04-24T08:27:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.494+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2019d170 con 0x7f0f20103480 2024-04-24T08:27:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.494+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2019d660 con 0x7f0f20103480 2024-04-24T08:27:07.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.495+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2010add0 con 0x7f0f20103480 2024-04-24T08:27:07.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.496+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f0f0c0225e0 con 0x7f0f20103480 2024-04-24T08:27:07.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.497+0000 7f0f16a00700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 0x7f0f04070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:07.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.497+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0f0c08ffb0 con 0x7f0f20103480 2024-04-24T08:27:07.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.497+0000 7f0f1de00700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 0x7f0f04070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:07.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.498+0000 7f0f1de00700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 0x7f0f04070df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f201032e0 tx=0x7f0f1000a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:07.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.502+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f0f0c05cdf0 con 0x7f0f20103480 2024-04-24T08:27:07.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:07 smithi090 ceph-mon[25016]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:07.648 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:07 smithi090 ceph-mon[25016]: pgmap v86: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-24T08:27:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:07 smithi149 ceph-mon[31282]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:07.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:07 smithi149 ceph-mon[31282]: pgmap v86: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-24T08:27:07.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.866+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f2010afc0 con 0x7f0f20103480 2024-04-24T08:27:07.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.866+0000 7f0f16a00700 1 -- 172.21.15.90:0/1259373370 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f0f0c05c980 con 0x7f0f20103480 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:27:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:27:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:27:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:27:07.869 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 msgr2=0x7f0f04070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 0x7f0f04070df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f201032e0 tx=0x7f0f1000a380 comp rx=0 tx=0).stop 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 msgr2=0x7f0f20197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b5c0 tx=0x7f0f0c00bd70 comp rx=0 tx=0).stop 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 shutdown_connections 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f0f0406e940 0x7f0f04070df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f20102280 0x7f0f20197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 --2- 172.21.15.90:0/1259373370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103480 0x7f0f20197fc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 >> 172.21.15.90:0/1259373370 conn(0x7f0f200fd8d0 msgr2=0x7f0f201066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 shutdown_connections 2024-04-24T08:27:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:07.870+0000 7f0f1f200700 1 -- 172.21.15.90:0/1259373370 wait complete. 2024-04-24T08:27:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 -- 172.21.15.90:0/2413961999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 msgr2=0x7f15e40ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 --2- 172.21.15.90:0/2413961999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e40ff660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009910 tx=0x7f15d0009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 -- 172.21.15.90:0/2413961999 shutdown_connections 2024-04-24T08:27:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 --2- 172.21.15.90:0/2413961999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e40ff660 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 --2- 172.21.15.90:0/2413961999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 0x7f15e40fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 -- 172.21.15.90:0/2413961999 >> 172.21.15.90:0/2413961999 conn(0x7f15e40fa4d0 msgr2=0x7f15e40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 -- 172.21.15.90:0/2413961999 shutdown_connections 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.032+0000 7f15ea000700 1 -- 172.21.15.90:0/2413961999 wait complete. 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.033+0000 7f15ea000700 1 Processor -- start 2024-04-24T08:27:08.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15ea000700 1 -- start start 2024-04-24T08:27:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15ea000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 0x7f15e4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15ea000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15ea000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e41984e0 con 0x7f15e40ff1f0 2024-04-24T08:27:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15ea000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e4198620 con 0x7f15e40fe8a0 2024-04-24T08:27:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43572/0 (socket says 172.21.15.90:43572) 2024-04-24T08:27:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15e3e00700 1 -- 172.21.15.90:0/3360270722 learned_addr learned my addr 172.21.15.90:0/3360270722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.034+0000 7f15e8c00700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 0x7f15e4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e3e00700 1 -- 172.21.15.90:0/3360270722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 msgr2=0x7f15e4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e3e00700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 0x7f15e4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e3e00700 1 -- 172.21.15.90:0/3360270722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00095f0 con 0x7f15e40ff1f0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e3e00700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005b40 tx=0x7f15d000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d001d070 con 0x7f15e40ff1f0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15d0022470 con 0x7f15e40ff1f0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.035+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d000f690 con 0x7f15e40ff1f0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.036+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e419d080 con 0x7f15e40ff1f0 2024-04-24T08:27:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.036+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e419d540 con 0x7f15e40ff1f0 2024-04-24T08:27:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.037+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e410ad50 con 0x7f15e40ff1f0 2024-04-24T08:27:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.038+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f15d0022a10 con 0x7f15e40ff1f0 2024-04-24T08:27:08.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.039+0000 7f15e1600700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 0x7f15cc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:08.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.039+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f15d008ff30 con 0x7f15e40ff1f0 2024-04-24T08:27:08.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.039+0000 7f15e8c00700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 0x7f15cc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:08.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.040+0000 7f15e8c00700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 0x7f15cc070df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15d800ac50 tx=0x7f15d800a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:08.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.044+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f15d005cd80 con 0x7f15e40ff1f0 2024-04-24T08:27:08.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.369+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15e419dbd0 con 0x7f15cc06e940 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.372+0000 7f15e1600700 1 -- 172.21.15.90:0/3360270722 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f15e419dbd0 con 0x7f15cc06e940 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:27:08.373 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:27:08.374 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:27:08.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.375+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 msgr2=0x7f15cc070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:08.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.375+0000 7f15ea000700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 0x7f15cc070df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15d800ac50 tx=0x7f15d800a380 comp rx=0 tx=0).stop 2024-04-24T08:27:08.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.375+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 msgr2=0x7f15e4197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:08.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.375+0000 7f15ea000700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005b40 tx=0x7f15d000bd70 comp rx=0 tx=0).stop 2024-04-24T08:27:08.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.376+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 shutdown_connections 2024-04-24T08:27:08.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.376+0000 7f15ea000700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f15cc06e940 0x7f15cc070df0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.376+0000 7f15ea000700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15e40fe8a0 0x7f15e4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.376+0000 7f15ea000700 1 --2- 172.21.15.90:0/3360270722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15e40ff1f0 0x7f15e4197f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:08.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.376+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 >> 172.21.15.90:0/3360270722 conn(0x7f15e40fa4d0 msgr2=0x7f15e4106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:08.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.377+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 shutdown_connections 2024-04-24T08:27:08.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:08.377+0000 7f15ea000700 1 -- 172.21.15.90:0/3360270722 wait complete. 2024-04-24T08:27:08.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:08 smithi149 ceph-mon[31282]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:08.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:08 smithi149 ceph-mon[31282]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:08.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:08 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1259373370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:08.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:08 smithi090 ceph-mon[25016]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:08.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:08 smithi090 ceph-mon[25016]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:08.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:08 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1259373370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:09.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:09 smithi149 ceph-mon[31282]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:09.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:09 smithi149 ceph-mon[31282]: pgmap v87: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 857 KiB/s rd, 828 KiB/s wr, 147 op/s 2024-04-24T08:27:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:09 smithi090 ceph-mon[25016]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:09.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:09 smithi090 ceph-mon[25016]: pgmap v87: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 857 KiB/s rd, 828 KiB/s wr, 147 op/s 2024-04-24T08:27:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:11 smithi149 ceph-mon[31282]: pgmap v88: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 847 KiB/s rd, 818 KiB/s wr, 121 op/s 2024-04-24T08:27:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:11 smithi090 ceph-mon[25016]: pgmap v88: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 847 KiB/s rd, 818 KiB/s wr, 121 op/s 2024-04-24T08:27:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:12 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:12 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:12 smithi149 ceph-mon[31282]: pgmap v89: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-24T08:27:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:12 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:12 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:13.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:12 smithi090 ceph-mon[25016]: pgmap v89: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-24T08:27:15.182 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:14 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:15.182 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:14 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:15.182 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:14 smithi090 ceph-mon[25016]: pgmap v90: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 698 KiB/s rd, 734 KiB/s wr, 110 op/s 2024-04-24T08:27:15.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:14 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:14 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:15.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:14 smithi149 ceph-mon[31282]: pgmap v90: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 698 KiB/s rd, 734 KiB/s wr, 110 op/s 2024-04-24T08:27:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:17 smithi149 ceph-mon[31282]: pgmap v91: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 727 KiB/s rd, 811 KiB/s wr, 134 op/s 2024-04-24T08:27:17.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:17 smithi090 ceph-mon[25016]: pgmap v91: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 727 KiB/s rd, 811 KiB/s wr, 134 op/s 2024-04-24T08:27:19.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:19 smithi090 ceph-mon[25016]: pgmap v92: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 757 KiB/s rd, 808 KiB/s wr, 131 op/s 2024-04-24T08:27:19.499 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:19 smithi149 ceph-mon[31282]: pgmap v92: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 757 KiB/s rd, 808 KiB/s wr, 131 op/s 2024-04-24T08:27:20.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:20.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:21.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:21 smithi090 ceph-mon[25016]: pgmap v93: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 735 KiB/s wr, 103 op/s 2024-04-24T08:27:21.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:21 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:21.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:21 smithi149 ceph-mon[31282]: pgmap v93: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 735 KiB/s wr, 103 op/s 2024-04-24T08:27:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:21 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:22 smithi149 ceph-mon[31282]: Upgrade: Updating grafana.smithi090 2024-04-24T08:27:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:22 smithi149 ceph-mon[31282]: Deploying daemon grafana.smithi090 on smithi090 2024-04-24T08:27:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:22 smithi149 ceph-mon[31282]: pgmap v94: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-04-24T08:27:23.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:22 smithi090 ceph-mon[25016]: Upgrade: Updating grafana.smithi090 2024-04-24T08:27:23.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:22 smithi090 ceph-mon[25016]: Deploying daemon grafana.smithi090 on smithi090 2024-04-24T08:27:23.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:22 smithi090 ceph-mon[25016]: pgmap v94: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-04-24T08:27:25.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:25 smithi090 ceph-mon[25016]: pgmap v95: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 764 KiB/s rd, 754 KiB/s wr, 116 op/s 2024-04-24T08:27:25.879 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:25 smithi149 ceph-mon[31282]: pgmap v95: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 764 KiB/s rd, 754 KiB/s wr, 116 op/s 2024-04-24T08:27:25.893 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-24T08:27:25.894 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-24T08:27:27.212 DEBUG:teuthology.parallel:result is None 2024-04-24T08:27:27.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:27 smithi090 ceph-mon[25016]: pgmap v96: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 781 KiB/s rd, 840 KiB/s wr, 140 op/s 2024-04-24T08:27:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:27 smithi149 ceph-mon[31282]: pgmap v96: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 781 KiB/s rd, 840 KiB/s wr, 140 op/s 2024-04-24T08:27:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:29 smithi090 ceph-mon[25016]: pgmap v97: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-04-24T08:27:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:29 smithi149 ceph-mon[31282]: pgmap v97: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-04-24T08:27:31.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:31 smithi090 ceph-mon[25016]: pgmap v98: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 688 KiB/s rd, 732 KiB/s wr, 121 op/s 2024-04-24T08:27:32.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:31 smithi149 ceph-mon[31282]: pgmap v98: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 688 KiB/s rd, 732 KiB/s wr, 121 op/s 2024-04-24T08:27:33.147 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:32 smithi090 ceph-mon[25016]: pgmap v99: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 914 KiB/s rd, 931 KiB/s wr, 165 op/s 2024-04-24T08:27:33.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:32 smithi149 ceph-mon[31282]: pgmap v99: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 914 KiB/s rd, 931 KiB/s wr, 165 op/s 2024-04-24T08:27:35.771 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:35 smithi090 ceph-mon[25016]: pgmap v100: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 550 KiB/s rd, 571 KiB/s wr, 104 op/s 2024-04-24T08:27:35.771 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:35 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:35.896 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:35 smithi149 ceph-mon[31282]: pgmap v100: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 550 KiB/s rd, 571 KiB/s wr, 104 op/s 2024-04-24T08:27:35.896 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:35 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:37.589 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:37 smithi090 ceph-mon[25016]: pgmap v101: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 729 KiB/s wr, 128 op/s 2024-04-24T08:27:37.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:37 smithi149 ceph-mon[31282]: pgmap v101: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 729 KiB/s wr, 128 op/s 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.592+0000 7fd00b200700 1 -- 172.21.15.90:0/975980314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c1034e0 msgr2=0x7fd00c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.592+0000 7fd00b200700 1 --2- 172.21.15.90:0/975980314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c1034e0 0x7fd00c103950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009910 tx=0x7fcff4009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 -- 172.21.15.90:0/975980314 shutdown_connections 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 --2- 172.21.15.90:0/975980314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c1034e0 0x7fd00c103950 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 --2- 172.21.15.90:0/975980314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c102240 0x7fd00c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 -- 172.21.15.90:0/975980314 >> 172.21.15.90:0/975980314 conn(0x7fd00c0fd8d0 msgr2=0x7fd00c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:38.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 -- 172.21.15.90:0/975980314 shutdown_connections 2024-04-24T08:27:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.593+0000 7fd00b200700 1 -- 172.21.15.90:0/975980314 wait complete. 2024-04-24T08:27:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.594+0000 7fd00b200700 1 Processor -- start 2024-04-24T08:27:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.594+0000 7fd00b200700 1 -- start start 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd00b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd00b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c1034e0 0x7fd00c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd00b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c198580 con 0x7fd00c102240 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd00b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1986c0 con 0x7fd00c1034e0 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd009e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd009e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42968/0 (socket says 172.21.15.90:42968) 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd009e00700 1 -- 172.21.15.90:0/701674059 learned_addr learned my addr 172.21.15.90:0/701674059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.595+0000 7fd003e00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c1034e0 0x7fd00c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd009e00700 1 -- 172.21.15.90:0/701674059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c1034e0 msgr2=0x7fd00c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd009e00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c1034e0 0x7fd00c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd009e00700 1 -- 172.21.15.90:0/701674059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40095f0 con 0x7fd00c102240 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd009e00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00d8d0 tx=0x7fcffc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcffc0098a0 con 0x7fd00c102240 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.596+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcffc010460 con 0x7fd00c102240 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.597+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c19d170 con 0x7fd00c102240 2024-04-24T08:27:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.597+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c19d6c0 con 0x7fd00c102240 2024-04-24T08:27:38.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.597+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcffc00f5f0 con 0x7fd00c102240 2024-04-24T08:27:38.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.598+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c10ad20 con 0x7fd00c102240 2024-04-24T08:27:38.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.600+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fcffc009a00 con 0x7fd00c102240 2024-04-24T08:27:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.601+0000 7fd002a00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 0x7fcff0070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.601+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcffc08e150 con 0x7fd00c102240 2024-04-24T08:27:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.601+0000 7fd003e00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 0x7fcff0070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:38.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.602+0000 7fd003e00700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 0x7fcff0070e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006010 tx=0x7fcff400b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.607+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fcffc05af90 con 0x7fd00c102240 2024-04-24T08:27:38.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.915+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd00c107e30 con 0x7fcff006e990 2024-04-24T08:27:38.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.919+0000 7fd002a00700 1 -- 172.21.15.90:0/701674059 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fd00c107e30 con 0x7fcff006e990 2024-04-24T08:27:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 msgr2=0x7fcff0070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 0x7fcff0070e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006010 tx=0x7fcff400b560 comp rx=0 tx=0).stop 2024-04-24T08:27:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 msgr2=0x7fd00c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00d8d0 tx=0x7fcffc00dbe0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 shutdown_connections 2024-04-24T08:27:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fcff006e990 0x7fcff0070e40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd00c102240 0x7fd00c197a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 --2- 172.21.15.90:0/701674059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd00c1034e0 0x7fd00c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:38.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.923+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 >> 172.21.15.90:0/701674059 conn(0x7fd00c0fd8d0 msgr2=0x7fd00c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:38.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.926+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 shutdown_connections 2024-04-24T08:27:38.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:38.926+0000 7fd00b200700 1 -- 172.21.15.90:0/701674059 wait complete. 2024-04-24T08:27:38.958 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 -- 172.21.15.90:0/2094184676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 msgr2=0x7fecc8102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 --2- 172.21.15.90:0/2094184676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8102860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009910 tx=0x7fecb8009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 -- 172.21.15.90:0/2094184676 shutdown_connections 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 --2- 172.21.15.90:0/2094184676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 0x7fecc8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 --2- 172.21.15.90:0/2094184676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8102860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 -- 172.21.15.90:0/2094184676 >> 172.21.15.90:0/2094184676 conn(0x7fecc80fdac0 msgr2=0x7fecc80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:39.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 -- 172.21.15.90:0/2094184676 shutdown_connections 2024-04-24T08:27:39.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.100+0000 7feccd200700 1 -- 172.21.15.90:0/2094184676 wait complete. 2024-04-24T08:27:39.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.101+0000 7feccd200700 1 Processor -- start 2024-04-24T08:27:39.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7feccd200700 1 -- start start 2024-04-24T08:27:39.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7feccd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7feccd200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 0x7fecc81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7feccd200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81987a0 con 0x7fecc8102450 2024-04-24T08:27:39.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7feccd200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc8198910 con 0x7fecc81036f0 2024-04-24T08:27:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7fecc5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7fecc5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42980/0 (socket says 172.21.15.90:42980) 2024-04-24T08:27:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.102+0000 7fecc5e00700 1 -- 172.21.15.90:0/3292570738 learned_addr learned my addr 172.21.15.90:0/3292570738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecc5400700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 0x7fecc81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecc5e00700 1 -- 172.21.15.90:0/3292570738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 msgr2=0x7fecc81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecc5e00700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 0x7fecc81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecc5e00700 1 -- 172.21.15.90:0/3292570738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80095f0 con 0x7fecc8102450 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecc5e00700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005b40 tx=0x7fecb800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.103+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecb801d070 con 0x7fecc8102450 2024-04-24T08:27:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.104+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fecb8022470 con 0x7fecc8102450 2024-04-24T08:27:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.104+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecb800f690 con 0x7fecc8102450 2024-04-24T08:27:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.104+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc819d340 con 0x7fecc8102450 2024-04-24T08:27:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.104+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc819d830 con 0x7fecc8102450 2024-04-24T08:27:39.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.105+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc810af30 con 0x7fecc8102450 2024-04-24T08:27:39.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.106+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fecb80225e0 con 0x7fecc8102450 2024-04-24T08:27:39.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.107+0000 7fecbea00700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 0x7fecac070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.107+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fecb80900a0 con 0x7fecc8102450 2024-04-24T08:27:39.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.108+0000 7fecc5400700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 0x7fecac070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.108+0000 7fecc5400700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 0x7fecac070df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005fd0 tx=0x7fecb400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:39.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.112+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fecb8059390 con 0x7fecc8102450 2024-04-24T08:27:39.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.413+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecc8108040 con 0x7fecac06e940 2024-04-24T08:27:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.415+0000 7fecbea00700 1 -- 172.21.15.90:0/3292570738 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fecc8108040 con 0x7fecac06e940 2024-04-24T08:27:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 msgr2=0x7fecac070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 0x7fecac070df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005fd0 tx=0x7fecb400a560 comp rx=0 tx=0).stop 2024-04-24T08:27:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 msgr2=0x7fecc8197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005b40 tx=0x7fecb800be20 comp rx=0 tx=0).stop 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 shutdown_connections 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fecac06e940 0x7fecac070df0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecc8102450 0x7fecc8197c60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 --2- 172.21.15.90:0/3292570738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc81036f0 0x7fecc81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.418+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 >> 172.21.15.90:0/3292570738 conn(0x7fecc80fdac0 msgr2=0x7fecc8106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.419+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 shutdown_connections 2024-04-24T08:27:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.419+0000 7feccd200700 1 -- 172.21.15.90:0/3292570738 wait complete. 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 -- 172.21.15.90:0/3885240679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320100240 msgr2=0x7f1320100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 --2- 172.21.15.90:0/3885240679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320100240 0x7f1320100650 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1310009910 tx=0x7f1310009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 -- 172.21.15.90:0/3885240679 shutdown_connections 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 --2- 172.21.15.90:0/3885240679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320101440 0x7f1320101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 --2- 172.21.15.90:0/3885240679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320100240 0x7f1320100650 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 -- 172.21.15.90:0/3885240679 >> 172.21.15.90:0/3885240679 conn(0x7f13200fb7f0 msgr2=0x7f13200fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:39.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.611+0000 7f1324c00700 1 -- 172.21.15.90:0/3885240679 shutdown_connections 2024-04-24T08:27:39.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.612+0000 7f1324c00700 1 -- 172.21.15.90:0/3885240679 wait complete. 2024-04-24T08:27:39.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 Processor -- start 2024-04-24T08:27:39.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 -- start start 2024-04-24T08:27:39.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320100240 0x7f1320074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1320073680 con 0x7f1320101440 2024-04-24T08:27:39.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.613+0000 7f1324c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200737c0 con 0x7f1320100240 2024-04-24T08:27:39.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43002/0 (socket says 172.21.15.90:43002) 2024-04-24T08:27:39.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 -- 172.21.15.90:0/516903589 learned_addr learned my addr 172.21.15.90:0/516903589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131de00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320100240 0x7f1320074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 -- 172.21.15.90:0/516903589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320100240 msgr2=0x7f1320074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320100240 0x7f1320074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 -- 172.21.15.90:0/516903589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100095f0 con 0x7f1320101440 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.614+0000 7f131d400700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f130c00c930 tx=0x7f130c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.615+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c007ad0 con 0x7f1320101440 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.615+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f130c007c30 con 0x7f1320101440 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.615+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320073aa0 con 0x7f1320101440 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.615+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c018630 con 0x7f1320101440 2024-04-24T08:27:39.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.615+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132019db10 con 0x7f1320101440 2024-04-24T08:27:39.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.616+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1320108d90 con 0x7f1320101440 2024-04-24T08:27:39.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.619+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f130c00f450 con 0x7f1320101440 2024-04-24T08:27:39.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.620+0000 7f1316a00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 0x7f1304070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:39.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.620+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f130c08eaa0 con 0x7f1320101440 2024-04-24T08:27:39.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.620+0000 7f131de00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 0x7f1304070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:39.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.621+0000 7f131de00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 0x7f1304070df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f1310005920 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:39.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.625+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f130c05b840 con 0x7f1320101440 2024-04-24T08:27:39.788 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:39 smithi090 ceph-mon[25016]: pgmap v102: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 705 KiB/s rd, 649 KiB/s wr, 115 op/s 2024-04-24T08:27:39.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.892+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1320105d90 con 0x7f130406e940 2024-04-24T08:27:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.922+0000 7f1316a00700 1 -- 172.21.15.90:0/516903589 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1320105d90 con 0x7f130406e940 2024-04-24T08:27:39.924 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:27:39.924 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (38s) 28s ago 14m 13.5M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:27:39.924 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (15m) 28s ago 15m 8052k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:27:39.924 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (13m) 87s ago 13m 7902k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (14m) 28s ago 14m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (13m) 87s ago 13m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (12m) 28s ago 14m 75.9M - 9.4.7 954c08fa6188 f1c47e8da09f 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (7m) 28s ago 7m 173M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (7m) 28s ago 7m 15.2M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (7m) 87s ago 7m 255M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (7m) 86s ago 7m 14.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (3m) 28s ago 16m 502M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (2m) 87s ago 13m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (16m) 28s ago 16m 57.4M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (13m) 87s ago 13m 47.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (110s) 28s ago 14m 7398k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (99s) 87s ago 13m 2919k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (12m) 28s ago 12m 204M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:27:39.925 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (11m) 28s ago 11m 215M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:27:39.926 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (11m) 28s ago 10m 178M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:27:39.926 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (10m) 87s ago 10m 208M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:27:39.926 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (9m) 87s ago 9m 181M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:27:39.926 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (9m) 87s ago 9m 172M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:27:39.926 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (64s) 28s ago 13m 53.6M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:27:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 msgr2=0x7f1304070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 0x7f1304070df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f1310005920 comp rx=0 tx=0).stop 2024-04-24T08:27:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 msgr2=0x7f1320073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f130c00c930 tx=0x7f130c00ccf0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 shutdown_connections 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f130406e940 0x7f1304070df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1320100240 0x7f1320074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 --2- 172.21.15.90:0/516903589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1320101440 0x7f1320073140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 >> 172.21.15.90:0/516903589 conn(0x7f13200fb7f0 msgr2=0x7f1320104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 shutdown_connections 2024-04-24T08:27:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:39.926+0000 7f1324c00700 1 -- 172.21.15.90:0/516903589 wait complete. 2024-04-24T08:27:40.004 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:39 smithi149 ceph-mon[31282]: pgmap v102: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 705 KiB/s rd, 649 KiB/s wr, 115 op/s 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 -- 172.21.15.90:0/1023536500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c107870 msgr2=0x7f7e7c107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 --2- 172.21.15.90:0/1023536500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c107870 0x7f7e7c107cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009910 tx=0x7f7e68009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 -- 172.21.15.90:0/1023536500 shutdown_connections 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 --2- 172.21.15.90:0/1023536500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c107870 0x7f7e7c107cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 --2- 172.21.15.90:0/1023536500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e7c106670 0x7f7e7c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 -- 172.21.15.90:0/1023536500 >> 172.21.15.90:0/1023536500 conn(0x7f7e7c075740 msgr2=0x7f7e7c077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 -- 172.21.15.90:0/1023536500 shutdown_connections 2024-04-24T08:27:40.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.096+0000 7f7e82e00700 1 -- 172.21.15.90:0/1023536500 wait complete. 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 Processor -- start 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 -- start start 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e7c107870 0x7f7e7c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c19c940 con 0x7f7e7c106670 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.097+0000 7f7e82e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c19ca80 con 0x7f7e7c107870 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33442/0 (socket says 172.21.15.90:33442) 2024-04-24T08:27:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 -- 172.21.15.90:0/2789403699 learned_addr learned my addr 172.21.15.90:0/2789403699 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 -- 172.21.15.90:0/2789403699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e7c107870 msgr2=0x7f7e7c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e7c107870 0x7f7e7c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.098+0000 7f7e81a00700 1 -- 172.21.15.90:0/2789403699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680095f0 con 0x7f7e7c106670 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e81a00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c910 tx=0x7f7e7000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e700041f0 con 0x7f7e7c106670 2024-04-24T08:27:40.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e70007db0 con 0x7f7e7c106670 2024-04-24T08:27:40.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c1a1540 con 0x7f7e7c106670 2024-04-24T08:27:40.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e70003d60 con 0x7f7e7c106670 2024-04-24T08:27:40.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.099+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c1a1a60 con 0x7f7e7c106670 2024-04-24T08:27:40.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.100+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c10f1c0 con 0x7f7e7c106670 2024-04-24T08:27:40.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.101+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f7e70007900 con 0x7f7e7c106670 2024-04-24T08:27:40.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.102+0000 7f7e7a000700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 0x7f7e64070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.102+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7e7008e640 con 0x7f7e7c106670 2024-04-24T08:27:40.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.103+0000 7f7e81000700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 0x7f7e64070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:40.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.103+0000 7f7e81000700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 0x7f7e64070df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005090 tx=0x7f7e68005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:40.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.107+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f7e7005b400 con 0x7f7e7c106670 2024-04-24T08:27:40.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.562+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e7c10f380 con 0x7f7e7c106670 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.563+0000 7f7e7a000700 1 -- 172.21.15.90:0/2789403699 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f7e7005af90 con 0x7f7e7c106670 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:27:40.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:27:40.565 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:27:40.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 msgr2=0x7f7e64070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 0x7f7e64070df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005090 tx=0x7f7e68005fb0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 msgr2=0x7f7e7c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c910 tx=0x7f7e7000ccd0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 shutdown_connections 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f7e6406e940 0x7f7e64070df0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c106670 0x7f7e7c19be70 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c910 tx=0x7f7e7000ccd0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 --2- 172.21.15.90:0/2789403699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e7c107870 0x7f7e7c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.567+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 >> 172.21.15.90:0/2789403699 conn(0x7f7e7c075740 msgr2=0x7f7e7c10aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.568+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 shutdown_connections 2024-04-24T08:27:40.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.568+0000 7f7e82e00700 1 -- 172.21.15.90:0/2789403699 wait complete. 2024-04-24T08:27:40.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:40 smithi090 ceph-mon[25016]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:40.738 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:40 smithi090 ceph-mon[25016]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:40.739 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:40.739 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:40.739 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:40 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:40.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.736+0000 7fd9ff600700 1 -- 172.21.15.90:0/2865997860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 msgr2=0x7fd9f8103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.736+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2865997860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8103140 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009910 tx=0x7fd9e8009c20 comp rx=0 tx=0).stop 2024-04-24T08:27:40.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 -- 172.21.15.90:0/2865997860 shutdown_connections 2024-04-24T08:27:40.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2865997860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2865997860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8103140 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 -- 172.21.15.90:0/2865997860 >> 172.21.15.90:0/2865997860 conn(0x7fd9f80fa780 msgr2=0x7fd9f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 -- 172.21.15.90:0/2865997860 shutdown_connections 2024-04-24T08:27:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.737+0000 7fd9ff600700 1 -- 172.21.15.90:0/2865997860 wait complete. 2024-04-24T08:27:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.738+0000 7fd9ff600700 1 Processor -- start 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.738+0000 7fd9ff600700 1 -- start start 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9ff600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9ff600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f81922d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9ff600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f818e880 con 0x7fd9f8100d60 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9ff600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f818e9f0 con 0x7fd9f8103680 2024-04-24T08:27:40.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9fcc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9fcc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33456/0 (socket says 172.21.15.90:33456) 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.739+0000 7fd9fcc00700 1 -- 172.21.15.90:0/2812946665 learned_addr learned my addr 172.21.15.90:0/2812946665 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9f7e00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f81922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9fcc00700 1 -- 172.21.15.90:0/2812946665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 msgr2=0x7fd9f81922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9fcc00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f81922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9fcc00700 1 -- 172.21.15.90:0/2812946665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e80095f0 con 0x7fd9f8100d60 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9f7e00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f81922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:27:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.740+0000 7fd9fcc00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800bda0 tx=0x7fd9e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.741+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9e801d070 con 0x7fd9f8100d60 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.741+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd9e800f460 con 0x7fd9f8100d60 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.741+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f818ec70 con 0x7fd9f8100d60 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.741+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f818f160 con 0x7fd9f8100d60 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.742+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9e8003c30 con 0x7fd9f8100d60 2024-04-24T08:27:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.742+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f80fc340 con 0x7fd9f8100d60 2024-04-24T08:27:40.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.745+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fd9e800fa00 con 0x7fd9f8100d60 2024-04-24T08:27:40.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.746+0000 7fd9f5600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 0x7fd9e0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:27:40.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.746+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd9e808fd00 con 0x7fd9f8100d60 2024-04-24T08:27:40.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.747+0000 7fd9f7e00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 0x7fd9e0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:27:40.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.748+0000 7fd9f7e00700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 0x7fd9e0070df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec005950 tx=0x7fd9ec0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:27:40.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:40.749+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fd9e805ca20 con 0x7fd9f8100d60 2024-04-24T08:27:40.751 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-24T08:27:40.752 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T08:27:41.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:40 smithi149 ceph-mon[31282]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:41.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:40 smithi149 ceph-mon[31282]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:41.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:41.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:41.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:40 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:41.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.061+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f8061190 con 0x7fd9e006e940 2024-04-24T08:27:41.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.066+0000 7fd9f5600700 1 -- 172.21.15.90:0/2812946665 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fd9f8061190 con 0x7fd9e006e940 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading grafana daemons", 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:27:41.068 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.069+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 msgr2=0x7fd9e0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.069+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 0x7fd9e0070df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec005950 tx=0x7fd9ec0058e0 comp rx=0 tx=0).stop 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.069+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 msgr2=0x7fd9f8191d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.069+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800bda0 tx=0x7fd9e800be80 comp rx=0 tx=0).stop 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 shutdown_connections 2024-04-24T08:27:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd9e006e940 0x7fd9e0070df0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9f8100d60 0x7fd9f8191d90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 --2- 172.21.15.90:0/2812946665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9f8103680 0x7fd9f81922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:27:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 >> 172.21.15.90:0/2812946665 conn(0x7fd9f80fa780 msgr2=0x7fd9f80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:27:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 shutdown_connections 2024-04-24T08:27:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:27:41.070+0000 7fd9ff600700 1 -- 172.21.15.90:0/2812946665 wait complete. 2024-04-24T08:27:41.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:41 smithi090 ceph-mon[25016]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:41.782 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:41 smithi090 ceph-mon[25016]: pgmap v103: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 397 KiB/s rd, 362 KiB/s wr, 79 op/s 2024-04-24T08:27:41.783 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:41 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2789403699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:42.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:41 smithi149 ceph-mon[31282]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:42.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:41 smithi149 ceph-mon[31282]: pgmap v103: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 397 KiB/s rd, 362 KiB/s wr, 79 op/s 2024-04-24T08:27:42.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:41 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2789403699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:42.072 DEBUG:teuthology.parallel:result is None 2024-04-24T08:27:42.073 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T08:27:42.167 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T08:27:42.167 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T08:27:42.279 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T08:27:42.279 DEBUG:teuthology.parallel:result is None 2024-04-24T08:27:42.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:42 smithi090 ceph-mon[25016]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:43.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:42 smithi149 ceph-mon[31282]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:27:43.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:43 smithi090 ceph-mon[25016]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 619 KiB/s rd, 578 KiB/s wr, 110 op/s 2024-04-24T08:27:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:43 smithi149 ceph-mon[31282]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 619 KiB/s rd, 578 KiB/s wr, 110 op/s 2024-04-24T08:27:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:45 smithi149 ceph-mon[31282]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 393 KiB/s rd, 380 KiB/s wr, 65 op/s 2024-04-24T08:27:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:45 smithi090 ceph-mon[25016]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 393 KiB/s rd, 380 KiB/s wr, 65 op/s 2024-04-24T08:27:47.847 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:47 smithi090 ceph-mon[25016]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 406 KiB/s rd, 393 KiB/s wr, 72 op/s 2024-04-24T08:27:48.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:47 smithi149 ceph-mon[31282]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 406 KiB/s rd, 393 KiB/s wr, 72 op/s 2024-04-24T08:27:49.909 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:49 smithi090 ceph-mon[25016]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 239 KiB/s rd, 234 KiB/s wr, 49 op/s 2024-04-24T08:27:50.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:49 smithi149 ceph-mon[31282]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 239 KiB/s rd, 234 KiB/s wr, 49 op/s 2024-04-24T08:27:51.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:50 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:51.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:50 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:27:52.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:51 smithi149 ceph-mon[31282]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 229 KiB/s wr, 38 op/s 2024-04-24T08:27:52.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:52.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:51 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:52.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:51 smithi090 ceph-mon[25016]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 229 KiB/s wr, 38 op/s 2024-04-24T08:27:52.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:52.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:51 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:52 smithi090 ceph-mon[25016]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 230 KiB/s wr, 40 op/s 2024-04-24T08:27:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:52 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:53.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:52 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:52 smithi149 ceph-mon[31282]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 230 KiB/s wr, 40 op/s 2024-04-24T08:27:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:52 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:52 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:55.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:55 smithi149 ceph-mon[31282]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-24T08:27:55.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:55 smithi090 ceph-mon[25016]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-24T08:27:57.891 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-24T08:27:57.891 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:57.892 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:57.892 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:57.892 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:57.892 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T08:27:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T08:27:59.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:59.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:59.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:59.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:59.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T08:27:59.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T08:27:59.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:27:59.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:59 smithi149 ceph-mon[31282]: Upgrade: Finalizing container_image settings 2024-04-24T08:28:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:59 smithi149 ceph-mon[31282]: Upgrade: Complete! 2024-04-24T08:28:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:27:59 smithi149 ceph-mon[31282]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 3 op/s 2024-04-24T08:28:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:59 smithi090 ceph-mon[25016]: Upgrade: Finalizing container_image settings 2024-04-24T08:28:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:59 smithi090 ceph-mon[25016]: Upgrade: Complete! 2024-04-24T08:28:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:27:59 smithi090 ceph-mon[25016]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 3 op/s 2024-04-24T08:28:02.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:01 smithi149 ceph-mon[31282]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-04-24T08:28:02.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:01 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:02.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:01 smithi090 ceph-mon[25016]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-04-24T08:28:02.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:01 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:03.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:02 smithi149 ceph-mon[31282]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-24T08:28:03.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:02 smithi090 ceph-mon[25016]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-24T08:28:05.981 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:05 smithi149 ceph-mon[31282]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:05.981 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:05 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:06.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:05 smithi090 ceph-mon[25016]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:06.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:05 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:08.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:07 smithi149 ceph-mon[31282]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:08.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:07 smithi090 ceph-mon[25016]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:09 smithi149 ceph-mon[31282]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:09 smithi090 ceph-mon[25016]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:11.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 -- 172.21.15.90:0/2621687197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1018e0 msgr2=0x7f5bcc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2621687197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc101cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009910 tx=0x7f5bc0009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 -- 172.21.15.90:0/2621687197 shutdown_connections 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2621687197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc102230 0x7f5bcc0fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2621687197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc101cf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 -- 172.21.15.90:0/2621687197 >> 172.21.15.90:0/2621687197 conn(0x7f5bcc0f9f10 msgr2=0x7f5bcc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 -- 172.21.15.90:0/2621687197 shutdown_connections 2024-04-24T08:28:11.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.243+0000 7f5bd4000700 1 -- 172.21.15.90:0/2621687197 wait complete. 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.244+0000 7f5bd4000700 1 Processor -- start 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.244+0000 7f5bd4000700 1 -- start start 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd4000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd4000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd4000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1988b0 con 0x7f5bcc102230 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd4000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1989f0 con 0x7f5bcc1018e0 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33128/0 (socket says 172.21.15.90:33128) 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd0c00700 1 -- 172.21.15.90:0/2914131362 learned_addr learned my addr 172.21.15.90:0/2914131362 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd1600700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd0c00700 1 -- 172.21.15.90:0/2914131362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1018e0 msgr2=0x7f5bcc197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.245+0000 7f5bd0c00700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bd0c00700 1 -- 172.21.15.90:0/2914131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00095f0 con 0x7f5bcc102230 2024-04-24T08:28:11.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bd0c00700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800d900 tx=0x7f5bb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb8004a00 con 0x7f5bcc102230 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bb800b5e0 con 0x7f5bcc102230 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb8009b50 con 0x7f5bcc102230 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc19d4a0 con 0x7f5bcc102230 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.246+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc19d9f0 con 0x7f5bcc102230 2024-04-24T08:28:11.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.247+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc105760 con 0x7f5bcc102230 2024-04-24T08:28:11.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.248+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f5bb8004c60 con 0x7f5bcc102230 2024-04-24T08:28:11.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.249+0000 7f5bca000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 0x7f5bb4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:11.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.250+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5bb808e3e0 con 0x7f5bcc102230 2024-04-24T08:28:11.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.250+0000 7f5bd1600700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 0x7f5bb4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:11.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.251+0000 7f5bd1600700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 0x7f5bb4070df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006010 tx=0x7f5bc000b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:11.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.255+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f5bb805b180 con 0x7f5bcc102230 2024-04-24T08:28:11.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.507+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bcc061190 con 0x7f5bb406e940 2024-04-24T08:28:11.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.509+0000 7f5bca000700 1 -- 172.21.15.90:0/2914131362 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5bcc061190 con 0x7f5bb406e940 2024-04-24T08:28:11.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 msgr2=0x7f5bb4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 0x7f5bb4070df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006010 tx=0x7f5bc000b540 comp rx=0 tx=0).stop 2024-04-24T08:28:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 msgr2=0x7f5bcc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800d900 tx=0x7f5bb800dcc0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 shutdown_connections 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5bb406e940 0x7f5bb4070df0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1018e0 0x7f5bcc197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 --2- 172.21.15.90:0/2914131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc102230 0x7f5bcc198290 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 >> 172.21.15.90:0/2914131362 conn(0x7f5bcc0f9f10 msgr2=0x7f5bcc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 shutdown_connections 2024-04-24T08:28:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:11.512+0000 7f5bd4000700 1 -- 172.21.15.90:0/2914131362 wait complete. 2024-04-24T08:28:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:11 smithi090 ceph-mon[25016]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:12.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:11 smithi149 ceph-mon[31282]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:12.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-24T08:28:12.709 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:12.856 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:12 smithi090 ceph-mon[25016]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:13.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:12 smithi149 ceph-mon[31282]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:13 smithi090 ceph-mon[25016]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:13 smithi149 ceph-mon[31282]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.841+0000 7fd0c2600700 1 -- 172.21.15.90:0/987210853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc10c8f0 msgr2=0x7fd0bc10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.841+0000 7fd0c2600700 1 --2- 172.21.15.90:0/987210853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc10ccc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b0a0 tx=0x7fd0ac00b3b0 comp rx=0 tx=0).stop 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 -- 172.21.15.90:0/987210853 shutdown_connections 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 --2- 172.21.15.90:0/987210853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc106b80 0x7fd0bc106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 --2- 172.21.15.90:0/987210853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc10ccc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:14.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 -- 172.21.15.90:0/987210853 >> 172.21.15.90:0/987210853 conn(0x7fd0bc076060 msgr2=0x7fd0bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 -- 172.21.15.90:0/987210853 shutdown_connections 2024-04-24T08:28:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.842+0000 7fd0c2600700 1 -- 172.21.15.90:0/987210853 wait complete. 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 Processor -- start 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 -- start start 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc19d060 con 0x7fd0bc106b80 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.843+0000 7fd0c2600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc1a0df0 con 0x7fd0bc10c8f0 2024-04-24T08:28:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33148/0 (socket says 172.21.15.90:33148) 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 -- 172.21.15.90:0/2457339653 learned_addr learned my addr 172.21.15.90:0/2457339653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0b2a00700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 -- 172.21.15.90:0/2457339653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc10c8f0 msgr2=0x7fd0bc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 -- 172.21.15.90:0/2457339653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009d00 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.844+0000 7fd0bb400700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b070 tx=0x7fd0ac00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.845+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ac00e040 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.845+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0ac003e60 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.845+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc1a1070 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.845+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc1a1560 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.845+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ac01d8b0 con 0x7fd0bc106b80 2024-04-24T08:28:14.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.846+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc109b20 con 0x7fd0bc106b80 2024-04-24T08:28:14.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.847+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fd0ac004820 con 0x7fd0bc106b80 2024-04-24T08:28:14.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.848+0000 7fd0b8c00700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 0x7fd0a0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:14.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.848+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd0ac090300 con 0x7fd0bc106b80 2024-04-24T08:28:14.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.848+0000 7fd0b2a00700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 0x7fd0a0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:14.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.849+0000 7fd0b2a00700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 0x7fd0a0070df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc19da60 tx=0x7fd0a4005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:14.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:14.853+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fd0ac0594f0 con 0x7fd0bc106b80 2024-04-24T08:28:15.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.178+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd0bc109500 con 0x7fd0bc106b80 2024-04-24T08:28:15.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.179+0000 7fd0b8c00700 1 -- 172.21.15.90:0/2457339653 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fd0ac05cbb0 con 0x7fd0bc106b80 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 msgr2=0x7fd0a0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 0x7fd0a0070df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc19da60 tx=0x7fd0a4005cf0 comp rx=0 tx=0).stop 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 msgr2=0x7fd0bc19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b070 tx=0x7fd0ac00cdd0 comp rx=0 tx=0).stop 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 shutdown_connections 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fd0a006e940 0x7fd0a0070df0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0bc106b80 0x7fd0bc19c440 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 --2- 172.21.15.90:0/2457339653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0bc10c8f0 0x7fd0bc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 >> 172.21.15.90:0/2457339653 conn(0x7fd0bc076060 msgr2=0x7fd0bc0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.182+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 shutdown_connections 2024-04-24T08:28:15.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:15.183+0000 7fd0c2600700 1 -- 172.21.15.90:0/2457339653 wait complete. 2024-04-24T08:28:15.198 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:28:15.901 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:15 smithi090 ceph-mon[25016]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:15.901 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:15 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2457339653' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:15.965 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-24T08:28:16.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:15 smithi149 ceph-mon[31282]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:16.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:15 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2457339653' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:16.355 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:17.736 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:17 smithi090 ceph-mon[25016]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:18.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:17 smithi149 ceph-mon[31282]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:18.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.536+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2873253562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc108470 msgr2=0x7f5dbc108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:18.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2873253562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc108470 0x7f5dbc108840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009910 tx=0x7f5db0009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:18.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2873253562 shutdown_connections 2024-04-24T08:28:18.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2873253562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc102470 0x7f5dbc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2873253562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc108470 0x7f5dbc108840 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2873253562 >> 172.21.15.90:0/2873253562 conn(0x7f5dbc0fdf90 msgr2=0x7f5dbc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2873253562 shutdown_connections 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.537+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2873253562 wait complete. 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.538+0000 7f5dc3c00700 1 Processor -- start 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.538+0000 7f5dc3c00700 1 -- start start 2024-04-24T08:28:18.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc3c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:18.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc3c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc108470 0x7f5dbc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:18.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc3c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc198e10 con 0x7f5dbc102470 2024-04-24T08:28:18.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc3c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc19cba0 con 0x7f5dbc108470 2024-04-24T08:28:18.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33162/0 (socket says 172.21.15.90:33162) 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 -- 172.21.15.90:0/2472566147 learned_addr learned my addr 172.21.15.90:0/2472566147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dbbe00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc108470 0x7f5dbc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 -- 172.21.15.90:0/2472566147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc108470 msgr2=0x7f5dbc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc108470 0x7f5dbc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.539+0000 7f5dc1200700 1 -- 172.21.15.90:0/2472566147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00095f0 con 0x7f5dbc102470 2024-04-24T08:28:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5dc1200700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5db0006010 tx=0x7f5db000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db001d070 con 0x7f5dbc102470 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5db0004fb0 con 0x7f5dbc102470 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db000f690 con 0x7f5dbc102470 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc19ce20 con 0x7f5dbc102470 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.540+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc19d310 con 0x7f5dbc102470 2024-04-24T08:28:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.541+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc10a9e0 con 0x7f5dbc102470 2024-04-24T08:28:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.543+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f5db00228a0 con 0x7f5dbc102470 2024-04-24T08:28:18.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.544+0000 7f5db9600700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 0x7f5da4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:18.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.544+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5db0090190 con 0x7f5dbc102470 2024-04-24T08:28:18.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.544+0000 7f5dbbe00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 0x7f5da4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:18.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.545+0000 7f5dbbe00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 0x7f5da4070df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005fd0 tx=0x7f5da800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:18.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.549+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f5db0059400 con 0x7f5dbc102470 2024-04-24T08:28:18.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.860+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5dbc04f2a0 con 0x7f5dbc102470 2024-04-24T08:28:18.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.861+0000 7f5db9600700 1 -- 172.21.15.90:0/2472566147 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f5db005cac0 con 0x7f5dbc102470 2024-04-24T08:28:18.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 msgr2=0x7f5da4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 0x7f5da4070df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005fd0 tx=0x7f5da800a560 comp rx=0 tx=0).stop 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 msgr2=0x7f5dbc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5db0006010 tx=0x7f5db000be20 comp rx=0 tx=0).stop 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 shutdown_connections 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f5da406e940 0x7f5da4070df0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dbc102470 0x7f5dbc1981f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 --2- 172.21.15.90:0/2472566147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dbc108470 0x7f5dbc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:18.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 >> 172.21.15.90:0/2472566147 conn(0x7f5dbc0fdf90 msgr2=0x7f5dbc1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:18.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 shutdown_connections 2024-04-24T08:28:18.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:18.864+0000 7f5dc3c00700 1 -- 172.21.15.90:0/2472566147 wait complete. 2024-04-24T08:28:18.880 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)" 2024-04-24T08:28:19.569 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-24T08:28:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:19 smithi090 ceph-mon[25016]: pgmap v122: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:19 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/2472566147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:19.970 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:20.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:19 smithi149 ceph-mon[31282]: pgmap v122: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:20.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:19 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/2472566147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:20.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:20 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:21.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:20 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:21.904 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:21 smithi090 ceph-mon[25016]: pgmap v123: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:22.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:21 smithi149 ceph-mon[31282]: pgmap v123: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:22.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.202+0000 7f1f01c00700 1 -- 172.21.15.90:0/3842843786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc106b80 msgr2=0x7f1efc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:22.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.202+0000 7f1f01c00700 1 --2- 172.21.15.90:0/3842843786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc106b80 0x7f1efc106ff0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009910 tx=0x7f1ee8009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 -- 172.21.15.90:0/3842843786 shutdown_connections 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 --2- 172.21.15.90:0/3842843786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc106b80 0x7f1efc106ff0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 --2- 172.21.15.90:0/3842843786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc10c8f0 0x7f1efc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 -- 172.21.15.90:0/3842843786 >> 172.21.15.90:0/3842843786 conn(0x7f1efc076060 msgr2=0x7f1efc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 -- 172.21.15.90:0/3842843786 shutdown_connections 2024-04-24T08:28:22.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.203+0000 7f1f01c00700 1 -- 172.21.15.90:0/3842843786 wait complete. 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 Processor -- start 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 -- start start 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc106b80 0x7f1efc19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc19d100 con 0x7f1efc10c8f0 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.204+0000 7f1f01c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a0e90 con 0x7f1efc106b80 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60694/0 (socket says 172.21.15.90:60694) 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 -- 172.21.15.90:0/1715614706 learned_addr learned my addr 172.21.15.90:0/1715614706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:22.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efaa00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc106b80 0x7f1efc19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 -- 172.21.15.90:0/1715614706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc106b80 msgr2=0x7f1efc19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc106b80 0x7f1efc19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.205+0000 7f1efa000700 1 -- 172.21.15.90:0/1715614706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80095f0 con 0x7f1efc10c8f0 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1efa000700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800b5c0 tx=0x7f1ee800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee801d070 con 0x7f1efc10c8f0 2024-04-24T08:28:22.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ee800f460 con 0x7f1efc10c8f0 2024-04-24T08:28:22.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc1a1110 con 0x7f1efc10c8f0 2024-04-24T08:28:22.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee8017610 con 0x7f1efc10c8f0 2024-04-24T08:28:22.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.206+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc1a1600 con 0x7f1efc10c8f0 2024-04-24T08:28:22.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.207+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc109b20 con 0x7f1efc10c8f0 2024-04-24T08:28:22.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.208+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f1ee800f5d0 con 0x7f1efc10c8f0 2024-04-24T08:28:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.209+0000 7f1ef3400700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 0x7f1ee0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.209+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1ee808ff10 con 0x7f1efc10c8f0 2024-04-24T08:28:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.210+0000 7f1efaa00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 0x7f1ee0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:22.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.210+0000 7f1efaa00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 0x7f1ee0070df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1eec006fd0 tx=0x7f1eec008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.215+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f1ee80591a0 con 0x7f1efc10c8f0 2024-04-24T08:28:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.529+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1efc04f2a0 con 0x7f1efc10c8f0 2024-04-24T08:28:22.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.530+0000 7f1ef3400700 1 -- 172.21.15.90:0/1715614706 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f1ee805c860 con 0x7f1efc10c8f0 2024-04-24T08:28:22.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 msgr2=0x7f1ee0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:22.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 0x7f1ee0070df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1eec006fd0 tx=0x7f1eec008040 comp rx=0 tx=0).stop 2024-04-24T08:28:22.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 msgr2=0x7f1efc19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800b5c0 tx=0x7f1ee800be00 comp rx=0 tx=0).stop 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 shutdown_connections 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f1ee006e940 0x7f1ee0070df0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1efc106b80 0x7f1efc19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 --2- 172.21.15.90:0/1715614706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1efc10c8f0 0x7f1efc19ca20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.533+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 >> 172.21.15.90:0/1715614706 conn(0x7f1efc076060 msgr2=0x7f1efc077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.534+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 shutdown_connections 2024-04-24T08:28:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:22.534+0000 7f1f01c00700 1 -- 172.21.15.90:0/1715614706 wait complete. 2024-04-24T08:28:22.551 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:28:23.211 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:22 smithi090 ceph-mon[25016]: pgmap v124: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:23.211 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:22 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1715614706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:22 smithi149 ceph-mon[31282]: pgmap v124: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:22 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1715614706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:23.600 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-24T08:28:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:25.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:25 smithi090 ceph-mon[25016]: pgmap v125: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:26.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:25 smithi149 ceph-mon[31282]: pgmap v125: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 -- 172.21.15.90:0/531209698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec0730f0 msgr2=0x7f65ec0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 --2- 172.21.15.90:0/531209698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec0730f0 0x7f65ec0734c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009910 tx=0x7f65e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 -- 172.21.15.90:0/531209698 shutdown_connections 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 --2- 172.21.15.90:0/531209698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec073a00 0x7f65ec10c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 --2- 172.21.15.90:0/531209698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec0730f0 0x7f65ec0734c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 -- 172.21.15.90:0/531209698 >> 172.21.15.90:0/531209698 conn(0x7f65ec0fbd80 msgr2=0x7f65ec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 -- 172.21.15.90:0/531209698 shutdown_connections 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.092+0000 7f65f3400700 1 -- 172.21.15.90:0/531209698 wait complete. 2024-04-24T08:28:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.093+0000 7f65f3400700 1 Processor -- start 2024-04-24T08:28:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65f3400700 1 -- start start 2024-04-24T08:28:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65f3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0730f0 0x7f65ec197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65f3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65f3400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec19c940 con 0x7f65ec073a00 2024-04-24T08:28:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65f3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec19cab0 con 0x7f65ec0730f0 2024-04-24T08:28:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65ebe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65ebe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60708/0 (socket says 172.21.15.90:60708) 2024-04-24T08:28:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.094+0000 7f65ebe00700 1 -- 172.21.15.90:0/3123220925 learned_addr learned my addr 172.21.15.90:0/3123220925 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65f0a00700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0730f0 0x7f65ec197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65ebe00700 1 -- 172.21.15.90:0/3123220925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0730f0 msgr2=0x7f65ec197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65ebe00700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0730f0 0x7f65ec197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65ebe00700 1 -- 172.21.15.90:0/3123220925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e00095f0 con 0x7f65ec073a00 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65ebe00700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c8a0 tx=0x7f65d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d800cea0 con 0x7f65ec073a00 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65d8004830 con 0x7f65ec073a00 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.095+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec19cd90 con 0x7f65ec073a00 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.096+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d80055b0 con 0x7f65ec073a00 2024-04-24T08:28:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.096+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec19d2e0 con 0x7f65ec073a00 2024-04-24T08:28:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.097+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec109c60 con 0x7f65ec073a00 2024-04-24T08:28:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.098+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f65d80049a0 con 0x7f65ec073a00 2024-04-24T08:28:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.099+0000 7f65e9600700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 0x7f65d4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.099+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f65d808e580 con 0x7f65ec073a00 2024-04-24T08:28:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.099+0000 7f65f0a00700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 0x7f65d4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.100+0000 7f65f0a00700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 0x7f65d4070df0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b5c0 tx=0x7f65e001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:26.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.104+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f65d80577f0 con 0x7f65ec073a00 2024-04-24T08:28:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:26.354+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}) v1 -- 0x7f65ec199360 con 0x7f65d406e940 2024-04-24T08:28:27.963 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:27 smithi090 ceph-mon[25016]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:27.963 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:27 smithi090 ceph-mon[25016]: pgmap v126: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:27 smithi149 ceph-mon[31282]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:27 smithi149 ceph-mon[31282]: pgmap v126: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:29.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.331+0000 7f65e9600700 1 -- 172.21.15.90:0/3123220925 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f65ec199360 con 0x7f65d406e940 2024-04-24T08:28:29.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 msgr2=0x7f65d4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:29.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 0x7f65d4070df0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b5c0 tx=0x7f65e001a040 comp rx=0 tx=0).stop 2024-04-24T08:28:29.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 msgr2=0x7f65ec198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:29.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c8a0 tx=0x7f65d800cc60 comp rx=0 tx=0).stop 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 shutdown_connections 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f65d406e940 0x7f65d4070df0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0730f0 0x7f65ec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 --2- 172.21.15.90:0/3123220925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65ec073a00 0x7f65ec198600 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 >> 172.21.15.90:0/3123220925 conn(0x7f65ec0fbd80 msgr2=0x7f65ec106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 shutdown_connections 2024-04-24T08:28:29.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:29.335+0000 7f65f3400700 1 -- 172.21.15.90:0/3123220925 wait complete. 2024-04-24T08:28:29.356 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:28:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:29 smithi090 ceph-mon[25016]: pgmap v127: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:29 smithi149 ceph-mon[31282]: pgmap v127: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:30.162 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch ps' 2024-04-24T08:28:30.590 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:31.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:31 smithi149 ceph-mon[31282]: pgmap v128: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:31.904 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:31 smithi090 ceph-mon[25016]: pgmap v128: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 -- 172.21.15.90:0/1909250055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 msgr2=0x7f0768106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 --2- 172.21.15.90:0/1909250055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f0768106ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0754009910 tx=0x7f0754009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 -- 172.21.15.90:0/1909250055 shutdown_connections 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 --2- 172.21.15.90:0/1909250055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f0768106ff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 --2- 172.21.15.90:0/1909250055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 0x7f076810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 -- 172.21.15.90:0/1909250055 >> 172.21.15.90:0/1909250055 conn(0x7f0768076060 msgr2=0x7f0768078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 -- 172.21.15.90:0/1909250055 shutdown_connections 2024-04-24T08:28:32.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.764+0000 7f076e000700 1 -- 172.21.15.90:0/1909250055 wait complete. 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.765+0000 7f076e000700 1 Processor -- start 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f076e000700 1 -- start start 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f076e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f076e000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 0x7f076819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f076e000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819d060 con 0x7f0768106b80 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f076e000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07681a0df0 con 0x7f076810c8f0 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f0767400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:32.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f0767400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38544/0 (socket says 172.21.15.90:38544) 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.766+0000 7f0767400700 1 -- 172.21.15.90:0/2584613802 learned_addr learned my addr 172.21.15.90:0/2584613802 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f0766a00700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 0x7f076819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f0767400700 1 -- 172.21.15.90:0/2584613802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 msgr2=0x7f076819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f0767400700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 0x7f076819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f0767400700 1 -- 172.21.15.90:0/2584613802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540095f0 con 0x7f0768106b80 2024-04-24T08:28:32.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f0767400700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f075800c8a0 tx=0x7f075800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f075800cea0 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0758004830 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07681a10d0 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.767+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07580055b0 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.768+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07681a1620 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.769+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076810ecc0 con 0x7f0768106b80 2024-04-24T08:28:32.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.770+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f07580049a0 con 0x7f0768106b80 2024-04-24T08:28:32.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.771+0000 7f075fe00700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 0x7f074c070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:32.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.771+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f075808e680 con 0x7f0768106b80 2024-04-24T08:28:32.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.771+0000 7f0766a00700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 0x7f074c070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:32.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.772+0000 7f0766a00700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 0x7f074c070df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0754000c00 tx=0x7f0754005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:32.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:32.776+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f0758057870 con 0x7f0768106b80 2024-04-24T08:28:33.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.018+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f07681a18a0 con 0x7f074c06e940 2024-04-24T08:28:33.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.049+0000 7f075fe00700 1 -- 172.21.15.90:0/2584613802 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f07681a18a0 con 0x7f074c06e940 2024-04-24T08:28:33.051 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:28:33.051 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (91s) 42s ago 15m 13.5M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:28:33.051 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (15m) 42s ago 15m 8098k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (14m) 2m ago 14m 7902k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (15m) 42s ago 15m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (14m) 2m ago 14m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (53s) 42s ago 15m 53.8M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (8m) 42s ago 8m 121M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (8m) 42s ago 8m 15.5M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (8m) 2m ago 8m 255M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (8m) 2m ago 8m 14.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (4m) 42s ago 17m 511M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (3m) 2m ago 14m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (17m) 42s ago 17m 55.5M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (14m) 2m ago 14m 47.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:28:33.052 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (2m) 42s ago 15m 7570k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (2m) 2m ago 14m 2919k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (13m) 42s ago 13m 215M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (12m) 42s ago 12m 224M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (11m) 42s ago 11m 188M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (11m) 2m ago 11m 208M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (10m) 2m ago 10m 181M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (10m) 2m ago 10m 172M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:28:33.053 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (117s) 42s ago 14m 57.6M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:28:33.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.053+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 msgr2=0x7f074c070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:33.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.053+0000 7f076e000700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 0x7f074c070df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0754000c00 tx=0x7f0754005fd0 comp rx=0 tx=0).stop 2024-04-24T08:28:33.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.053+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 msgr2=0x7f076819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:33.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.053+0000 7f076e000700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f075800c8a0 tx=0x7f075800cc60 comp rx=0 tx=0).stop 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 shutdown_connections 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f074c06e940 0x7f074c070df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0768106b80 0x7f076819c440 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 --2- 172.21.15.90:0/2584613802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f076810c8f0 0x7f076819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 >> 172.21.15.90:0/2584613802 conn(0x7f0768076060 msgr2=0x7f0768077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 shutdown_connections 2024-04-24T08:28:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:33.054+0000 7f076e000700 1 -- 172.21.15.90:0/2584613802 wait complete. 2024-04-24T08:28:33.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:32 smithi149 ceph-mon[31282]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:33.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:32 smithi090 ceph-mon[25016]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:34.163 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:33 smithi090 ceph-mon[25016]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:34.221 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T08:28:34.222 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:28:34.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-24T08:28:34.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:33 smithi149 ceph-mon[31282]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:34.634 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:35.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:34 smithi090 ceph-mon[25016]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:35.179 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:34 smithi149 ceph-mon[31282]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:36.218 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:35 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:35 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 -- 172.21.15.90:0/400405449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6860073a00 msgr2=0x7f686010c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 --2- 172.21.15.90:0/400405449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6860073a00 0x7f686010c560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f684c009910 tx=0x7f684c009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 -- 172.21.15.90:0/400405449 shutdown_connections 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 --2- 172.21.15.90:0/400405449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6860073a00 0x7f686010c560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 --2- 172.21.15.90:0/400405449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68600730f0 0x7f68600734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 -- 172.21.15.90:0/400405449 >> 172.21.15.90:0/400405449 conn(0x7f68600fbd80 msgr2=0x7f68600fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 -- 172.21.15.90:0/400405449 shutdown_connections 2024-04-24T08:28:36.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.746+0000 7f6865200700 1 -- 172.21.15.90:0/400405449 wait complete. 2024-04-24T08:28:36.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.747+0000 7f6865200700 1 Processor -- start 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f6865200700 1 -- start start 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f6865200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f6865200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6860073a00 0x7f68601987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f6865200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686019cb10 con 0x7f68600730f0 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f6865200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686019cc80 con 0x7f6860073a00 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f685de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f685de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38562/0 (socket says 172.21.15.90:38562) 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.748+0000 7f685de00700 1 -- 172.21.15.90:0/1169705616 learned_addr learned my addr 172.21.15.90:0/1169705616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.749+0000 7f685d400700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6860073a00 0x7f68601987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.749+0000 7f685de00700 1 -- 172.21.15.90:0/1169705616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6860073a00 msgr2=0x7f68601987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.749+0000 7f685de00700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6860073a00 0x7f68601987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.749+0000 7f685de00700 1 -- 172.21.15.90:0/1169705616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0095f0 con 0x7f68600730f0 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.749+0000 7f685de00700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f685000b4b0 tx=0x7f685000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.750+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6850011800 con 0x7f68600730f0 2024-04-24T08:28:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.750+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6850011e20 con 0x7f68600730f0 2024-04-24T08:28:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.750+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686019cf60 con 0x7f68600730f0 2024-04-24T08:28:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.750+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685000f2e0 con 0x7f68600730f0 2024-04-24T08:28:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.750+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686019d4b0 con 0x7f68600730f0 2024-04-24T08:28:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.751+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6860109c60 con 0x7f68600730f0 2024-04-24T08:28:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.752+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f6850010380 con 0x7f68600730f0 2024-04-24T08:28:36.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.753+0000 7f6856a00700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 0x7f6844070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:36.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.753+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f685008e320 con 0x7f68600730f0 2024-04-24T08:28:36.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.753+0000 7f685d400700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 0x7f6844070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:36.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.754+0000 7f685d400700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 0x7f6844070df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f684c006010 tx=0x7f684c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.758+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f6850057630 con 0x7f68600730f0 2024-04-24T08:28:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.989+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f686004f2a0 con 0x7f68600730f0 2024-04-24T08:28:36.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.996+0000 7f6856a00700 1 -- 172.21.15.90:0/1169705616 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f685005acf0 con 0x7f68600730f0 2024-04-24T08:28:37.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 msgr2=0x7f6844070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:37.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 0x7f6844070df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f684c006010 tx=0x7f684c01a040 comp rx=0 tx=0).stop 2024-04-24T08:28:37.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 msgr2=0x7f6860198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f685000b4b0 tx=0x7f685000b7c0 comp rx=0 tx=0).stop 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 shutdown_connections 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f684406e940 0x7f6844070df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68600730f0 0x7f6860198160 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 --2- 172.21.15.90:0/1169705616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6860073a00 0x7f68601987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:36.999+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 >> 172.21.15.90:0/1169705616 conn(0x7f68600fbd80 msgr2=0x7f6860106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:37.000+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 shutdown_connections 2024-04-24T08:28:37.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:37.000+0000 7f6865200700 1 -- 172.21.15.90:0/1169705616 wait complete. 2024-04-24T08:28:37.020 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:36 smithi090 ceph-mon[25016]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:37.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:36 smithi149 ceph-mon[31282]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:37.792 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T08:28:37.792 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:28:37.793 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T08:28:38.209 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:37 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1169705616' entity='client.admin' 2024-04-24T08:28:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:28:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:28:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:37 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:38.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:37 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1169705616' entity='client.admin' 2024-04-24T08:28:38.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:38.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:28:38.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:28:38.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:37 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:39 smithi149 ceph-mon[31282]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:39.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:39 smithi090 ceph-mon[25016]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:40.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 -- 172.21.15.90:0/1194669175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f545810c8f0 msgr2=0x7f545810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:40.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 --2- 172.21.15.90:0/1194669175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f545810c8f0 0x7f545810ccc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5448009910 tx=0x7f5448009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:40.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 -- 172.21.15.90:0/1194669175 shutdown_connections 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 --2- 172.21.15.90:0/1194669175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5458106b80 0x7f5458106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 --2- 172.21.15.90:0/1194669175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f545810c8f0 0x7f545810ccc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 -- 172.21.15.90:0/1194669175 >> 172.21.15.90:0/1194669175 conn(0x7f5458076020 msgr2=0x7f5458078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 -- 172.21.15.90:0/1194669175 shutdown_connections 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.500+0000 7f545e200700 1 -- 172.21.15.90:0/1194669175 wait complete. 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.501+0000 7f545e200700 1 Processor -- start 2024-04-24T08:28:40.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.501+0000 7f545e200700 1 -- start start 2024-04-24T08:28:40.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f545e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:40.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f545e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 0x7f545819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f545e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545819d0f0 con 0x7f5458106b80 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f545e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54581a0e80 con 0x7f545810c8f0 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f5457400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f5457400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41846/0 (socket says 172.21.15.90:41846) 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.502+0000 7f5457400700 1 -- 172.21.15.90:0/818489537 learned_addr learned my addr 172.21.15.90:0/818489537 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5456a00700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 0x7f545819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5457400700 1 -- 172.21.15.90:0/818489537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 msgr2=0x7f545819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5457400700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 0x7f545819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5457400700 1 -- 172.21.15.90:0/818489537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480095f0 con 0x7f5458106b80 2024-04-24T08:28:40.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5456a00700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 0x7f545819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f5457400700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f544800b5c0 tx=0x7f544800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544801d070 con 0x7f5458106b80 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f544800f460 con 0x7f5458106b80 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.503+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54581a1100 con 0x7f5458106b80 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.504+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5448017610 con 0x7f5458106b80 2024-04-24T08:28:40.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.504+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54581a15f0 con 0x7f5458106b80 2024-04-24T08:28:40.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.505+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545804f2a0 con 0x7f5458106b80 2024-04-24T08:28:40.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.506+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f544800f5d0 con 0x7f5458106b80 2024-04-24T08:28:40.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.507+0000 7f544fe00700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 0x7f543c070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:40.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.507+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f544808ffa0 con 0x7f5458106b80 2024-04-24T08:28:40.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.507+0000 7f5456a00700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 0x7f543c070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.508+0000 7f5456a00700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 0x7f543c070df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f545819daf0 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.512+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f5448059190 con 0x7f5458106b80 2024-04-24T08:28:40.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.738+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f545819d830 con 0x7f5458106b80 2024-04-24T08:28:40.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.738+0000 7f544fe00700 1 -- 172.21.15.90:0/818489537 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f544805c850 con 0x7f5458106b80 2024-04-24T08:28:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 msgr2=0x7f543c070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 0x7f543c070df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f545819daf0 tx=0x7f5444008040 comp rx=0 tx=0).stop 2024-04-24T08:28:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 msgr2=0x7f545819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f544800b5c0 tx=0x7f544800be00 comp rx=0 tx=0).stop 2024-04-24T08:28:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 shutdown_connections 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f543c06e940 0x7f543c070df0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5458106b80 0x7f545819c4d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 --2- 172.21.15.90:0/818489537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545810c8f0 0x7f545819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 >> 172.21.15.90:0/818489537 conn(0x7f5458076020 msgr2=0x7f54580779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.742+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 shutdown_connections 2024-04-24T08:28:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:40.743+0000 7f545e200700 1 -- 172.21.15.90:0/818489537 wait complete. 2024-04-24T08:28:41.525 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T08:28:41.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:41 smithi090 ceph-mon[25016]: pgmap v133: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:41.927 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:42.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:41 smithi149 ceph-mon[31282]: pgmap v133: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:43.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:42 smithi090 ceph-mon[25016]: pgmap v134: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:43.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:42 smithi149 ceph-mon[31282]: pgmap v134: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:44.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 -- 172.21.15.90:0/309538607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 msgr2=0x7f09701028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:44.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 --2- 172.21.15.90:0/309538607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701028e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f095c009910 tx=0x7f095c009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:44.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 -- 172.21.15.90:0/309538607 shutdown_connections 2024-04-24T08:28:44.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 --2- 172.21.15.90:0/309538607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701028e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 --2- 172.21.15.90:0/309538607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 0x7f0970108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 -- 172.21.15.90:0/309538607 >> 172.21.15.90:0/309538607 conn(0x7f09700fdf90 msgr2=0x7f09701003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:44.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 -- 172.21.15.90:0/309538607 shutdown_connections 2024-04-24T08:28:44.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.039+0000 7f0976600700 1 -- 172.21.15.90:0/309538607 wait complete. 2024-04-24T08:28:44.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.040+0000 7f0976600700 1 Processor -- start 2024-04-24T08:28:44.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f0976600700 1 -- start start 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f0976600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f0976600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 0x7f0970198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f0976600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970198e10 con 0x7f0970102470 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f0976600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097019cba0 con 0x7f0970108470 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f096f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f096f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41858/0 (socket says 172.21.15.90:41858) 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.041+0000 7f096f400700 1 -- 172.21.15.90:0/634715124 learned_addr learned my addr 172.21.15.90:0/634715124 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.042+0000 7f096ea00700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 0x7f0970198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.042+0000 7f096f400700 1 -- 172.21.15.90:0/634715124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 msgr2=0x7f0970198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.042+0000 7f096f400700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 0x7f0970198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.042+0000 7f096f400700 1 -- 172.21.15.90:0/634715124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0095f0 con 0x7f0970102470 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.042+0000 7f096f400700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f096000c8a0 tx=0x7f096000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.043+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096000cea0 con 0x7f0970102470 2024-04-24T08:28:44.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.043+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0960004830 con 0x7f0970102470 2024-04-24T08:28:44.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.043+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097019ce80 con 0x7f0970102470 2024-04-24T08:28:44.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.043+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09600055b0 con 0x7f0970102470 2024-04-24T08:28:44.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.043+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097019d3d0 con 0x7f0970102470 2024-04-24T08:28:44.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.044+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950005320 con 0x7f0970102470 2024-04-24T08:28:44.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.045+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f09600049a0 con 0x7f0970102470 2024-04-24T08:28:44.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.046+0000 7f0967e00700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 0x7f0954070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:44.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.046+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f096008e540 con 0x7f0970102470 2024-04-24T08:28:44.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.046+0000 7f096ea00700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 0x7f0954070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:44.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.047+0000 7f096ea00700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 0x7f0954070df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0970199810 tx=0x7f095c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:44.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.051+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f09600577b0 con 0x7f0970102470 2024-04-24T08:28:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.281+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f0950005190 con 0x7f0970102470 2024-04-24T08:28:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.282+0000 7f0967e00700 1 -- 172.21.15.90:0/634715124 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0960004c50 con 0x7f0970102470 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.286+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 msgr2=0x7f0954070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.286+0000 7f0976600700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 0x7f0954070df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0970199810 tx=0x7f095c01a040 comp rx=0 tx=0).stop 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.286+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 msgr2=0x7f09701981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.286+0000 7f0976600700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f096000c8a0 tx=0x7f096000cc60 comp rx=0 tx=0).stop 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 shutdown_connections 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f095406e940 0x7f0954070df0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0970102470 0x7f09701981f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 --2- 172.21.15.90:0/634715124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0970108470 0x7f0970198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:44.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 >> 172.21.15.90:0/634715124 conn(0x7f09700fdf90 msgr2=0x7f09700ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:44.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 shutdown_connections 2024-04-24T08:28:44.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:44.287+0000 7f0976600700 1 -- 172.21.15.90:0/634715124 wait complete. 2024-04-24T08:28:45.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T08:28:45.765 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:45.914 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:45 smithi090 ceph-mon[25016]: pgmap v135: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:45 smithi149 ceph-mon[31282]: pgmap v135: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:47.731 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:47 smithi090 ceph-mon[25016]: pgmap v136: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:47.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 -- 172.21.15.90:0/2684224426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 msgr2=0x7f9d2410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:47.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 --2- 172.21.15.90:0/2684224426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d2410c5e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009940 tx=0x7f9d10009c50 comp rx=0 tx=0).stop 2024-04-24T08:28:47.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 -- 172.21.15.90:0/2684224426 shutdown_connections 2024-04-24T08:28:47.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 --2- 172.21.15.90:0/2684224426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d2410c5e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 --2- 172.21.15.90:0/2684224426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 0x7f9d24073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 -- 172.21.15.90:0/2684224426 >> 172.21.15.90:0/2684224426 conn(0x7f9d240fbda0 msgr2=0x7f9d240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 -- 172.21.15.90:0/2684224426 shutdown_connections 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.904+0000 7f9d29200700 1 -- 172.21.15.90:0/2684224426 wait complete. 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.905+0000 7f9d29200700 1 Processor -- start 2024-04-24T08:28:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d29200700 1 -- start start 2024-04-24T08:28:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d29200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 0x7f9d241981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d29200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d29200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24198dd0 con 0x7f9d240739a0 2024-04-24T08:28:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d29200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2419cb60 con 0x7f9d24073090 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.906+0000 7f9d21400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41872/0 (socket says 172.21.15.90:41872) 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 -- 172.21.15.90:0/395661533 learned_addr learned my addr 172.21.15.90:0/395661533 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21e00700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 0x7f9d241981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 -- 172.21.15.90:0/395661533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 msgr2=0x7f9d241981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 0x7f9d241981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 -- 172.21.15.90:0/395661533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d240739a0 2024-04-24T08:28:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.907+0000 7f9d21400700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d100055f0 tx=0x7f9d100056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.908+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1001d070 con 0x7f9d240739a0 2024-04-24T08:28:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.908+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d1000f460 con 0x7f9d240739a0 2024-04-24T08:28:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.908+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2419cde0 con 0x7f9d240739a0 2024-04-24T08:28:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.908+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d10021640 con 0x7f9d240739a0 2024-04-24T08:28:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.908+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2419d2d0 con 0x7f9d240739a0 2024-04-24T08:28:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.909+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24109ce0 con 0x7f9d240739a0 2024-04-24T08:28:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.910+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f9d1000f5d0 con 0x7f9d240739a0 2024-04-24T08:28:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.911+0000 7f9d1aa00700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 0x7f9d08070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.911+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9d1008fca0 con 0x7f9d240739a0 2024-04-24T08:28:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.911+0000 7f9d21e00700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 0x7f9d08070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:47.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.912+0000 7f9d21e00700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 0x7f9d08070df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005fd0 tx=0x7f9d1400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:47.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:47.916+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f9d10094070 con 0x7f9d240739a0 2024-04-24T08:28:48.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:47 smithi149 ceph-mon[31282]: pgmap v136: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:48.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.145+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f9d2404f2a0 con 0x7f9d240739a0 2024-04-24T08:28:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.146+0000 7f9d1aa00700 1 -- 172.21.15.90:0/395661533 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9d10058f30 con 0x7f9d240739a0 2024-04-24T08:28:48.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 msgr2=0x7f9d08070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:48.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 0x7f9d08070df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005fd0 tx=0x7f9d1400a560 comp rx=0 tx=0).stop 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 msgr2=0x7f9d241986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d100055f0 tx=0x7f9d100056d0 comp rx=0 tx=0).stop 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 shutdown_connections 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f9d0806e940 0x7f9d08070df0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d24073090 0x7f9d241981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 --2- 172.21.15.90:0/395661533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d240739a0 0x7f9d241986f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.149+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 >> 172.21.15.90:0/395661533 conn(0x7f9d240fbda0 msgr2=0x7f9d24106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.150+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 shutdown_connections 2024-04-24T08:28:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:48.150+0000 7f9d29200700 1 -- 172.21.15.90:0/395661533 wait complete. 2024-04-24T08:28:49.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-24T08:28:49.453 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:49.909 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:49 smithi090 ceph-mon[25016]: pgmap v137: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:50.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:49 smithi149 ceph-mon[31282]: pgmap v137: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:51.013 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:50 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:51.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:50 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:28:51.835 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:51 smithi090 ceph-mon[25016]: pgmap v138: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:51.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 -- 172.21.15.90:0/2051523671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 msgr2=0x7fde4c100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 --2- 172.21.15.90:0/2051523671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c100660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009910 tx=0x7fde3c009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 -- 172.21.15.90:0/2051523671 shutdown_connections 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 --2- 172.21.15.90:0/2051523671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 0x7fde4c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 --2- 172.21.15.90:0/2051523671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c100660 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 -- 172.21.15.90:0/2051523671 >> 172.21.15.90:0/2051523671 conn(0x7fde4c0fbb10 msgr2=0x7fde4c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 -- 172.21.15.90:0/2051523671 shutdown_connections 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.887+0000 7fde4b200700 1 -- 172.21.15.90:0/2051523671 wait complete. 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 Processor -- start 2024-04-24T08:28:51.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 -- start start 2024-04-24T08:28:51.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:51.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 0x7fde4c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:51.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19c5d0 con 0x7fde4c100290 2024-04-24T08:28:51.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.889+0000 7fde4b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c195a80 con 0x7fde4c100c30 2024-04-24T08:28:51.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:51.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51734/0 (socket says 172.21.15.90:51734) 2024-04-24T08:28:51.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 -- 172.21.15.90:0/3947542740 learned_addr learned my addr 172.21.15.90:0/3947542740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49400700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 0x7fde4c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 -- 172.21.15.90:0/3947542740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 msgr2=0x7fde4c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 0x7fde4c19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 -- 172.21.15.90:0/3947542740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0095f0 con 0x7fde4c100290 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.890+0000 7fde49e00700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b5c0 tx=0x7fde3c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.891+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3c01d070 con 0x7fde4c100290 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.891+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde3c00f460 con 0x7fde4c100290 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.891+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c195d00 con 0x7fde4c100290 2024-04-24T08:28:51.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.891+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3c017610 con 0x7fde4c100290 2024-04-24T08:28:51.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.892+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c1961f0 con 0x7fde4c100290 2024-04-24T08:28:51.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.893+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c1085a0 con 0x7fde4c100290 2024-04-24T08:28:51.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.894+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fde3c00f5d0 con 0x7fde4c100290 2024-04-24T08:28:51.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.895+0000 7fde42a00700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 0x7fde30070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:51.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.896+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fde3c08ffa0 con 0x7fde4c100290 2024-04-24T08:28:51.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.896+0000 7fde49400700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 0x7fde30070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:51.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.897+0000 7fde49400700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 0x7fde30070df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde4c104e80 tx=0x7fde38005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:51.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:51.900+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fde3c05ccc0 con 0x7fde4c100290 2024-04-24T08:28:52.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:51 smithi149 ceph-mon[31282]: pgmap v138: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:28:52.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.153+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}) v1 -- 0x7fde4c1964d0 con 0x7fde3006e940 2024-04-24T08:28:52.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.162+0000 7fde42a00700 1 -- 172.21.15.90:0/3947542740 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fde4c1964d0 con 0x7fde3006e940 2024-04-24T08:28:52.164 INFO:teuthology.orchestra.run.smithi090.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 msgr2=0x7fde30070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 0x7fde30070df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde4c104e80 tx=0x7fde38005cf0 comp rx=0 tx=0).stop 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 msgr2=0x7fde4c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b5c0 tx=0x7fde3c00be00 comp rx=0 tx=0).stop 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 shutdown_connections 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fde3006e940 0x7fde30070df0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde4c100290 0x7fde4c19ba00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 --2- 172.21.15.90:0/3947542740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde4c100c30 0x7fde4c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.165+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 >> 172.21.15.90:0/3947542740 conn(0x7fde4c0fbb10 msgr2=0x7fde4c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.166+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 shutdown_connections 2024-04-24T08:28:52.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:52.166+0000 7fde4b200700 1 -- 172.21.15.90:0/3947542740 wait complete. 2024-04-24T08:28:53.063 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T08:28:53.064 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:28:53.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-24T08:28:53.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:53 smithi090 ceph-mon[25016]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:53.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:53 smithi090 ceph-mon[25016]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:28:53.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:53 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:53.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:53 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:53.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:53 smithi090 ceph-mon[25016]: pgmap v139: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:53.470 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:28:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:53 smithi149 ceph-mon[31282]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:53 smithi149 ceph-mon[31282]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:28:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:53 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:53 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:53 smithi149 ceph-mon[31282]: pgmap v139: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:54 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:28:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:54 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:28:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:54 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:54 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:28:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:54 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:28:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:54 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:55.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:55 smithi090 ceph-mon[25016]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:28:55.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:55 smithi090 ceph-mon[25016]: pgmap v140: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:55 smithi149 ceph-mon[31282]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:28:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:55 smithi149 ceph-mon[31282]: pgmap v140: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 -- 172.21.15.90:0/2646868325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 msgr2=0x7f015c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 --2- 172.21.15.90:0/2646868325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c104eb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0148009910 tx=0x7f0148009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 -- 172.21.15.90:0/2646868325 shutdown_connections 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 --2- 172.21.15.90:0/2646868325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c104eb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 --2- 172.21.15.90:0/2646868325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 -- 172.21.15.90:0/2646868325 >> 172.21.15.90:0/2646868325 conn(0x7f015c075130 msgr2=0x7f015c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 -- 172.21.15.90:0/2646868325 shutdown_connections 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.940+0000 7f0164600700 1 -- 172.21.15.90:0/2646868325 wait complete. 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.942+0000 7f0164600700 1 Processor -- start 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.942+0000 7f0164600700 1 -- start start 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0164600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0164600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0164600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c196a80 con 0x7f015c069140 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0164600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c19a810 con 0x7f015c0687a0 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0161200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0161200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51742/0 (socket says 172.21.15.90:51742) 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0161200700 1 -- 172.21.15.90:0/952593837 learned_addr learned my addr 172.21.15.90:0/952593837 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.943+0000 7f0161c00700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.944+0000 7f0161200700 1 -- 172.21.15.90:0/952593837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 msgr2=0x7f015c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.944+0000 7f0161200700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.944+0000 7f0161200700 1 -- 172.21.15.90:0/952593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480095f0 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.944+0000 7f0161c00700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.944+0000 7f0161200700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f014800bda0 tx=0x7f014800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.945+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f014801d070 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.945+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0148004fb0 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.945+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c19aa90 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.945+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f014800f460 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.945+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c19af80 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.946+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c108750 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.947+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f0148022800 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.948+0000 7f015a000700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 0x7f0144070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.948+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f014808fed0 con 0x7f015c069140 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.949+0000 7f0161c00700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 0x7f0144070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.949+0000 7f0161c00700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 0x7f0144070df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0150005950 tx=0x7f0150005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:55.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:55.954+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f0148059290 con 0x7f015c069140 2024-04-24T08:28:56.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.216+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f015c19b260 con 0x7f014406e940 2024-04-24T08:28:56.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.218+0000 7f015a000700 1 -- 172.21.15.90:0/952593837 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f015c19b260 con 0x7f014406e940 2024-04-24T08:28:56.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.222+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 msgr2=0x7f0144070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.222+0000 7f0164600700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 0x7f0144070df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0150005950 tx=0x7f0150005f50 comp rx=0 tx=0).stop 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.222+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 msgr2=0x7f015c1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.222+0000 7f0164600700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f014800bda0 tx=0x7f014800be80 comp rx=0 tx=0).stop 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.222+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 shutdown_connections 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f014406e940 0x7f0144070df0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f015c0687a0 0x7f015c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 --2- 172.21.15.90:0/952593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f015c069140 0x7f015c1963a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 >> 172.21.15.90:0/952593837 conn(0x7f015c075130 msgr2=0x7f015c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:56.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 shutdown_connections 2024-04-24T08:28:56.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.223+0000 7f0164600700 1 -- 172.21.15.90:0/952593837 wait complete. 2024-04-24T08:28:56.240 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 -- 172.21.15.90:0/1421302589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 msgr2=0x7ffab41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 --2- 172.21.15.90:0/1421302589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab41028e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009910 tx=0x7ffaa0009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 -- 172.21.15.90:0/1421302589 shutdown_connections 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 --2- 172.21.15.90:0/1421302589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab41028e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 --2- 172.21.15.90:0/1421302589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 0x7ffab4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 -- 172.21.15.90:0/1421302589 >> 172.21.15.90:0/1421302589 conn(0x7ffab40fdf90 msgr2=0x7ffab41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 -- 172.21.15.90:0/1421302589 shutdown_connections 2024-04-24T08:28:56.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.383+0000 7ffaba400700 1 -- 172.21.15.90:0/1421302589 wait complete. 2024-04-24T08:28:56.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.384+0000 7ffaba400700 1 Processor -- start 2024-04-24T08:28:56.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.384+0000 7ffaba400700 1 -- start start 2024-04-24T08:28:56.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffaba400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffaba400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 0x7ffab41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffaba400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4198e90 con 0x7ffab4102470 2024-04-24T08:28:56.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffaba400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab419cc20 con 0x7ffab4108470 2024-04-24T08:28:56.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffab3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffab3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51764/0 (socket says 172.21.15.90:51764) 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffab3400700 1 -- 172.21.15.90:0/1975584622 learned_addr learned my addr 172.21.15.90:0/1975584622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.385+0000 7ffab2a00700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 0x7ffab41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.386+0000 7ffab3400700 1 -- 172.21.15.90:0/1975584622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 msgr2=0x7ffab41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.386+0000 7ffab3400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 0x7ffab41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.386+0000 7ffab3400700 1 -- 172.21.15.90:0/1975584622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00095f0 con 0x7ffab4102470 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.386+0000 7ffab3400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c8a0 tx=0x7ffaa400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.387+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa40078e0 con 0x7ffab4102470 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.387+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab419cf00 con 0x7ffab4102470 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.387+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaa400f450 con 0x7ffab4102470 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.387+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa400e5e0 con 0x7ffab4102470 2024-04-24T08:28:56.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.387+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab419d450 con 0x7ffab4102470 2024-04-24T08:28:56.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.388+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab410a970 con 0x7ffab4102470 2024-04-24T08:28:56.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.389+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7ffaa400f5c0 con 0x7ffab4102470 2024-04-24T08:28:56.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.390+0000 7ffaabe00700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 0x7ffa98070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.390+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffaa408e7f0 con 0x7ffab4102470 2024-04-24T08:28:56.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.390+0000 7ffab2a00700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 0x7ffa98070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.391+0000 7ffab2a00700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 0x7ffa98070df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffab4199890 tx=0x7ffaa001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:56.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.395+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7ffaa4057a60 con 0x7ffab4102470 2024-04-24T08:28:56.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.669+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab4199710 con 0x7ffa9806e940 2024-04-24T08:28:56.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.671+0000 7ffaabe00700 1 -- 172.21.15.90:0/1975584622 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ffaa001e040 con 0x7ffa9806e940 2024-04-24T08:28:56.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 msgr2=0x7ffa98070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 0x7ffa98070df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffab4199890 tx=0x7ffaa001a040 comp rx=0 tx=0).stop 2024-04-24T08:28:56.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 msgr2=0x7ffab4198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c8a0 tx=0x7ffaa400cbb0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 shutdown_connections 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7ffa9806e940 0x7ffa98070df0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffab4102470 0x7ffab4198270 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 --2- 172.21.15.90:0/1975584622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffab4108470 0x7ffab41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 >> 172.21.15.90:0/1975584622 conn(0x7ffab40fdf90 msgr2=0x7ffab40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 shutdown_connections 2024-04-24T08:28:56.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.675+0000 7ffaba400700 1 -- 172.21.15.90:0/1975584622 wait complete. 2024-04-24T08:28:56.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 -- 172.21.15.90:0/1047102568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08108460 msgr2=0x7f8f08108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1047102568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08108460 0x7f8f08108830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009910 tx=0x7f8ef8009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 -- 172.21.15.90:0/1047102568 shutdown_connections 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1047102568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08102460 0x7f8f081028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1047102568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08108460 0x7f8f08108830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.823+0000 7f8f0e600700 1 -- 172.21.15.90:0/1047102568 >> 172.21.15.90:0/1047102568 conn(0x7f8f080fdf40 msgr2=0x7f8f08100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.824+0000 7f8f0e600700 1 -- 172.21.15.90:0/1047102568 shutdown_connections 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.824+0000 7f8f0e600700 1 -- 172.21.15.90:0/1047102568 wait complete. 2024-04-24T08:28:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 Processor -- start 2024-04-24T08:28:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 -- start start 2024-04-24T08:28:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08108460 0x7f8f08198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f08198e20 con 0x7f8f08102460 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f0e600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819cbb0 con 0x7f8f08108460 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f07400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f07400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51794/0 (socket says 172.21.15.90:51794) 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.825+0000 7f8f07400700 1 -- 172.21.15.90:0/1618506987 learned_addr learned my addr 172.21.15.90:0/1618506987 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f06a00700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08108460 0x7f8f08198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f07400700 1 -- 172.21.15.90:0/1618506987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08108460 msgr2=0x7f8f08198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f07400700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08108460 0x7f8f08198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:56.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f07400700 1 -- 172.21.15.90:0/1618506987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80095f0 con 0x7f8f08102460 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f07400700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef801d070 con 0x7f8f08102460 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ef800f460 con 0x7f8f08102460 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0819ce30 con 0x7f8f08102460 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.826+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef8017610 con 0x7f8f08102460 2024-04-24T08:28:56.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.827+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0819d320 con 0x7f8f08102460 2024-04-24T08:28:56.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.828+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0810a9d0 con 0x7f8f08102460 2024-04-24T08:28:56.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.829+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f8ef800f5d0 con 0x7f8f08102460 2024-04-24T08:28:56.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.830+0000 7f8effe00700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 0x7f8eec070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:56.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.830+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8ef808ff00 con 0x7f8f08102460 2024-04-24T08:28:56.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.830+0000 7f8f06a00700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 0x7f8eec070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:56.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.831+0000 7f8f06a00700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 0x7f8eec070df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006fd0 tx=0x7f8ef4008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:56.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:56.835+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f8ef8059190 con 0x7f8f08102460 2024-04-24T08:28:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.077+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f08199670 con 0x7f8eec06e940 2024-04-24T08:28:57.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.092+0000 7f8effe00700 1 -- 172.21.15.90:0/1618506987 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8f08199670 con 0x7f8eec06e940 2024-04-24T08:28:57.093 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (115s) 66s ago 15m 13.5M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (16m) 66s ago 16m 8098k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (14m) 2m ago 14m 7902k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (16m) 66s ago 16m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (14m) 2m ago 14m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (77s) 66s ago 15m 53.8M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (8m) 66s ago 8m 121M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (8m) 66s ago 8m 15.5M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (8m) 2m ago 8m 255M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (8m) 2m ago 8m 14.8M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (4m) 66s ago 17m 511M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (3m) 2m ago 14m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (17m) 66s ago 17m 55.5M 2048M 18.2.0 dc2bc1663786 ecda201dc4f0 2024-04-24T08:28:57.094 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (14m) 2m ago 14m 47.8M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (3m) 66s ago 16m 7570k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (2m) 2m ago 14m 2919k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (13m) 66s ago 13m 215M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (12m) 66s ago 12m 224M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (12m) 66s ago 12m 188M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (11m) 2m ago 11m 208M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (11m) 2m ago 11m 181M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (10m) 2m ago 10m 172M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:28:57.095 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (2m) 66s ago 15m 57.6M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 msgr2=0x7f8eec070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 0x7f8eec070df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006fd0 tx=0x7f8ef4008040 comp rx=0 tx=0).stop 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 msgr2=0x7f8f08198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef800be00 comp rx=0 tx=0).stop 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 shutdown_connections 2024-04-24T08:28:57.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f8eec06e940 0x7f8eec070df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f08102460 0x7f8f08198200 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 --2- 172.21.15.90:0/1618506987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f08108460 0x7f8f08198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 >> 172.21.15.90:0/1618506987 conn(0x7f8f080fdf40 msgr2=0x7f8f080fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 shutdown_connections 2024-04-24T08:28:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.095+0000 7f8f0e600700 1 -- 172.21.15.90:0/1618506987 wait complete. 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 -- 172.21.15.90:0/3074861012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0106b80 msgr2=0x7f89c0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 --2- 172.21.15.90:0/3074861012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0106b80 0x7f89c0106ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009910 tx=0x7f89ac009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 -- 172.21.15.90:0/3074861012 shutdown_connections 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 --2- 172.21.15.90:0/3074861012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0106b80 0x7f89c0106ff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 --2- 172.21.15.90:0/3074861012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c010c660 0x7f89c010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.243+0000 7f89c5c00700 1 -- 172.21.15.90:0/3074861012 >> 172.21.15.90:0/3074861012 conn(0x7f89c0075e90 msgr2=0x7f89c00782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.244+0000 7f89c5c00700 1 -- 172.21.15.90:0/3074861012 shutdown_connections 2024-04-24T08:28:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.244+0000 7f89c5c00700 1 -- 172.21.15.90:0/3074861012 wait complete. 2024-04-24T08:28:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 Processor -- start 2024-04-24T08:28:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 -- start start 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c0106b80 0x7f89c019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019cf60 con 0x7f89c010c660 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.245+0000 7f89c5c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c01a0cf0 con 0x7f89c0106b80 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51810/0 (socket says 172.21.15.90:51810) 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 -- 172.21.15.90:0/881305747 learned_addr learned my addr 172.21.15.90:0/881305747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bfe00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c0106b80 0x7f89c019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 -- 172.21.15.90:0/881305747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c0106b80 msgr2=0x7f89c019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c0106b80 0x7f89c019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 -- 172.21.15.90:0/881305747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0095f0 con 0x7f89c010c660 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.246+0000 7f89bf400700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b5c0 tx=0x7f89ac00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.247+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ac01d070 con 0x7f89c010c660 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.247+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89ac00f460 con 0x7f89c010c660 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.247+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ac017610 con 0x7f89c010c660 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.247+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c01a0f70 con 0x7f89c010c660 2024-04-24T08:28:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.247+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c01a1460 con 0x7f89c010c660 2024-04-24T08:28:57.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.248+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c010eb60 con 0x7f89c010c660 2024-04-24T08:28:57.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.249+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f89ac00f5d0 con 0x7f89c010c660 2024-04-24T08:28:57.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.250+0000 7f89bcc00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 0x7f89a4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.250+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f89ac08fed0 con 0x7f89c010c660 2024-04-24T08:28:57.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.250+0000 7f89bfe00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 0x7f89a4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.251+0000 7f89bfe00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 0x7f89a4070df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f89b0006fd0 tx=0x7f89b0008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:57.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.255+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f89ac059140 con 0x7f89c010c660 2024-04-24T08:28:57.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.568+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f89c0062380 con 0x7f89c010c660 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.569+0000 7f89bcc00700 1 -- 172.21.15.90:0/881305747 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f89ac05c800 con 0x7f89c010c660 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:28:57.570 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:28:57.571 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 msgr2=0x7f89a4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 0x7f89a4070df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f89b0006fd0 tx=0x7f89b0008040 comp rx=0 tx=0).stop 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 msgr2=0x7f89c019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b5c0 tx=0x7f89ac00be00 comp rx=0 tx=0).stop 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 shutdown_connections 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f89a406e940 0x7f89a4070df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c0106b80 0x7f89c019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 --2- 172.21.15.90:0/881305747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c010c660 0x7f89c019c880 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.572+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 >> 172.21.15.90:0/881305747 conn(0x7f89c0075e90 msgr2=0x7f89c0077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:57.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.573+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 shutdown_connections 2024-04-24T08:28:57.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.573+0000 7f89c5c00700 1 -- 172.21.15.90:0/881305747 wait complete. 2024-04-24T08:28:57.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 -- 172.21.15.90:0/33423447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 msgr2=0x7f9290104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 --2- 172.21.15.90:0/33423447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290104e50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f927c009910 tx=0x7f927c009c20 comp rx=0 tx=0).stop 2024-04-24T08:28:57.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 -- 172.21.15.90:0/33423447 shutdown_connections 2024-04-24T08:28:57.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 --2- 172.21.15.90:0/33423447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290104e50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 --2- 172.21.15.90:0/33423447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 0x7f92901008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.725+0000 7f9296200700 1 -- 172.21.15.90:0/33423447 >> 172.21.15.90:0/33423447 conn(0x7f92900fbda0 msgr2=0x7f92900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:57.727 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:57 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.726+0000 7f9296200700 1 -- 172.21.15.90:0/33423447 shutdown_connections 2024-04-24T08:28:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.726+0000 7f9296200700 1 -- 172.21.15.90:0/33423447 wait complete. 2024-04-24T08:28:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.727+0000 7f9296200700 1 Processor -- start 2024-04-24T08:28:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.727+0000 7f9296200700 1 -- start start 2024-04-24T08:28:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f9296200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 0x7f92901980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f9296200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f9296200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290198d00 con 0x7f9290100e80 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f9296200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929019ca90 con 0x7f92901004e0 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f928ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f928ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51824/0 (socket says 172.21.15.90:51824) 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f928ea00700 1 -- 172.21.15.90:0/1848616431 learned_addr learned my addr 172.21.15.90:0/1848616431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.728+0000 7f928f400700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 0x7f92901980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.729+0000 7f928ea00700 1 -- 172.21.15.90:0/1848616431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 msgr2=0x7f92901980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:57.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.729+0000 7f928ea00700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 0x7f92901980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:57.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.729+0000 7f928ea00700 1 -- 172.21.15.90:0/1848616431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c0095f0 con 0x7f9290100e80 2024-04-24T08:28:57.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.729+0000 7f928ea00700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f927c00bda0 tx=0x7f927c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.730+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c01d070 con 0x7f9290100e80 2024-04-24T08:28:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.730+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f927c00f460 con 0x7f9290100e80 2024-04-24T08:28:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.730+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929019cd10 con 0x7f9290100e80 2024-04-24T08:28:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.730+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c017630 con 0x7f9290100e80 2024-04-24T08:28:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.730+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929019d200 con 0x7f9290100e80 2024-04-24T08:28:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.731+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92901045d0 con 0x7f9290100e80 2024-04-24T08:28:57.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.733+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f927c00fab0 con 0x7f9290100e80 2024-04-24T08:28:57.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.734+0000 7f9287e00700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 0x7f9274070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:57.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.734+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f927c08fdd0 con 0x7f9290100e80 2024-04-24T08:28:57.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.735+0000 7f928f400700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 0x7f9274070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:57.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.735+0000 7f928f400700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 0x7f9274070df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9280005950 tx=0x7f928000a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:57.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:57.740+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f927c059060 con 0x7f9290100e80 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:58.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:57 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:28:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.026+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f929004f2a0 con 0x7f9290100e80 2024-04-24T08:28:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.027+0000 7f9287e00700 1 -- 172.21.15.90:0/1848616431 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1849 (secure 0 0 0) 0x7f927c05c720 con 0x7f9290100e80 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:28:58.029 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:28:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:28:58.031 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:28:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 msgr2=0x7f9274070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 0x7f9274070df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9280005950 tx=0x7f928000a300 comp rx=0 tx=0).stop 2024-04-24T08:28:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 msgr2=0x7f9290198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f927c00bda0 tx=0x7f927c00bdd0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 shutdown_connections 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f927406e940 0x7f9274070df0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92901004e0 0x7f92901980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 --2- 172.21.15.90:0/1848616431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9290100e80 0x7f9290198620 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 >> 172.21.15.90:0/1848616431 conn(0x7f92900fbda0 msgr2=0x7f92900fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.032+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 shutdown_connections 2024-04-24T08:28:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.033+0000 7f9296200700 1 -- 172.21.15.90:0/1848616431 wait complete. 2024-04-24T08:28:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:28:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.187+0000 7fc6fe600700 1 -- 172.21.15.90:0/2275664809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f8073090 msgr2=0x7fc6f8073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.187+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2275664809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f8073090 0x7fc6f8073460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009940 tx=0x7fc6e8009c50 comp rx=0 tx=0).stop 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 -- 172.21.15.90:0/2275664809 shutdown_connections 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2275664809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f80739a0 0x7fc6f810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2275664809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f8073090 0x7fc6f8073460 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 -- 172.21.15.90:0/2275664809 >> 172.21.15.90:0/2275664809 conn(0x7fc6f80fbda0 msgr2=0x7fc6f80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 -- 172.21.15.90:0/2275664809 shutdown_connections 2024-04-24T08:28:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.188+0000 7fc6fe600700 1 -- 172.21.15.90:0/2275664809 wait complete. 2024-04-24T08:28:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 Processor -- start 2024-04-24T08:28:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 -- start start 2024-04-24T08:28:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f8073090 0x7fc6f81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8198de0 con 0x7fc6f80739a0 2024-04-24T08:28:58.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.189+0000 7fc6fe600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819cb70 con 0x7fc6f8073090 2024-04-24T08:28:58.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51836/0 (socket says 172.21.15.90:51836) 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 -- 172.21.15.90:0/2030505463 learned_addr learned my addr 172.21.15.90:0/2030505463 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f7400700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f8073090 0x7fc6f81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 -- 172.21.15.90:0/2030505463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f8073090 msgr2=0x7fc6f81981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f8073090 0x7fc6f81981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.190+0000 7fc6f6a00700 1 -- 172.21.15.90:0/2030505463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80095f0 con 0x7fc6f80739a0 2024-04-24T08:28:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6f6a00700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b4b0 tx=0x7fc6e400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e4011800 con 0x7fc6f80739a0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6e4011e20 con 0x7fc6f80739a0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f819ce50 con 0x7fc6f80739a0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e400f2e0 con 0x7fc6f80739a0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.191+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f819d3a0 con 0x7fc6f80739a0 2024-04-24T08:28:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.192+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f8109ce0 con 0x7fc6f80739a0 2024-04-24T08:28:58.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.194+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fc6e4010380 con 0x7fc6f80739a0 2024-04-24T08:28:58.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.195+0000 7fc6efe00700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 0x7fc6dc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.195+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc6e408e8b0 con 0x7fc6f80739a0 2024-04-24T08:28:58.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.195+0000 7fc6f7400700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 0x7fc6dc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.196+0000 7fc6f7400700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 0x7fc6dc070df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005cd0 tx=0x7fc6e8005c20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:58.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.199+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7fc6e4057bc0 con 0x7fc6f80739a0 2024-04-24T08:28:58.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.460+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 --> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6f819d690 con 0x7fc6dc06e940 2024-04-24T08:28:58.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.462+0000 7fc6efe00700 1 -- 172.21.15.90:0/2030505463 <== mgr.14660 v2:172.21.15.90:6800/3900128298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc6f819d690 con 0x7fc6dc06e940 2024-04-24T08:28:58.463 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:28:58.463 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:28:58.463 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:28:58.463 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/23 daemons upgraded", 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:28:58.464 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:28:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 msgr2=0x7fc6dc070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 0x7fc6dc070df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005cd0 tx=0x7fc6e8005c20 comp rx=0 tx=0).stop 2024-04-24T08:28:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 msgr2=0x7fc6f8198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b4b0 tx=0x7fc6e400b870 comp rx=0 tx=0).stop 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 shutdown_connections 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7fc6dc06e940 0x7fc6dc070df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6f8073090 0x7fc6f81981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 --2- 172.21.15.90:0/2030505463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6f80739a0 0x7fc6f8198700 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 >> 172.21.15.90:0/2030505463 conn(0x7fc6f80fbda0 msgr2=0x7fc6f8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 shutdown_connections 2024-04-24T08:28:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.465+0000 7fc6fe600700 1 -- 172.21.15.90:0/2030505463 wait complete. 2024-04-24T08:28:58.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 -- 172.21.15.90:0/2302413025 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d01004e0 msgr2=0x7f17d01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 --2- 172.21.15.90:0/2302413025 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d01004e0 0x7f17d01008b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17c400b0a0 tx=0x7f17c400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 -- 172.21.15.90:0/2302413025 shutdown_connections 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 --2- 172.21.15.90:0/2302413025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d0100e80 0x7f17d0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 --2- 172.21.15.90:0/2302413025 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d01004e0 0x7f17d01008b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 -- 172.21.15.90:0/2302413025 >> 172.21.15.90:0/2302413025 conn(0x7f17d00fbda0 msgr2=0x7f17d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 -- 172.21.15.90:0/2302413025 shutdown_connections 2024-04-24T08:28:58.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.623+0000 7f17d7a00700 1 -- 172.21.15.90:0/2302413025 wait complete. 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.624+0000 7f17d7a00700 1 Processor -- start 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.624+0000 7f17d7a00700 1 -- start start 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17d7a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d01004e0 0x7f17d0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17d7a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17d7a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0198e80 con 0x7f17d0100e80 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17d7a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d019cc10 con 0x7f17d01004e0 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17cfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17cfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51846/0 (socket says 172.21.15.90:51846) 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17cfe00700 1 -- 172.21.15.90:0/626912417 learned_addr learned my addr 172.21.15.90:0/626912417 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.625+0000 7f17d5000700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d01004e0 0x7f17d0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cfe00700 1 -- 172.21.15.90:0/626912417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d01004e0 msgr2=0x7f17d0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cfe00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d01004e0 0x7f17d0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cfe00700 1 -- 172.21.15.90:0/626912417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c4009d00 con 0x7f17d0100e80 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cfe00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00b480 tx=0x7f17bc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc0118c0 con 0x7f17d0100e80 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17bc010380 con 0x7f17d0100e80 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc00f4f0 con 0x7f17d0100e80 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d019cef0 con 0x7f17d0100e80 2024-04-24T08:28:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.626+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d019d440 con 0x7f17d0100e80 2024-04-24T08:28:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.627+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d01045d0 con 0x7f17d0100e80 2024-04-24T08:28:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.628+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 33) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f17bc0104f0 con 0x7f17d0100e80 2024-04-24T08:28:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.629+0000 7f17cd600700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 0x7f17b8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:28:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.629+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f17bc08e500 con 0x7f17d0100e80 2024-04-24T08:28:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.630+0000 7f17d5000700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 0x7f17b8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:28:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.630+0000 7f17d5000700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 0x7f17b8070df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009480 tx=0x7f17c400b7f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:28:58.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.634+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187527 (secure 0 0 0) 0x7f17bc057770 con 0x7f17d0100e80 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: Upgrade: Target is version 19.0.0-3116-g1f81fb39 (squid) 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, digests ['quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644'] 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: Upgrade: Setting container_image for all mgr 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T08:28:58.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/881305747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:58.678 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:58 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/1848616431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:28:58.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.966+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f17d019d710 con 0x7f17d0100e80 2024-04-24T08:28:58.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.967+0000 7f17cd600700 1 -- 172.21.15.90:0/626912417 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f17bc05ae30 con 0x7f17d0100e80 2024-04-24T08:28:58.968 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:28:58.969 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:28:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:28:58.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 msgr2=0x7f17b8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 0x7f17b8070df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009480 tx=0x7f17c400b7f0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 msgr2=0x7f17d01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00b480 tx=0x7f17bc00b840 comp rx=0 tx=0).stop 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 shutdown_connections 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:6800/3900128298,v1:172.21.15.90:6801/3900128298] conn(0x7f17b806e940 0x7f17b8070df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d01004e0 0x7f17d0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 --2- 172.21.15.90:0/626912417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17d0100e80 0x7f17d01987a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.970+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 >> 172.21.15.90:0/626912417 conn(0x7f17d00fbda0 msgr2=0x7f17d00fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.971+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 shutdown_connections 2024-04-24T08:28:58.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:28:58.971+0000 7f17d7a00700 1 -- 172.21.15.90:0/626912417 wait complete. 2024-04-24T08:28:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:28:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: Upgrade: Target is version 19.0.0-3116-g1f81fb39 (squid) 2024-04-24T08:28:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, digests ['quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644'] 2024-04-24T08:28:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: Upgrade: Setting container_image for all mgr 2024-04-24T08:28:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T08:28:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/881305747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:28:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:58 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/1848616431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:29:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:59 smithi149 ceph-mon[31282]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:59 smithi149 ceph-mon[31282]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:28:59 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/626912417' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:29:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:59 smithi090 ceph-mon[25016]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:59 smithi090 ceph-mon[25016]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:28:59 smithi090 ceph-mon[25016]: from='client.? 172.21.15.90:0/626912417' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: Upgrade: Updating mon.smithi090 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: Deploying daemon mon.smithi090 on smithi090 2024-04-24T08:29:00.919 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:00 smithi090 ceph-mon[25016]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: Upgrade: Updating mon.smithi090 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: Deploying daemon mon.smithi090 on smithi090 2024-04-24T08:29:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:00 smithi149 ceph-mon[31282]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:02.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:02 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:29:03.142 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:02 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[25012]: 2024-04-24T08:29:02.881+0000 7fa7ab000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:29:03.143 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:02 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[25012]: 2024-04-24T08:29:02.881+0000 7fa7ab000700 -1 mon.smithi090@0(leader) e2 *** Got Signal Terminated *** 2024-04-24T08:29:03.479 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:03 smithi090 podman[85107]: 2024-04-24 08:29:03.139895721 +0000 UTC m=+0.410966580 container died ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T08:29:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:03 smithi090 podman[85107]: 2024-04-24 08:29:03.617075849 +0000 UTC m=+0.888146715 container cleanup ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True) 2024-04-24T08:29:03.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:03 smithi090 bash[85107]: ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090 2024-04-24T08:29:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:03 smithi090 podman[85131]: 2024-04-24 08:29:03.941738228 +0000 UTC m=+0.785129396 container remove ecda201dc4f080cbe67035f5a34799653a2cd77fc4d4daa1a30780219e680cb9 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux ) 2024-04-24T08:29:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:04 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service: Deactivated successfully. 2024-04-24T08:29:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:04 smithi090 systemd[1]: Stopped Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:29:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:04 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service: Consumed 33.558s CPU time. 2024-04-24T08:29:05.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 systemd[1]: Starting Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:29:05.670 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 podman[85263]: 2024-04-24 08:29:05.255964137 +0000 UTC m=+0.034407462 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:29:05.670 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 podman[85263]: 2024-04-24 08:29:05.379225057 +0000 UTC m=+0.157668419 container create 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-24T08:29:05.670 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 podman[85263]: 2024-04-24 08:29:05.668220133 +0000 UTC m=+0.446663501 container init 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T08:29:06.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 podman[85263]: 2024-04-24 08:29:05.679835825 +0000 UTC m=+0.458279194 container start 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:29:06.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:29:06.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: pidfile_write: ignore empty --pid-file 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: load: jerasure load: lrc 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Git sha 0 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: DB SUMMARY 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: DB Session ID: CXZDB4NX5BS7KE934ETO 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1732 Bytes 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi090/store.db dir, Total Num: 1, files: 000035.sst 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi090/store.db: 000033.log size: 945548 ; 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:29:06.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.env: 0x560b63642cc0 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.info_log: 0x560b70b5bb00 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.statistics: (nil) 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.use_fsync: 0 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:29:06.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:29:06.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:29:06.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:29:06.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:29:06.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.db_log_dir: 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.wal_dir: 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:29:06.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.write_buffer_manager: 0x560b70b2a1e0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.unordered_write: 0 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:29:06.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.row_cache: None 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.wal_filter: None 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.wal_compression: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:29:06.051 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:29:06.052 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_open_files: -1 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Compression algorithms supported: 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kZSTD supported: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:29:06.053 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000015 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.merge_operator: 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_filter: None 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:29:06.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560b70b5bce0) 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_type: 0 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_index_type: 0 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_shortening: 1 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: checksum: 4 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: no_block_cache: 0 2024-04-24T08:29:06.055 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache: 0x560b70b5d090 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_options: 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: capacity : 536870912 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_shard_bits : 4 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: strict_capacity_limit : 0 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_compressed: (nil) 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: persistent_cache: (nil) 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size: 4096 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size_deviation: 10 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_restart_interval: 16 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_block_restart_interval: 1 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_block_size: 4096 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: partition_filters: 0 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: use_delta_encoding: 1 2024-04-24T08:29:06.056 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: filter_policy: bloomfilter 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: whole_key_filtering: 1 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: verify_compression: 0 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: format_version: 5 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_index_compression: 1 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_align: 0 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: prepopulate_block_cache: 0 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression: NoCompression 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:29:06.057 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.num_levels: 7 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:29:06.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:29:06.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:29:06.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:29:06.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.ttl: 2592000 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:29:06.062 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16512, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 775784da-522a-4526-b09e-6d404a0b9e36 2024-04-24T08:29:06.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947345737933, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947345748538, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 875693, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16513, "largest_seqno": 17779, "table_properties": {"data_size": 870369, "index_size": 3003, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1285, "raw_key_size": 12079, "raw_average_key_size": 23, "raw_value_size": 859671, "raw_average_value_size": 1688, "num_data_blocks": 136, "num_entries": 509, "num_filter_entries": 509, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713947345, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "775784da-522a-4526-b09e-6d404a0b9e36", "db_session_id": "CXZDB4NX5BS7KE934ETO", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947345748653, "job": 1, "event": "recovery_finished"} 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi090/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560b70c4a000 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: DB pointer 0x560b70c34000 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** DB Stats ** 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:06.064 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L0 1/0 855.17 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L6 1/0 7.96 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Sum 2/0 8.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.065 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.2 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:29:06.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative compaction: 0.00 GB write, 48.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval compaction: 0.00 GB write, 48.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache BinnedLRUCache@0x560b70b5d090#2 capacity: 512.00 MB usage: 4.48 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.1e-05 secs_since: 0 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.41 KB,0.000268221%) IndexBlock(1,3.08 KB,0.000587106%) Misc(1,0.00 KB,0%) 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: starting mon.smithi090 rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi090 fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???) e2 preinit fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).mds e17 new map 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).mds e17 print_map 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: e17 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: legacy client fscid: 1 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.067 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Filesystem 'cephfs' (1) 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: fs_name cephfs 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: epoch 17 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: tableserver 0 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: root 0 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: session_timeout 60 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: session_autoclose 300 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_file_size 1099511627776 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_xattr_size 65536 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: required_client_features {} 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: last_failure 0 2024-04-24T08:29:06.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: last_failure_osd_epoch 47 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_mds 1 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: in 0 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: up {0=24223} 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: failed 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: damaged 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: stopped 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_pools [3] 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_pool 2 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: inline_data enabled 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: balancer 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: bal_rank_mask -1 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: standby_count_wanted 1 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: qdb_cluster leader: 0 members: 2024-04-24T08:29:06.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Standby daemons: 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 ceph-mon[85282]: mon.smithi090@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-24T08:29:06.070 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 bash[85263]: 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 2024-04-24T08:29:06.071 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:05 smithi090 systemd[1]: Started Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:29:06.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:06.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:06.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: pgmap v145: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:06.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:06.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: mon.smithi090 calling monitor election 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: monmap epoch 2 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: last_changed 2024-04-24T08:14:30.195856+0000 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: created 2024-04-24T08:10:48.372320+0000 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: min_mon_release 18 (reef) 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: election_strategy: 1 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.smithi149 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: mgrmap e33: smithi090.jsrfxo(active, since 4m), standbys: smithi149.vavftl 2024-04-24T08:29:06.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:06.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:06.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:06.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: from='mgr.14660 ' entity='' 2024-04-24T08:29:06.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:06 smithi090 ceph-mon[85282]: mgrmap e34: smithi090.jsrfxo(active, since 4m), standbys: smithi149.vavftl 2024-04-24T08:29:06.373 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:06.373 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: pgmap v145: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: from='mgr.14660 172.21.15.90:0/1731743178' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: mon.smithi090 calling monitor election 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: monmap epoch 2 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: last_changed 2024-04-24T08:14:30.195856+0000 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: created 2024-04-24T08:10:48.372320+0000 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: min_mon_release 18 (reef) 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: election_strategy: 1 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.smithi149 2024-04-24T08:29:06.374 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: mgrmap e33: smithi090.jsrfxo(active, since 4m), standbys: smithi149.vavftl 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: from='mgr.14660 ' entity='' 2024-04-24T08:29:06.375 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:06 smithi149 ceph-mon[31282]: mgrmap e34: smithi090.jsrfxo(active, since 4m), standbys: smithi149.vavftl 2024-04-24T08:29:15.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: Active manager daemon smithi090.jsrfxo restarted 2024-04-24T08:29:15.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:29:15.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: osdmap e50: 6 total, 6 up, 6 in 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: mgrmap e35: smithi090.jsrfxo(active, starting, since 0.0106647s), standbys: smithi149.vavftl 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:29:15.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:29:15.261 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:15 smithi090 ceph-mon[85282]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: Active manager daemon smithi090.jsrfxo restarted 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: Activating manager daemon smithi090.jsrfxo 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: osdmap e50: 6 total, 6 up, 6 in 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: mgrmap e35: smithi090.jsrfxo(active, starting, since 0.0106647s), standbys: smithi149.vavftl 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi090.jsrfxo", "id": "smithi090.jsrfxo"}]: dispatch 2024-04-24T08:29:15.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr metadata", "who": "smithi149.vavftl", "id": "smithi149.vavftl"}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:29:15.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:15 smithi149 ceph-mon[31282]: Manager daemon smithi090.jsrfxo is now available 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:29:16.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:29:16.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:16 smithi090 ceph-mon[85282]: mgrmap e36: smithi090.jsrfxo(active, since 1.02503s), standbys: smithi149.vavftl 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl restarted 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: Standby manager daemon smithi149.vavftl started 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/crt"}]: dispatch 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:29:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.vavftl/key"}]: dispatch 2024-04-24T08:29:16.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.? 172.21.15.149:0/3478183505' entity='mgr.smithi149.vavftl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:29:16.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:16.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:29:16.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.jsrfxo/trash_purge_schedule"}]: dispatch 2024-04-24T08:29:16.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:16 smithi149 ceph-mon[31282]: mgrmap e36: smithi090.jsrfxo(active, since 1.02503s), standbys: smithi149.vavftl 2024-04-24T08:29:17.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:17 smithi149 ceph-mon[31282]: [24/Apr/2024:08:29:15] ENGINE Bus STARTING 2024-04-24T08:29:17.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:17 smithi149 ceph-mon[31282]: [24/Apr/2024:08:29:15] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:29:17.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:17 smithi149 ceph-mon[31282]: [24/Apr/2024:08:29:15] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:29:17.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:17 smithi149 ceph-mon[31282]: [24/Apr/2024:08:29:15] ENGINE Bus STARTED 2024-04-24T08:29:17.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:17 smithi149 ceph-mon[31282]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:17 smithi090 ceph-mon[85282]: [24/Apr/2024:08:29:15] ENGINE Bus STARTING 2024-04-24T08:29:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:17 smithi090 ceph-mon[85282]: [24/Apr/2024:08:29:15] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T08:29:17.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:17 smithi090 ceph-mon[85282]: [24/Apr/2024:08:29:15] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T08:29:17.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:17 smithi090 ceph-mon[85282]: [24/Apr/2024:08:29:15] ENGINE Bus STARTED 2024-04-24T08:29:17.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:17 smithi090 ceph-mon[85282]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:18.369 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:18 smithi149 ceph-mon[31282]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:18.369 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:18 smithi149 ceph-mon[31282]: mgrmap e37: smithi090.jsrfxo(active, since 2s), standbys: smithi149.vavftl 2024-04-24T08:29:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:18 smithi090 ceph-mon[85282]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:18 smithi090 ceph-mon[85282]: mgrmap e37: smithi090.jsrfxo(active, since 2s), standbys: smithi149.vavftl 2024-04-24T08:29:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:20 smithi149 ceph-mon[31282]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:20 smithi149 ceph-mon[31282]: mgrmap e38: smithi090.jsrfxo(active, since 4s), standbys: smithi149.vavftl 2024-04-24T08:29:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:20 smithi090 ceph-mon[85282]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:20 smithi090 ceph-mon[85282]: mgrmap e38: smithi090.jsrfxo(active, since 4s), standbys: smithi149.vavftl 2024-04-24T08:29:21.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:21 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:21.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:21 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:21.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:21 smithi149 ceph-mon[31282]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:21.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:21 smithi090 ceph-mon[85282]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:29:23.678 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.679 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:23 smithi090 ceph-mon[85282]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-24T08:29:23.693 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:23 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.693 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:23 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.694 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:23 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.694 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:23 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:23.694 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:23 smithi149 ceph-mon[31282]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-24T08:29:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:25 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:25 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:25 smithi149 ceph-mon[31282]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-24T08:29:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:25 smithi090 ceph-mon[85282]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-24T08:29:28.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:28 smithi149 ceph-mon[31282]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T08:29:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:28 smithi090 ceph-mon[85282]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T08:29:29.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 -- 172.21.15.90:0/3585057255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff548100e80 msgr2=0x7ff548104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 --2- 172.21.15.90:0/3585057255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff548100e80 0x7ff548104e50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff534009910 tx=0x7ff534009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 -- 172.21.15.90:0/3585057255 shutdown_connections 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 --2- 172.21.15.90:0/3585057255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff548100e80 0x7ff548104e50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 --2- 172.21.15.90:0/3585057255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5481004e0 0x7ff5481008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 -- 172.21.15.90:0/3585057255 >> 172.21.15.90:0/3585057255 conn(0x7ff5480fbda0 msgr2=0x7ff5480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 -- 172.21.15.90:0/3585057255 shutdown_connections 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.138+0000 7ff54d000700 1 -- 172.21.15.90:0/3585057255 wait complete. 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.139+0000 7ff54d000700 1 Processor -- start 2024-04-24T08:29:29.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff54d000700 1 -- start start 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff54d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff54d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff548100e80 0x7ff548073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff54d000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff548073ad0 con 0x7ff5481004e0 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff54d000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff548073c10 con 0x7ff548100e80 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff545e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff545e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48246/0 (socket says 172.21.15.90:48246) 2024-04-24T08:29:29.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.140+0000 7ff545e00700 1 -- 172.21.15.90:0/2544021942 learned_addr learned my addr 172.21.15.90:0/2544021942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.141+0000 7ff545400700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff548100e80 0x7ff548073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.141+0000 7ff545e00700 1 -- 172.21.15.90:0/2544021942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff548100e80 msgr2=0x7ff548073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.141+0000 7ff545e00700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff548100e80 0x7ff548073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.141+0000 7ff545e00700 1 -- 172.21.15.90:0/2544021942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff5481004e0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.141+0000 7ff545e00700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d8d0 tx=0x7ff53800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.142+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff538009940 con 0x7ff5481004e0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.142+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5481a2100 con 0x7ff5481004e0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.142+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff538010460 con 0x7ff5481004e0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.142+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53800f5f0 con 0x7ff5481004e0 2024-04-24T08:29:29.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.142+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5481a2510 con 0x7ff5481004e0 2024-04-24T08:29:29.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.143+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff548108850 con 0x7ff5481004e0 2024-04-24T08:29:29.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.144+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7ff5380105d0 con 0x7ff5481004e0 2024-04-24T08:29:29.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.145+0000 7ff53ea00700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 0x7ff52c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.145+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff53808eff0 con 0x7ff5481004e0 2024-04-24T08:29:29.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.146+0000 7ff545400700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 0x7ff52c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.146+0000 7ff545400700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 0x7ff52c0712a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff548074840 tx=0x7ff53401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:29.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.151+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff538057c00 con 0x7ff5481004e0 2024-04-24T08:29:29.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.407+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff548061190 con 0x7ff52c06edf0 2024-04-24T08:29:29.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.410+0000 7ff53ea00700 1 -- 172.21.15.90:0/2544021942 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7ff548061190 con 0x7ff52c06edf0 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 msgr2=0x7ff52c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 0x7ff52c0712a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff548074840 tx=0x7ff53401a040 comp rx=0 tx=0).stop 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 msgr2=0x7ff548073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d8d0 tx=0x7ff53800dc90 comp rx=0 tx=0).stop 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 shutdown_connections 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff52c06edf0 0x7ff52c0712a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5481004e0 0x7ff548073050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 --2- 172.21.15.90:0/2544021942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff548100e80 0x7ff548073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.414+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 >> 172.21.15.90:0/2544021942 conn(0x7ff5480fbda0 msgr2=0x7ff5480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:29.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.415+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 shutdown_connections 2024-04-24T08:29:29.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.415+0000 7ff54d000700 1 -- 172.21.15.90:0/2544021942 wait complete. 2024-04-24T08:29:29.439 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 -- 172.21.15.90:0/1019534793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a41004e0 msgr2=0x7f41a41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 --2- 172.21.15.90:0/1019534793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a41004e0 0x7f41a41008b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4194009910 tx=0x7f4194009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 -- 172.21.15.90:0/1019534793 shutdown_connections 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 --2- 172.21.15.90:0/1019534793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a4100e80 0x7f41a4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 --2- 172.21.15.90:0/1019534793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a41004e0 0x7f41a41008b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 -- 172.21.15.90:0/1019534793 >> 172.21.15.90:0/1019534793 conn(0x7f41a40fbda0 msgr2=0x7f41a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.584+0000 7f41aac00700 1 -- 172.21.15.90:0/1019534793 shutdown_connections 2024-04-24T08:29:29.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.585+0000 7f41aac00700 1 -- 172.21.15.90:0/1019534793 wait complete. 2024-04-24T08:29:29.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 Processor -- start 2024-04-24T08:29:29.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 -- start start 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a41004e0 0x7f41a4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4073ad0 con 0x7f41a4100e80 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.586+0000 7f41aac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4073c10 con 0x7f41a41004e0 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48258/0 (socket says 172.21.15.90:48258) 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 -- 172.21.15.90:0/4146667067 learned_addr learned my addr 172.21.15.90:0/4146667067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f41a3e00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a41004e0 0x7f41a4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 -- 172.21.15.90:0/4146667067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a41004e0 msgr2=0x7f41a4073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a41004e0 0x7f41a4073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.587+0000 7f419b400700 1 -- 172.21.15.90:0/4146667067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41940095f0 con 0x7f41a4100e80 2024-04-24T08:29:29.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f419b400700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d900 tx=0x7f418c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:29.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c009900 con 0x7f41a4100e80 2024-04-24T08:29:29.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f418c010460 con 0x7f41a4100e80 2024-04-24T08:29:29.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a41a20c0 con 0x7f41a4100e80 2024-04-24T08:29:29.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c00f5f0 con 0x7f41a4100e80 2024-04-24T08:29:29.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.588+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a41a24d0 con 0x7f41a4100e80 2024-04-24T08:29:29.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.589+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a41087f0 con 0x7f41a4100e80 2024-04-24T08:29:29.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.590+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f418c0105d0 con 0x7f41a4100e80 2024-04-24T08:29:29.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.591+0000 7f41a1600700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 0x7f41880712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:29.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.591+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f418c08eec0 con 0x7f41a4100e80 2024-04-24T08:29:29.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.591+0000 7f41a3e00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 0x7f41880712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:29.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.592+0000 7f41a3e00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 0x7f41880712a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f419401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:29.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.596+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f418c057a90 con 0x7f41a4100e80 2024-04-24T08:29:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.857+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a4061190 con 0x7f418806edf0 2024-04-24T08:29:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.859+0000 7f41a1600700 1 -- 172.21.15.90:0/4146667067 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f41a4061190 con 0x7f418806edf0 2024-04-24T08:29:29.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 msgr2=0x7f41880712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 0x7f41880712a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f419401a040 comp rx=0 tx=0).stop 2024-04-24T08:29:29.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 msgr2=0x7f41a4073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d900 tx=0x7f418c00dcc0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 shutdown_connections 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f418806edf0 0x7f41880712a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a41004e0 0x7f41a4073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 --2- 172.21.15.90:0/4146667067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41a4100e80 0x7f41a4073590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 >> 172.21.15.90:0/4146667067 conn(0x7f41a40fbda0 msgr2=0x7f41a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 shutdown_connections 2024-04-24T08:29:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:29.862+0000 7f41aac00700 1 -- 172.21.15.90:0/4146667067 wait complete. 2024-04-24T08:29:30.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.020+0000 7fbb9f000700 1 -- 172.21.15.90:0/222636594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 msgr2=0x7fbb98106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 --2- 172.21.15.90:0/222636594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb98106ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009910 tx=0x7fbb80009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:30.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 -- 172.21.15.90:0/222636594 shutdown_connections 2024-04-24T08:29:30.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 --2- 172.21.15.90:0/222636594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb98106ff0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 --2- 172.21.15.90:0/222636594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 0x7fbb9810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 -- 172.21.15.90:0/222636594 >> 172.21.15.90:0/222636594 conn(0x7fbb98076020 msgr2=0x7fbb98078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:30.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 -- 172.21.15.90:0/222636594 shutdown_connections 2024-04-24T08:29:30.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.021+0000 7fbb9f000700 1 -- 172.21.15.90:0/222636594 wait complete. 2024-04-24T08:29:30.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.022+0000 7fbb9f000700 1 Processor -- start 2024-04-24T08:29:30.023 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:29 smithi090 ceph-mon[85282]: Detected new or changed devices on smithi149 2024-04-24T08:29:30.023 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:30.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:30.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:29:30.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:29 smithi090 ceph-mon[85282]: pgmap v10: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-24T08:29:30.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.022+0000 7fbb9f000700 1 -- start start 2024-04-24T08:29:30.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.022+0000 7fbb9f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb9f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 0x7fbb9819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb9f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9819d140 con 0x7fbb98106b80 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb9f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981a0ed0 con 0x7fbb9810c8f0 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46056/0 (socket says 172.21.15.90:46056) 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 -- 172.21.15.90:0/1162490683 learned_addr learned my addr 172.21.15.90:0/1162490683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb8fe00700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 0x7fbb9819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 -- 172.21.15.90:0/1162490683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 msgr2=0x7fbb9819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 0x7fbb9819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.023+0000 7fbb97e00700 1 -- 172.21.15.90:0/1162490683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800095f0 con 0x7fbb98106b80 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb97e00700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b4b0 tx=0x7fbb8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb88011800 con 0x7fbb98106b80 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb88011e20 con 0x7fbb98106b80 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb981a11b0 con 0x7fbb98106b80 2024-04-24T08:29:30.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8800f2e0 con 0x7fbb98106b80 2024-04-24T08:29:30.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.024+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb981a1700 con 0x7fbb98106b80 2024-04-24T08:29:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:29 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi149 2024-04-24T08:29:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:29 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:29 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:29 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:29:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:29 smithi149 ceph-mon[31282]: pgmap v10: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-24T08:29:30.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.026+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9804f2a0 con 0x7fbb98106b80 2024-04-24T08:29:30.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.027+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fbb88010380 con 0x7fbb98106b80 2024-04-24T08:29:30.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.028+0000 7fbb95600700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 0x7fbb7c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.028+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb8808eb40 con 0x7fbb98106b80 2024-04-24T08:29:30.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.028+0000 7fbb8fe00700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 0x7fbb7c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.029+0000 7fbb8fe00700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 0x7fbb7c0712a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb9819db40 tx=0x7fbb8001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.033+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbb880578d0 con 0x7fbb98106b80 2024-04-24T08:29:30.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.285+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb981a1ae0 con 0x7fbb7c06edf0 2024-04-24T08:29:30.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.298+0000 7fbb95600700 1 -- 172.21.15.90:0/1162490683 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbb981a1ae0 con 0x7fbb7c06edf0 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (2m) 7s ago 16m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (16m) 7s ago 16m 8232k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (15m) 9s ago 15m 8150k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (16m) 7s ago 16m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (15m) 9s ago 15m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (110s) 7s ago 16m 73.9M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (9m) 7s ago 9m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:29:30.299 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (9m) 7s ago 9m 16.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (9m) 9s ago 9m 154M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (9m) 9s ago 9m 16.2M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (5m) 7s ago 18m 480M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (4m) 9s ago 15m 448M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (25s) 7s ago 18m 28.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (15m) 9s ago 15m 48.6M 2048M 18.2.0 dc2bc1663786 db86e18af0bf 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (3m) 7s ago 16m 7889k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (3m) 9s ago 15m 7671k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (14m) 7s ago 14m 217M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (13m) 7s ago 13m 225M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (12m) 7s ago 12m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (12m) 9s ago 12m 248M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:29:30.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (11m) 9s ago 11m 220M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:29:30.301 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (11m) 9s ago 11m 202M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:29:30.301 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (2m) 7s ago 15m 62.3M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 msgr2=0x7fbb7c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 0x7fbb7c0712a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb9819db40 tx=0x7fbb8001a040 comp rx=0 tx=0).stop 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 msgr2=0x7fbb9819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b4b0 tx=0x7fbb8800b870 comp rx=0 tx=0).stop 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 shutdown_connections 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7c06edf0 0x7fbb7c0712a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb98106b80 0x7fbb9819c520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 --2- 172.21.15.90:0/1162490683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb9810c8f0 0x7fbb9819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 >> 172.21.15.90:0/1162490683 conn(0x7fbb98076020 msgr2=0x7fbb98077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 shutdown_connections 2024-04-24T08:29:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.301+0000 7fbb9f000700 1 -- 172.21.15.90:0/1162490683 wait complete. 2024-04-24T08:29:30.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.466+0000 7fc911e00700 1 -- 172.21.15.90:0/1696083666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 msgr2=0x7fc90c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.466+0000 7fc911e00700 1 --2- 172.21.15.90:0/1696083666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c104500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009910 tx=0x7fc8fc009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:30.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 -- 172.21.15.90:0/1696083666 shutdown_connections 2024-04-24T08:29:30.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 --2- 172.21.15.90:0/1696083666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 0x7fc90c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 --2- 172.21.15.90:0/1696083666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c104500 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 -- 172.21.15.90:0/1696083666 >> 172.21.15.90:0/1696083666 conn(0x7fc90c075130 msgr2=0x7fc90c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 -- 172.21.15.90:0/1696083666 shutdown_connections 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.467+0000 7fc911e00700 1 -- 172.21.15.90:0/1696083666 wait complete. 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.468+0000 7fc911e00700 1 Processor -- start 2024-04-24T08:29:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.468+0000 7fc911e00700 1 -- start start 2024-04-24T08:29:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc911e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 0x7fc90c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc911e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc911e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c198c60 con 0x7fc90c104130 2024-04-24T08:29:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc911e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c19c9f0 con 0x7fc90c0ffbd0 2024-04-24T08:29:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc90a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc90a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46060/0 (socket says 172.21.15.90:46060) 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc90a000700 1 -- 172.21.15.90:0/824700714 learned_addr learned my addr 172.21.15.90:0/824700714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.469+0000 7fc90aa00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 0x7fc90c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc90a000700 1 -- 172.21.15.90:0/824700714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 msgr2=0x7fc90c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc90a000700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 0x7fc90c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc90a000700 1 -- 172.21.15.90:0/824700714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc0095f0 con 0x7fc90c104130 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc90a000700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d8d0 tx=0x7fc8f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f80098a0 con 0x7fc90c104130 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f8010460 con 0x7fc90c104130 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.470+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c19ccd0 con 0x7fc90c104130 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.471+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c19d220 con 0x7fc90c104130 2024-04-24T08:29:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.471+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f800f5f0 con 0x7fc90c104130 2024-04-24T08:29:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.472+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c04f2a0 con 0x7fc90c104130 2024-04-24T08:29:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.473+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fc8f800f750 con 0x7fc90c104130 2024-04-24T08:29:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.474+0000 7fc910a00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 0x7fc8f00712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.474+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc8f808ee90 con 0x7fc90c104130 2024-04-24T08:29:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.474+0000 7fc90aa00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 0x7fc8f00712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.475+0000 7fc90aa00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 0x7fc8f00712f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009fd0 tx=0x7fc8fc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.479+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc8f8057b00 con 0x7fc90c104130 2024-04-24T08:29:30.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.812+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc90c19d490 con 0x7fc90c104130 2024-04-24T08:29:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.813+0000 7fc910a00700 1 -- 172.21.15.90:0/824700714 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7fc8f805b1c0 con 0x7fc90c104130 2024-04-24T08:29:30.814 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:29:30.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:29:30.816 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:29:30.816 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 11, 2024-04-24T08:29:30.816 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 3 2024-04-24T08:29:30.816 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:29:30.816 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 msgr2=0x7fc8f00712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 0x7fc8f00712f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009fd0 tx=0x7fc8fc01a040 comp rx=0 tx=0).stop 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 msgr2=0x7fc90c198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d8d0 tx=0x7fc8f800dbe0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 shutdown_connections 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc8f006ee40 0x7fc8f00712f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffbd0 0x7fc90c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 --2- 172.21.15.90:0/824700714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc90c104130 0x7fc90c198580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 >> 172.21.15.90:0/824700714 conn(0x7fc90c075130 msgr2=0x7fc90c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 shutdown_connections 2024-04-24T08:29:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.816+0000 7fc911e00700 1 -- 172.21.15.90:0/824700714 wait complete. 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.976+0000 7f85a7800700 1 -- 172.21.15.90:0/1383558662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 msgr2=0x7f85a0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.976+0000 7f85a7800700 1 --2- 172.21.15.90:0/1383558662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0104e50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f858c009910 tx=0x7f858c009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 -- 172.21.15.90:0/1383558662 shutdown_connections 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 --2- 172.21.15.90:0/1383558662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0104e50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 --2- 172.21.15.90:0/1383558662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 0x7f85a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 -- 172.21.15.90:0/1383558662 >> 172.21.15.90:0/1383558662 conn(0x7f85a00fbda0 msgr2=0x7f85a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 -- 172.21.15.90:0/1383558662 shutdown_connections 2024-04-24T08:29:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.977+0000 7f85a7800700 1 -- 172.21.15.90:0/1383558662 wait complete. 2024-04-24T08:29:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.978+0000 7f85a7800700 1 Processor -- start 2024-04-24T08:29:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.979+0000 7f85a7800700 1 -- start start 2024-04-24T08:29:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.979+0000 7f85a7800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 0x7f85a0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.979+0000 7f85a7800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.979+0000 7f85a7800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0196b50 con 0x7f85a0100e80 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.979+0000 7f85a7800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a019a8e0 con 0x7f85a01004e0 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46074/0 (socket says 172.21.15.90:46074) 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 -- 172.21.15.90:0/3909781227 learned_addr learned my addr 172.21.15.90:0/3909781227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f85a4e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 0x7f85a0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 -- 172.21.15.90:0/3909781227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 msgr2=0x7f85a0195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 0x7f85a0195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 -- 172.21.15.90:0/3909781227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0095f0 con 0x7f85a0100e80 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.980+0000 7f859fe00700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f858c00bda0 tx=0x7f858c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.981+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858c01d070 con 0x7f85a0100e80 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.981+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f858c005dc0 con 0x7f85a0100e80 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.981+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a019ab60 con 0x7f85a0100e80 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.981+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858c017400 con 0x7f85a0100e80 2024-04-24T08:29:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.981+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a019b050 con 0x7f85a0100e80 2024-04-24T08:29:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.982+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8580005320 con 0x7f85a0100e80 2024-04-24T08:29:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.984+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f858c003a60 con 0x7f85a0100e80 2024-04-24T08:29:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.985+0000 7f859d600700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 0x7f85880712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.985+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f858c0903e0 con 0x7f85a0100e80 2024-04-24T08:29:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.985+0000 7f85a4e00700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 0x7f85880712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:30.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.986+0000 7f85a4e00700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 0x7f85880712a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f859400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:30.990+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f858c058f30 con 0x7f85a0100e80 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:31.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:29:31.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.269+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8580005cc0 con 0x7f85a0100e80 2024-04-24T08:29:31.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.270+0000 7f859d600700 1 -- 172.21.15.90:0/3909781227 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f858c05c5f0 con 0x7f85a0100e80 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:29:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:29:31.272 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:29:31.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:29:31.274 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:31.274 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.273+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 msgr2=0x7f85880712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.273+0000 7f85a7800700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 0x7f85880712a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f859400b410 comp rx=0 tx=0).stop 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.273+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 msgr2=0x7f85a0196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.273+0000 7f85a7800700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f858c00bda0 tx=0x7f858c00bdd0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 shutdown_connections 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f858806edf0 0x7f85880712a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a01004e0 0x7f85a0195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 --2- 172.21.15.90:0/3909781227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0100e80 0x7f85a0196470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 >> 172.21.15.90:0/3909781227 conn(0x7f85a00fbda0 msgr2=0x7f85a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:31.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 shutdown_connections 2024-04-24T08:29:31.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.274+0000 7f85a7800700 1 -- 172.21.15.90:0/3909781227 wait complete. 2024-04-24T08:29:31.276 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:30 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:29:31.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 -- 172.21.15.90:0/2830648474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22500739a0 msgr2=0x7f225010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 --2- 172.21.15.90:0/2830648474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22500739a0 0x7f225010c5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f223c009940 tx=0x7f223c009c50 comp rx=0 tx=0).stop 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 -- 172.21.15.90:0/2830648474 shutdown_connections 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 --2- 172.21.15.90:0/2830648474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22500739a0 0x7f225010c5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 --2- 172.21.15.90:0/2830648474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250073090 0x7f2250073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 -- 172.21.15.90:0/2830648474 >> 172.21.15.90:0/2830648474 conn(0x7f22500fbda0 msgr2=0x7f22500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 -- 172.21.15.90:0/2830648474 shutdown_connections 2024-04-24T08:29:31.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.428+0000 7f2255000700 1 -- 172.21.15.90:0/2830648474 wait complete. 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.429+0000 7f2255000700 1 Processor -- start 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.429+0000 7f2255000700 1 -- start start 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f2255000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f2255000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22500739a0 0x7f2250198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f2255000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250198e40 con 0x7f2250073090 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f2255000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225019cbd0 con 0x7f22500739a0 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f224de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f224de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46078/0 (socket says 172.21.15.90:46078) 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f224de00700 1 -- 172.21.15.90:0/419248608 learned_addr learned my addr 172.21.15.90:0/419248608 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.430+0000 7f224d400700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22500739a0 0x7f2250198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f224de00700 1 -- 172.21.15.90:0/419248608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22500739a0 msgr2=0x7f2250198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f224de00700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22500739a0 0x7f2250198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f224de00700 1 -- 172.21.15.90:0/419248608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0095f0 con 0x7f2250073090 2024-04-24T08:29:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f224de00700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f224000d8d0 tx=0x7f224000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22400098a0 con 0x7f2250073090 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2240010460 con 0x7f2250073090 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225019ceb0 con 0x7f2250073090 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224000f5f0 con 0x7f2250073090 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.431+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225019d3d0 con 0x7f2250073090 2024-04-24T08:29:31.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.433+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250109ce0 con 0x7f2250073090 2024-04-24T08:29:31.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.434+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f2240009a00 con 0x7f2250073090 2024-04-24T08:29:31.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.434+0000 7f2246a00700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 0x7f22340712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.435+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f224008eff0 con 0x7f2250073090 2024-04-24T08:29:31.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.435+0000 7f224d400700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 0x7f22340712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.436+0000 7f224d400700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 0x7f22340712a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f223c006010 tx=0x7f223c005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.440+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f22400c3ff0 con 0x7f2250073090 2024-04-24T08:29:31.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.689+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2250199660 con 0x7f223406edf0 2024-04-24T08:29:31.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.691+0000 7f2246a00700 1 -- 172.21.15.90:0/419248608 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2250199660 con 0x7f223406edf0 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "3/23 daemons upgraded", 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:29:31.692 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:29:31.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 msgr2=0x7f22340712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 0x7f22340712a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f223c006010 tx=0x7f223c005a90 comp rx=0 tx=0).stop 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 msgr2=0x7f2250198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f224000d8d0 tx=0x7f224000dbe0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 shutdown_connections 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f223406edf0 0x7f22340712a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2250073090 0x7f2250198220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 --2- 172.21.15.90:0/419248608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22500739a0 0x7f2250198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.693+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 >> 172.21.15.90:0/419248608 conn(0x7f22500fbda0 msgr2=0x7f2250106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.694+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 shutdown_connections 2024-04-24T08:29:31.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.694+0000 7f2255000700 1 -- 172.21.15.90:0/419248608 wait complete. 2024-04-24T08:29:31.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 -- 172.21.15.90:0/1842024619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 msgr2=0x7f3d700734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 --2- 172.21.15.90:0/1842024619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d700734c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009910 tx=0x7f3d60009c20 comp rx=0 tx=0).stop 2024-04-24T08:29:31.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 -- 172.21.15.90:0/1842024619 shutdown_connections 2024-04-24T08:29:31.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 --2- 172.21.15.90:0/1842024619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 0x7f3d7010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 --2- 172.21.15.90:0/1842024619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d700734c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 -- 172.21.15.90:0/1842024619 >> 172.21.15.90:0/1842024619 conn(0x7f3d700fbda0 msgr2=0x7f3d700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:31.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 -- 172.21.15.90:0/1842024619 shutdown_connections 2024-04-24T08:29:31.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.839+0000 7f3d75400700 1 -- 172.21.15.90:0/1842024619 wait complete. 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.840+0000 7f3d75400700 1 Processor -- start 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.840+0000 7f3d75400700 1 -- start start 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d75400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 0x7f3d70198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d75400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019cb10 con 0x7f3d700730f0 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d75400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019cc80 con 0x7f3d70073a00 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46108/0 (socket says 172.21.15.90:46108) 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 -- 172.21.15.90:0/851688776 learned_addr learned my addr 172.21.15.90:0/851688776 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:29:31.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6de00700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 0x7f3d70198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 -- 172.21.15.90:0/851688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 msgr2=0x7f3d70198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 0x7f3d70198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.841+0000 7f3d6e800700 1 -- 172.21.15.90:0/851688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600095f0 con 0x7f3d700730f0 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d6e800700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3d60006010 tx=0x7f3d6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d6001d070 con 0x7f3d700730f0 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d6000f460 con 0x7f3d700730f0 2024-04-24T08:29:31.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d60017610 con 0x7f3d700730f0 2024-04-24T08:29:31.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7019cf60 con 0x7f3d700730f0 2024-04-24T08:29:31.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.842+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7019d4b0 con 0x7f3d700730f0 2024-04-24T08:29:31.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.844+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d70109ce0 con 0x7f3d700730f0 2024-04-24T08:29:31.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.845+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f3d6000f5d0 con 0x7f3d700730f0 2024-04-24T08:29:31.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.845+0000 7f3d67400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 0x7f3d540712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:29:31.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.846+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3d60090520 con 0x7f3d700730f0 2024-04-24T08:29:31.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.846+0000 7f3d6de00700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 0x7f3d540712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:29:31.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.847+0000 7f3d6de00700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 0x7f3d540712a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009a10 tx=0x7f3d5c009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:29:31.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:31.851+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3d600590f0 con 0x7f3d700730f0 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: Detected new or changed devices on smithi090 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/824700714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-24T08:29:32.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:29:32.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:31 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3909781227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:29:32.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.176+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d701994c0 con 0x7f3d700730f0 2024-04-24T08:29:32.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.176+0000 7f3d67400700 1 -- 172.21.15.90:0/851688776 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3d6005c7b0 con 0x7f3d700730f0 2024-04-24T08:29:32.178 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:32.178 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:32.178 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:32.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 msgr2=0x7f3d540712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:32.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 0x7f3d540712a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009a10 tx=0x7f3d5c009190 comp rx=0 tx=0).stop 2024-04-24T08:29:32.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 msgr2=0x7f3d701981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3d60006010 tx=0x7f3d6000bd70 comp rx=0 tx=0).stop 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 shutdown_connections 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3d5406edf0 0x7f3d540712a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d700730f0 0x7f3d701981b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 --2- 172.21.15.90:0/851688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d70073a00 0x7f3d70198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 >> 172.21.15.90:0/851688776 conn(0x7f3d700fbda0 msgr2=0x7f3d70106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 shutdown_connections 2024-04-24T08:29:32.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:29:32.179+0000 7f3d75400700 1 -- 172.21.15.90:0/851688776 wait complete. 2024-04-24T08:29:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: Detected new or changed devices on smithi090 2024-04-24T08:29:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T08:29:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T08:29:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/824700714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:29:32.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:29:32.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-24T08:29:32.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:29:32.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:31 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/3909781227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:29:33.146 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:32 smithi149 ceph-mon[31282]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:29:33.146 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:32 smithi149 ceph-mon[31282]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:33.146 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:32 smithi149 ceph-mon[31282]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:29:33.146 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:32 smithi149 ceph-mon[31282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:29:33.146 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:32 smithi149 ceph-mon[31282]: from='client.? 172.21.15.90:0/851688776' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:29:33.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:32 smithi090 ceph-mon[85282]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:29:33.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:32 smithi090 ceph-mon[85282]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:29:33.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:32 smithi090 ceph-mon[85282]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:29:33.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:32 smithi090 ceph-mon[85282]: Updating smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:29:33.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:32 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/851688776' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:29:34.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:29:34.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:29:34.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T08:29:34.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: Updating smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:29:34.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T08:29:34.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:29:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:33 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T08:29:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:36 smithi149 ceph-mon[31282]: pgmap v13: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:36 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:36 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:36 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:36.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:36 smithi149 ceph-mon[31282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:36 smithi090 ceph-mon[85282]: pgmap v13: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:36.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:37.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:37 smithi149 ceph-mon[31282]: Upgrade: Updating mon.smithi149 2024-04-24T08:29:37.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:37 smithi149 ceph-mon[31282]: Deploying daemon mon.smithi149 on smithi149 2024-04-24T08:29:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:37 smithi090 ceph-mon[85282]: Upgrade: Updating mon.smithi149 2024-04-24T08:29:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:37 smithi090 ceph-mon[85282]: Deploying daemon mon.smithi149 on smithi149 2024-04-24T08:29:38.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:38 smithi149 ceph-mon[31282]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:38 smithi090 ceph-mon[85282]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:39.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:38 smithi149 systemd[1]: Stopping Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:29:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:39 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149[31268]: 2024-04-24T08:29:39.400+0000 7f57bd400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi149 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:29:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:39 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149[31268]: 2024-04-24T08:29:39.400+0000 7f57bd400700 -1 mon.smithi149@1(peon) e2 *** Got Signal Terminated *** 2024-04-24T08:29:39.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:39 smithi149 podman[66326]: 2024-04-24 08:29:39.566632701 +0000 UTC m=+0.324961431 container died db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, ceph=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T08:29:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 podman[66326]: 2024-04-24 08:29:40.018547671 +0000 UTC m=+0.776876424 container cleanup db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T08:29:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 bash[66326]: ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149 2024-04-24T08:29:40.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 podman[66340]: 2024-04-24 08:29:40.303165094 +0000 UTC m=+0.723293879 container remove db86e18af0bfa2997799231af7645d02612e63faacc1795abaf25965d8651093 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T08:29:41.225 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi149.service: Deactivated successfully. 2024-04-24T08:29:41.225 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 systemd[1]: Stopped Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:29:41.225 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:40 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi149.service: Consumed 19.861s CPU time. 2024-04-24T08:29:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:41 smithi149 systemd[1]: Starting Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:29:41.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:41 smithi149 podman[66471]: 2024-04-24 08:29:41.506385005 +0000 UTC m=+0.037642880 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:29:41.935 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:41 smithi149 podman[66471]: 2024-04-24 08:29:41.629439321 +0000 UTC m=+0.160697221 container create f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T08:29:41.937 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:41 smithi149 podman[66471]: 2024-04-24 08:29:41.933213133 +0000 UTC m=+0.464471076 container init f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:29:42.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:41 smithi149 podman[66471]: 2024-04-24 08:29:41.944492 +0000 UTC m=+0.475749873 container start f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:29:42.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 bash[66471]: f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 2024-04-24T08:29:42.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 systemd[1]: Started Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: pidfile_write: ignore empty --pid-file 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: load: jerasure load: lrc 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Git sha 0 2024-04-24T08:29:42.779 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: DB SUMMARY 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: DB Session ID: VJ90KARG7ZFCML5XHPBJ 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1626 Bytes 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi149/store.db dir, Total Num: 1, files: 000030.sst 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi149/store.db: 000028.log size: 4242576 ; 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:29:42.780 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.env: 0x56187aa5dcc0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.info_log: 0x5618badebb00 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.statistics: (nil) 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.use_fsync: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:29:42.781 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.db_log_dir: 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.wal_dir: 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:29:42.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.write_buffer_manager: 0x5618badba1e0 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:29:42.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.unordered_write: 0 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.row_cache: None 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.wal_filter: None 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:29:42.784 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.wal_compression: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:29:42.785 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:29:42.786 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_open_files: -1 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Compression algorithms supported: 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kZSTD supported: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:29:42.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000010 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.merge_operator: 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_filter: None 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:29:42.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:29:42.789 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:29:42.789 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5618badebce0) 2024-04-24T08:29:42.789 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:29:42.789 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_type: 0 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_index_type: 0 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_shortening: 1 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:29:42.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: checksum: 4 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: no_block_cache: 0 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache: 0x5618baded090 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_options: 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: capacity : 536870912 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_shard_bits : 4 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: strict_capacity_limit : 0 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_compressed: (nil) 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: persistent_cache: (nil) 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size: 4096 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size_deviation: 10 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_restart_interval: 16 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_block_restart_interval: 1 2024-04-24T08:29:42.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: metadata_block_size: 4096 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: partition_filters: 0 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: use_delta_encoding: 1 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: filter_policy: bloomfilter 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: whole_key_filtering: 1 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: verify_compression: 0 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: format_version: 5 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_index_compression: 1 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_align: 0 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_auto_readahead_size: 262144 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: prepopulate_block_cache: 0 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:29:42.792 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression: NoCompression 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.num_levels: 7 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:29:42.793 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:29:42.794 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:29:42.795 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:29:42.796 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:29:42.797 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.ttl: 2592000 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:29:42.798 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16840, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 65e150f3-c259-4a1b-abcb-8c7e876726d3 2024-04-24T08:29:42.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947382407792, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947382440936, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2648281, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16845, "largest_seqno": 18843, "table_properties": {"data_size": 2640455, "index_size": 4860, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 18129, "raw_average_key_size": 23, "raw_value_size": 2624182, "raw_average_value_size": 3425, "num_data_blocks": 221, "num_entries": 766, "num_filter_entries": 766, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713947382, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "65e150f3-c259-4a1b-abcb-8c7e876726d3", "db_session_id": "VJ90KARG7ZFCML5XHPBJ", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947382441380, "job": 1, "event": "recovery_finished"} 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi149/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5618baeda000 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: DB pointer 0x5618baec4000 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** DB Stats ** 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:29:42.800 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: L0 1/0 2.53 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 221.8 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: L6 1/0 7.96 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T08:29:42.801 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Sum 2/0 10.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 221.8 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 221.8 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 221.8 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:29:42.802 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative compaction: 0.00 GB write, 53.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval compaction: 0.00 GB write, 53.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache BinnedLRUCache@0x5618baded090#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: starting mon.smithi149 rank 1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi149 fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???) e2 preinit fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).mds e17 new map 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).mds e17 print_map 2024-04-24T08:29:42.803 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: e17 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: legacy client fscid: 1 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Filesystem 'cephfs' (1) 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: fs_name cephfs 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: epoch 17 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: tableserver 0 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: root 0 2024-04-24T08:29:42.804 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: session_timeout 60 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: session_autoclose 300 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_file_size 1099511627776 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_xattr_size 65536 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: required_client_features {} 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: last_failure 0 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: last_failure_osd_epoch 47 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_mds 1 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: in 0 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: up {0=24223} 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: failed 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: damaged 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: stopped 2024-04-24T08:29:42.805 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_pools [3] 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: metadata_pool 2 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: inline_data enabled 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: balancer 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: bal_rank_mask -1 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: standby_count_wanted 1 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: qdb_cluster leader: 0 members: 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Standby daemons: 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:42.806 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:29:42.807 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T08:29:42.807 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:42.807 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:42.807 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T08:29:42.807 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:42 smithi149 ceph-mon[66490]: mon.smithi149@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-04-24T08:29:43.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:43 smithi149 ceph-mon[66490]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:43.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:43 smithi090 ceph-mon[85282]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:44.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:44.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: mon.smithi149 calling monitor election 2024-04-24T08:29:44.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: mon.smithi090 calling monitor election 2024-04-24T08:29:44.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:44.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: monmap epoch 3 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: last_changed 2024-04-24T08:29:43.017487+0000 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: created 2024-04-24T08:10:48.372320+0000 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: min_mon_release 19 (squid) 2024-04-24T08:29:44.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: election_strategy: 1 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.smithi149 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: osdmap e50: 6 total, 6 up, 6 in 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: mgrmap e38: smithi090.jsrfxo(active, since 28s), standbys: smithi149.vavftl 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:44.296 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:44.406 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:44.406 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: mon.smithi149 calling monitor election 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: mon.smithi090 calling monitor election 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: mon.smithi090 is new leader, mons smithi090,smithi149 in quorum (ranks 0,1) 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: monmap epoch 3 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: fsid 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: last_changed 2024-04-24T08:29:43.017487+0000 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: created 2024-04-24T08:10:48.372320+0000 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: min_mon_release 19 (squid) 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: election_strategy: 1 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-24T08:29:44.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.smithi149 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: osdmap e50: 6 total, 6 up, 6 in 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: mgrmap e38: smithi090.jsrfxo(active, since 28s), standbys: smithi149.vavftl 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:44.408 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:29:46.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:46 smithi149 ceph-mon[66490]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:46.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:46 smithi090 ceph-mon[85282]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:29:48.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:48 smithi149 ceph-mon[66490]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:48.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:48 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:48.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:48 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:48 smithi090 ceph-mon[85282]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:48 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:48.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:48 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:50 smithi149 ceph-mon[66490]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:50 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:50 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:50 smithi090 ceph-mon[85282]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:50 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:50 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:52.419 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:52 smithi149 ceph-mon[66490]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:52.423 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:52 smithi090 ceph-mon[85282]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:54 smithi149 ceph-mon[66490]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:54 smithi090 ceph-mon[85282]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:56.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:29:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:57.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:57 smithi090 ceph-mon[85282]: Reconfiguring mon.smithi090 (monmap changed)... 2024-04-24T08:29:57.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:57 smithi090 ceph-mon[85282]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-24T08:29:57.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:57 smithi149 ceph-mon[66490]: Reconfiguring mon.smithi090 (monmap changed)... 2024-04-24T08:29:57.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:57 smithi149 ceph-mon[66490]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-24T08:29:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:58.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:58.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:58.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:29:58.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:29:58.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.jsrfxo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:29:58.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:29:59.232 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:59 smithi090 ceph-mon[85282]: Reconfiguring mgr.smithi090.jsrfxo (monmap changed)... 2024-04-24T08:29:59.233 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:29:59 smithi090 ceph-mon[85282]: Reconfiguring daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:29:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:59 smithi149 ceph-mon[66490]: Reconfiguring mgr.smithi090.jsrfxo (monmap changed)... 2024-04-24T08:29:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:29:59 smithi149 ceph-mon[66490]: Reconfiguring daemon mgr.smithi090.jsrfxo on smithi090 2024-04-24T08:30:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:00 smithi149 ceph-mon[66490]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:00 smithi149 ceph-mon[66490]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:00 smithi090 ceph-mon[85282]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:00 smithi090 ceph-mon[85282]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:01.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: Unable to update caps for client.ceph-exporter.smithi090 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:30:01.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:01 smithi090 ceph-mon[85282]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:01.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:01.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: Unable to update caps for client.ceph-exporter.smithi090 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:30:01.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:01 smithi149 ceph-mon[66490]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 -- 172.21.15.90:0/3350308047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000730b0 msgr2=0x7f5400073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 --2- 172.21.15.90:0/3350308047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000730b0 0x7f5400073480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009940 tx=0x7f53f0009c50 comp rx=0 tx=0).stop 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 -- 172.21.15.90:0/3350308047 shutdown_connections 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 --2- 172.21.15.90:0/3350308047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000739c0 0x7f540010c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 --2- 172.21.15.90:0/3350308047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000730b0 0x7f5400073480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 -- 172.21.15.90:0/3350308047 >> 172.21.15.90:0/3350308047 conn(0x7f54000fbd80 msgr2=0x7f54000fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 -- 172.21.15.90:0/3350308047 shutdown_connections 2024-04-24T08:30:02.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.349+0000 7f5405400700 1 -- 172.21.15.90:0/3350308047 wait complete. 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.350+0000 7f5405400700 1 Processor -- start 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f5405400700 1 -- start start 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f5405400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000730b0 0x7f5400198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f5405400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f5405400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5400198c20 con 0x7f54000739c0 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f5405400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5400198d60 con 0x7f54000730b0 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f53fde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f53fde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39066/0 (socket says 172.21.15.90:39066) 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.351+0000 7f53fde00700 1 -- 172.21.15.90:0/2231732782 learned_addr learned my addr 172.21.15.90:0/2231732782 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53fe800700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000730b0 0x7f5400198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53fde00700 1 -- 172.21.15.90:0/2231732782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000730b0 msgr2=0x7f5400198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53fde00700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000730b0 0x7f5400198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53fde00700 1 -- 172.21.15.90:0/2231732782 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00095f0 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53fde00700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c930 tx=0x7f53ec00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.352+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53ec007ad0 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.353+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53ec007c30 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.353+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540019cbb0 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.353+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53ec018610 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.353+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540019d100 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.354+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5400109ca0 con 0x7f54000739c0 2024-04-24T08:30:02.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.355+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f53ec00f450 con 0x7f54000739c0 2024-04-24T08:30:02.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.356+0000 7f53f7400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 0x7f53e40712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.356+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f53ec08f290 con 0x7f54000739c0 2024-04-24T08:30:02.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.356+0000 7f53fe800700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 0x7f53e40712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:02.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.357+0000 7f53fe800700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 0x7f53e40712a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005cd0 tx=0x7f53f0005c20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:02.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.361+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f53ec057f00 con 0x7f54000739c0 2024-04-24T08:30:02.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.619+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f540019d380 con 0x7f53e406edf0 2024-04-24T08:30:02.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.622+0000 7f53f7400700 1 -- 172.21.15.90:0/2231732782 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f540019d380 con 0x7f53e406edf0 2024-04-24T08:30:02.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 msgr2=0x7f53e40712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 0x7f53e40712a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005cd0 tx=0x7f53f0005c20 comp rx=0 tx=0).stop 2024-04-24T08:30:02.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 msgr2=0x7f54001985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c930 tx=0x7f53ec00ccf0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 shutdown_connections 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f53e406edf0 0x7f53e40712a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54000730b0 0x7f5400198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 --2- 172.21.15.90:0/2231732782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54000739c0 0x7f54001985d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 >> 172.21.15.90:0/2231732782 conn(0x7f54000fbd80 msgr2=0x7f5400106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 shutdown_connections 2024-04-24T08:30:02.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.625+0000 7f5405400700 1 -- 172.21.15.90:0/2231732782 wait complete. 2024-04-24T08:30:02.644 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 -- 172.21.15.90:0/2070268945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a4104130 msgr2=0x7f67a4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 --2- 172.21.15.90:0/2070268945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a4104130 0x7f67a4104500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6794009910 tx=0x7f6794009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 -- 172.21.15.90:0/2070268945 shutdown_connections 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 --2- 172.21.15.90:0/2070268945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a40ffbd0 0x7f67a4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 --2- 172.21.15.90:0/2070268945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a4104130 0x7f67a4104500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.784+0000 7f67a8c00700 1 -- 172.21.15.90:0/2070268945 >> 172.21.15.90:0/2070268945 conn(0x7f67a4075130 msgr2=0x7f67a4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.785+0000 7f67a8c00700 1 -- 172.21.15.90:0/2070268945 shutdown_connections 2024-04-24T08:30:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.785+0000 7f67a8c00700 1 -- 172.21.15.90:0/2070268945 wait complete. 2024-04-24T08:30:02.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 Processor -- start 2024-04-24T08:30:02.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 -- start start 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4104130 0x7f67a4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4198bc0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.786+0000 7f67a8c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4198d00 con 0x7f67a4104130 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39088/0 (socket says 172.21.15.90:39088) 2024-04-24T08:30:02.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 -- 172.21.15.90:0/1419579683 learned_addr learned my addr 172.21.15.90:0/1419579683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 -- 172.21.15.90:0/1419579683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4104130 msgr2=0x7f67a4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4104130 0x7f67a4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.787+0000 7f67a1e00700 1 -- 172.21.15.90:0/1419579683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940095f0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f67a1e00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67940092e0 tx=0x7f679400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679401d070 con 0x7f67a40ffbd0 2024-04-24T08:30:02.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f679400f460 con 0x7f67a40ffbd0 2024-04-24T08:30:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a419caf0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6794017630 con 0x7f67a40ffbd0 2024-04-24T08:30:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.788+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a419cfe0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.789+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a404f2a0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.790+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f679400f5d0 con 0x7f67a40ffbd0 2024-04-24T08:30:02.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.791+0000 7f679aa00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 0x7f67880712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:02.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.791+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6794090740 con 0x7f67a40ffbd0 2024-04-24T08:30:02.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.792+0000 7f67a1400700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 0x7f67880712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:02.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.792+0000 7f67a1400700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 0x7f67880712a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6790005fd0 tx=0x7f6790005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:02.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:02.797+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6794059290 con 0x7f67a40ffbd0 2024-04-24T08:30:03.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.054+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a419d460 con 0x7f678806edf0 2024-04-24T08:30:03.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.056+0000 7f679aa00700 1 -- 172.21.15.90:0/1419579683 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f67a419d460 con 0x7f678806edf0 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 msgr2=0x7f67880712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 0x7f67880712a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6790005fd0 tx=0x7f6790005d00 comp rx=0 tx=0).stop 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 msgr2=0x7f67a4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67940092e0 tx=0x7f679400be00 comp rx=0 tx=0).stop 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 shutdown_connections 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f678806edf0 0x7f67880712a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67a40ffbd0 0x7f67a4198030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 --2- 172.21.15.90:0/1419579683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4104130 0x7f67a4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 >> 172.21.15.90:0/1419579683 conn(0x7f67a4075130 msgr2=0x7f67a41095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:03.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 shutdown_connections 2024-04-24T08:30:03.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.060+0000 7f67a8c00700 1 -- 172.21.15.90:0/1419579683 wait complete. 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 -- 172.21.15.90:0/2273227339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c0687a0 msgr2=0x7ff56c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 --2- 172.21.15.90:0/2273227339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c0687a0 0x7ff56c068b70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009910 tx=0x7ff55c009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 -- 172.21.15.90:0/2273227339 shutdown_connections 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 --2- 172.21.15.90:0/2273227339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c069140 0x7ff56c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 --2- 172.21.15.90:0/2273227339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c0687a0 0x7ff56c068b70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 -- 172.21.15.90:0/2273227339 >> 172.21.15.90:0/2273227339 conn(0x7ff56c075130 msgr2=0x7ff56c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.222+0000 7ff572e00700 1 -- 172.21.15.90:0/2273227339 shutdown_connections 2024-04-24T08:30:03.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.223+0000 7ff572e00700 1 -- 172.21.15.90:0/2273227339 wait complete. 2024-04-24T08:30:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.224+0000 7ff572e00700 1 Processor -- start 2024-04-24T08:30:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.224+0000 7ff572e00700 1 -- start start 2024-04-24T08:30:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff572e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c0687a0 0x7ff56c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff572e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff572e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c100720 con 0x7ff56c069140 2024-04-24T08:30:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff572e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c100860 con 0x7ff56c0687a0 2024-04-24T08:30:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff56b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff56b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39108/0 (socket says 172.21.15.90:39108) 2024-04-24T08:30:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.225+0000 7ff56b400700 1 -- 172.21.15.90:0/2040335737 learned_addr learned my addr 172.21.15.90:0/2040335737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.226+0000 7ff56be00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c0687a0 0x7ff56c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.226+0000 7ff56b400700 1 -- 172.21.15.90:0/2040335737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c0687a0 msgr2=0x7ff56c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.226+0000 7ff56b400700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c0687a0 0x7ff56c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.226+0000 7ff56b400700 1 -- 172.21.15.90:0/2040335737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c0095f0 con 0x7ff56c069140 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.226+0000 7ff56b400700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c960 tx=0x7ff55800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.227+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff558007a70 con 0x7ff56c069140 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.227+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff558007bd0 con 0x7ff56c069140 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.227+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff558018630 con 0x7ff56c069140 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.227+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c1a20b0 con 0x7ff56c069140 2024-04-24T08:30:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.227+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c1a2410 con 0x7ff56c069140 2024-04-24T08:30:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.228+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c108770 con 0x7ff56c069140 2024-04-24T08:30:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.229+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7ff55800f450 con 0x7ff56c069140 2024-04-24T08:30:03.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.230+0000 7ff568c00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 0x7ff5500712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.231+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff55808ef70 con 0x7ff56c069140 2024-04-24T08:30:03.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.231+0000 7ff56be00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 0x7ff5500712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.232+0000 7ff56be00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 0x7ff5500712a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff55c006010 tx=0x7ff55c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:03.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.236+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff5580c3ff0 con 0x7ff56c069140 2024-04-24T08:30:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.515+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff56c061190 con 0x7ff55006edf0 2024-04-24T08:30:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.528+0000 7ff568c00700 1 -- 172.21.15.90:0/2040335737 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff56c061190 con 0x7ff55006edf0 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (3m) 40s ago 17m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (17m) 40s ago 17m 8232k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (15m) 15s ago 15m 8182k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (17m) 40s ago 17m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (15m) 15s ago 15m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (2m) 40s ago 16m 73.9M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (9m) 40s ago 9m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (9m) 40s ago 9m 16.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:30:03.530 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (9m) 15s ago 9m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (9m) 15s ago 9m 16.5M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (6m) 40s ago 18m 480M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (4m) 15s ago 15m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (58s) 40s ago 19m 28.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (22s) 15s ago 15m 19.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (4m) 40s ago 17m 7889k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (4m) 15s ago 15m 8220k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (14m) 40s ago 14m 217M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (14m) 40s ago 14m 225M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (13m) 40s ago 13m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (12m) 15s ago 12m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:30:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (12m) 15s ago 12m 220M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:30:03.532 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (11m) 15s ago 11m 202M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:30:03.532 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (3m) 40s ago 16m 62.3M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 msgr2=0x7ff5500712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 0x7ff5500712a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff55c006010 tx=0x7ff55c01a040 comp rx=0 tx=0).stop 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 msgr2=0x7ff56c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c960 tx=0x7ff55800cc70 comp rx=0 tx=0).stop 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 shutdown_connections 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.531+0000 7ff572e00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff55006edf0 0x7ff5500712a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.532+0000 7ff572e00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff56c0687a0 0x7ff56c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.532+0000 7ff572e00700 1 --2- 172.21.15.90:0/2040335737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff56c069140 0x7ff56c1001e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.532+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 >> 172.21.15.90:0/2040335737 conn(0x7ff56c075130 msgr2=0x7ff56c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.532+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 shutdown_connections 2024-04-24T08:30:03.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.532+0000 7ff572e00700 1 -- 172.21.15.90:0/2040335737 wait complete. 2024-04-24T08:30:03.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 -- 172.21.15.90:0/3236832652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 msgr2=0x7f29d4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 --2- 172.21.15.90:0/3236832652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4068b70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009910 tx=0x7f29c4009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 -- 172.21.15.90:0/3236832652 shutdown_connections 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 --2- 172.21.15.90:0/3236832652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 --2- 172.21.15.90:0/3236832652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4068b70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 -- 172.21.15.90:0/3236832652 >> 172.21.15.90:0/3236832652 conn(0x7f29d4075130 msgr2=0x7f29d4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 -- 172.21.15.90:0/3236832652 shutdown_connections 2024-04-24T08:30:03.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.683+0000 7f29d9200700 1 -- 172.21.15.90:0/3236832652 wait complete. 2024-04-24T08:30:03.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.684+0000 7f29d9200700 1 Processor -- start 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.684+0000 7f29d9200700 1 -- start start 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4196b00 con 0x7f29d40687a0 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d9200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d419a890 con 0x7f29d4069140 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39130/0 (socket says 172.21.15.90:39130) 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 -- 172.21.15.90:0/3900214085 learned_addr learned my addr 172.21.15.90:0/3900214085 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1400700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 -- 172.21.15.90:0/3900214085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 msgr2=0x7f29d4196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1e00700 1 -- 172.21.15.90:0/3900214085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40095f0 con 0x7f29d40687a0 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.685+0000 7f29d1400700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:30:03.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.686+0000 7f29d1e00700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29c40092e0 tx=0x7f29c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:03.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.686+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29c401d070 con 0x7f29d40687a0 2024-04-24T08:30:03.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.686+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29c4005e00 con 0x7f29d40687a0 2024-04-24T08:30:03.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.686+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29c4017400 con 0x7f29d40687a0 2024-04-24T08:30:03.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.686+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d419ab10 con 0x7f29d40687a0 2024-04-24T08:30:03.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.687+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d419b000 con 0x7f29d40687a0 2024-04-24T08:30:03.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.688+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d4108790 con 0x7f29d40687a0 2024-04-24T08:30:03.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.689+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f29c400f460 con 0x7f29d40687a0 2024-04-24T08:30:03.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.690+0000 7f29caa00700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 0x7f29b80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:03.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.690+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f29c4090550 con 0x7f29d40687a0 2024-04-24T08:30:03.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.690+0000 7f29d1400700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 0x7f29b80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:03.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.691+0000 7f29d1400700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 0x7f29b80712a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005fd0 tx=0x7f29c0005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:03.698+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f29c4059120 con 0x7f29d40687a0 2024-04-24T08:30:03.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:03.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:03.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:03.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-24T08:30:03.776 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:03.777 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:03.777 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-24T08:30:03.777 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:03.777 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:03 smithi090 ceph-mon[85282]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:04.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:04.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:03 smithi149 ceph-mon[66490]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.048+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29d4108170 con 0x7f29d40687a0 2024-04-24T08:30:04.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.049+0000 7f29caa00700 1 -- 172.21.15.90:0/3900214085 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f29c405c7e0 con 0x7f29d40687a0 2024-04-24T08:30:04.050 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:30:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:30:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:30:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:30:04.051 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.052+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 msgr2=0x7f29b80712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.052+0000 7f29d9200700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 0x7f29b80712a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005fd0 tx=0x7f29c0005ee0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.052+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 msgr2=0x7f29d4195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.052+0000 7f29d9200700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29c40092e0 tx=0x7f29c400be20 comp rx=0 tx=0).stop 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 shutdown_connections 2024-04-24T08:30:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f29b806edf0 0x7f29b80712a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40687a0 0x7f29d4195ee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 --2- 172.21.15.90:0/3900214085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29d4069140 0x7f29d4196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 >> 172.21.15.90:0/3900214085 conn(0x7f29d4075130 msgr2=0x7f29d40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 shutdown_connections 2024-04-24T08:30:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.053+0000 7f29d9200700 1 -- 172.21.15.90:0/3900214085 wait complete. 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 -- 172.21.15.90:0/2852534282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 msgr2=0x7f345810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 --2- 172.21.15.90:0/2852534282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345810ca30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3448009910 tx=0x7f3448009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 -- 172.21.15.90:0/2852534282 shutdown_connections 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 --2- 172.21.15.90:0/2852534282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 0x7f3458106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 --2- 172.21.15.90:0/2852534282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345810ca30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 -- 172.21.15.90:0/2852534282 >> 172.21.15.90:0/2852534282 conn(0x7f3458075e90 msgr2=0x7f34580782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 -- 172.21.15.90:0/2852534282 shutdown_connections 2024-04-24T08:30:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.207+0000 7f3457e00700 1 -- 172.21.15.90:0/2852534282 wait complete. 2024-04-24T08:30:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.208+0000 7f3457e00700 1 Processor -- start 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f3457e00700 1 -- start start 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f3457e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 0x7f345819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f3457e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f3457e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819cea0 con 0x7f345810c660 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f3457e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581a0c30 con 0x7f3458106b80 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f344fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f344fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39156/0 (socket says 172.21.15.90:39156) 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.209+0000 7f344fe00700 1 -- 172.21.15.90:0/1480380398 learned_addr learned my addr 172.21.15.90:0/1480380398 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f3456a00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 0x7f345819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344fe00700 1 -- 172.21.15.90:0/1480380398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 msgr2=0x7f345819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344fe00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 0x7f345819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344fe00700 1 -- 172.21.15.90:0/1480380398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34480095f0 con 0x7f345810c660 2024-04-24T08:30:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344fe00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f344000d8d0 tx=0x7f344000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3440009940 con 0x7f345810c660 2024-04-24T08:30:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3440010460 con 0x7f345810c660 2024-04-24T08:30:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.210+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f344000f5f0 con 0x7f345810c660 2024-04-24T08:30:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.211+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34581a0f10 con 0x7f345810c660 2024-04-24T08:30:04.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.211+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581a1460 con 0x7f345810c660 2024-04-24T08:30:04.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.212+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345810eb40 con 0x7f345810c660 2024-04-24T08:30:04.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.213+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f34400105d0 con 0x7f345810c660 2024-04-24T08:30:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.214+0000 7f344f400700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 0x7f343c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.214+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f344008ef90 con 0x7f345810c660 2024-04-24T08:30:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.214+0000 7f3456a00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 0x7f343c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.215+0000 7f3456a00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 0x7f343c0712a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f344801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:04.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.219+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3440057c00 con 0x7f345810c660 2024-04-24T08:30:04.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.496+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3458062380 con 0x7f345810c660 2024-04-24T08:30:04.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.497+0000 7f344f400700 1 -- 172.21.15.90:0/1480380398 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f344005b2c0 con 0x7f345810c660 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:30:04.500 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:30:04.501 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:30:04.502 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:04.503 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:04.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 msgr2=0x7f343c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 0x7f343c0712a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f344801a040 comp rx=0 tx=0).stop 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 msgr2=0x7f345819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f344000d8d0 tx=0x7f344000dc90 comp rx=0 tx=0).stop 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 shutdown_connections 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f343c06edf0 0x7f343c0712a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3458106b80 0x7f345819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 --2- 172.21.15.90:0/1480380398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f345810c660 0x7f345819c7c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.500+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 >> 172.21.15.90:0/1480380398 conn(0x7f3458075e90 msgr2=0x7f3458077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:04.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.501+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 shutdown_connections 2024-04-24T08:30:04.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.501+0000 7f3457e00700 1 -- 172.21.15.90:0/1480380398 wait complete. 2024-04-24T08:30:04.506 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:30:04.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.658+0000 7fa139200700 1 -- 172.21.15.90:0/3624026761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134106320 msgr2=0x7fa1341066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.658+0000 7fa139200700 1 --2- 172.21.15.90:0/3624026761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134106320 0x7fa1341066f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa124009910 tx=0x7fa124009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:04.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 -- 172.21.15.90:0/3624026761 shutdown_connections 2024-04-24T08:30:04.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 --2- 172.21.15.90:0/3624026761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134068540 0x7fa1340689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 --2- 172.21.15.90:0/3624026761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134106320 0x7fa1341066f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 -- 172.21.15.90:0/3624026761 >> 172.21.15.90:0/3624026761 conn(0x7fa134075130 msgr2=0x7fa134075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:04.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 -- 172.21.15.90:0/3624026761 shutdown_connections 2024-04-24T08:30:04.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.659+0000 7fa139200700 1 -- 172.21.15.90:0/3624026761 wait complete. 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 Processor -- start 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 -- start start 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134106320 0x7fa1341001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341007b0 con 0x7fa134068540 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa139200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa134100920 con 0x7fa134106320 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa131e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa131e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39166/0 (socket says 172.21.15.90:39166) 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.660+0000 7fa131e00700 1 -- 172.21.15.90:0/3935853182 learned_addr learned my addr 172.21.15.90:0/3935853182 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.661+0000 7fa131e00700 1 -- 172.21.15.90:0/3935853182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134106320 msgr2=0x7fa1341001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.661+0000 7fa129400700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134106320 0x7fa1341001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.661+0000 7fa131e00700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134106320 0x7fa1341001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.661+0000 7fa131e00700 1 -- 172.21.15.90:0/3935853182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240095f0 con 0x7fa134068540 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.661+0000 7fa131e00700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b5c0 tx=0x7fa12400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.662+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa12401d070 con 0x7fa134068540 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.662+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa12400f460 con 0x7fa134068540 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.662+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1341a2040 con 0x7fa134068540 2024-04-24T08:30:04.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.662+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa124017610 con 0x7fa134068540 2024-04-24T08:30:04.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.662+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1341a23f0 con 0x7fa134068540 2024-04-24T08:30:04.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.663+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134108790 con 0x7fa134068540 2024-04-24T08:30:04.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.664+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fa12400f5d0 con 0x7fa134068540 2024-04-24T08:30:04.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.665+0000 7fa12b400700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 0x7fa1180712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:04.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.665+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa124090690 con 0x7fa134068540 2024-04-24T08:30:04.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.665+0000 7fa129400700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 0x7fa1180712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:04.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.666+0000 7fa129400700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 0x7fa1180712a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa134101490 tx=0x7fa11c005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:04.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.670+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa124059280 con 0x7fa134068540 2024-04-24T08:30:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:04 smithi090 ceph-mon[85282]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:04 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3900214085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:04.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:04 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1480380398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:30:04.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.937+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa134061190 con 0x7fa11806edf0 2024-04-24T08:30:04.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.940+0000 7fa12b400700 1 -- 172.21.15.90:0/3935853182 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fa134061190 con 0x7fa11806edf0 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:30:04.942 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:30:04.943 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "4/23 daemons upgraded", 2024-04-24T08:30:04.943 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T08:30:04.943 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:30:04.943 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.943+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 msgr2=0x7fa1180712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.943+0000 7fa139200700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 0x7fa1180712a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa134101490 tx=0x7fa11c005cf0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.943+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 msgr2=0x7fa1340ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.943+0000 7fa139200700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b5c0 tx=0x7fa12400be00 comp rx=0 tx=0).stop 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 shutdown_connections 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa11806edf0 0x7fa1180712a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa134068540 0x7fa1340ffca0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 --2- 172.21.15.90:0/3935853182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa134106320 0x7fa1341001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:04.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 >> 172.21.15.90:0/3935853182 conn(0x7fa134075130 msgr2=0x7fa1340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:04.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 shutdown_connections 2024-04-24T08:30:04.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:04.944+0000 7fa139200700 1 -- 172.21.15.90:0/3935853182 wait complete. 2024-04-24T08:30:05.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:04 smithi149 ceph-mon[66490]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:05.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:04 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3900214085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:05.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:04 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1480380398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 -- 172.21.15.90:0/833589369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 msgr2=0x7ffac8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 --2- 172.21.15.90:0/833589369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac8068b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009910 tx=0x7ffabc009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 -- 172.21.15.90:0/833589369 shutdown_connections 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 --2- 172.21.15.90:0/833589369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 --2- 172.21.15.90:0/833589369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac8068b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 -- 172.21.15.90:0/833589369 >> 172.21.15.90:0/833589369 conn(0x7ffac8075130 msgr2=0x7ffac8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 -- 172.21.15.90:0/833589369 shutdown_connections 2024-04-24T08:30:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.098+0000 7ffad0600700 1 -- 172.21.15.90:0/833589369 wait complete. 2024-04-24T08:30:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.099+0000 7ffad0600700 1 Processor -- start 2024-04-24T08:30:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffad0600700 1 -- start start 2024-04-24T08:30:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffad0600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffad0600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac81001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffad0600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8100720 con 0x7ffac80687a0 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffad0600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8100860 con 0x7ffac8069140 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffacdc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffacdc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39174/0 (socket says 172.21.15.90:39174) 2024-04-24T08:30:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.100+0000 7ffacdc00700 1 -- 172.21.15.90:0/446606835 learned_addr learned my addr 172.21.15.90:0/446606835 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacd200700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac81001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacdc00700 1 -- 172.21.15.90:0/446606835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 msgr2=0x7ffac81001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacdc00700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac81001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacdc00700 1 -- 172.21.15.90:0/446606835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0095f0 con 0x7ffac80687a0 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacd200700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac81001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.101+0000 7ffacdc00700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b5c0 tx=0x7ffabc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.102+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffabc01d070 con 0x7ffac80687a0 2024-04-24T08:30:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.102+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffabc00f460 con 0x7ffac80687a0 2024-04-24T08:30:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.102+0000 7ffad0600700 1 -- 172.21.15.90:0/446606835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac8100ae0 con 0x7ffac80687a0 2024-04-24T08:30:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.102+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffabc017610 con 0x7ffac80687a0 2024-04-24T08:30:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.102+0000 7ffad0600700 1 -- 172.21.15.90:0/446606835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81a2460 con 0x7ffac80687a0 2024-04-24T08:30:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.103+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac804f2a0 con 0x7ffac80687a0 2024-04-24T08:30:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.105+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7ffabc00f5d0 con 0x7ffac80687a0 2024-04-24T08:30:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.106+0000 7ffac6000700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 0x7ffab00712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.107+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffabc0906f0 con 0x7ffac80687a0 2024-04-24T08:30:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.107+0000 7ffacd200700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 0x7ffab00712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.108+0000 7ffacd200700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 0x7ffab00712a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffac8101490 tx=0x7ffab4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:05.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.112+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffabc059280 con 0x7ffac80687a0 2024-04-24T08:30:05.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.436+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffac81011d0 con 0x7ffac80687a0 2024-04-24T08:30:05.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.437+0000 7ffac6000700 1 -- 172.21.15.90:0/446606835 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ffabc05c940 con 0x7ffac80687a0 2024-04-24T08:30:05.438 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:05.439 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:30:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 msgr2=0x7ffab00712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 0x7ffab00712a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffac8101490 tx=0x7ffab4008040 comp rx=0 tx=0).stop 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 msgr2=0x7ffac80ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b5c0 tx=0x7ffabc00be00 comp rx=0 tx=0).stop 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 shutdown_connections 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ffab006edf0 0x7ffab00712a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffac80687a0 0x7ffac80ffca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 --2- 172.21.15.90:0/446606835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac8069140 0x7ffac81001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.440+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 >> 172.21.15.90:0/446606835 conn(0x7ffac8075130 msgr2=0x7ffac80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.441+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 shutdown_connections 2024-04-24T08:30:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:05.441+0000 7ffabb400700 1 -- 172.21.15.90:0/446606835 wait complete. 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:06.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.0 on smithi090 2024-04-24T08:30:06.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:06.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:05 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/446606835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:30:06.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.0 on smithi090 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:06.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:05 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/446606835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:30:08.367 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:08 smithi090 ceph-mon[85282]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:08.367 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:08.367 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:08.367 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:30:08.367 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:08 smithi149 ceph-mon[66490]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:30:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:09.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:09 smithi149 ceph-mon[66490]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T08:30:09.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:09 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.1 on smithi090 2024-04-24T08:30:09.392 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:09 smithi090 ceph-mon[85282]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T08:30:09.392 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:09 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.1 on smithi090 2024-04-24T08:30:10.144 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:10 smithi149 ceph-mon[66490]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:10.144 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:10 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:10.144 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:10 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:10.144 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:10 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:30:10.144 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:10 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:10 smithi090 ceph-mon[85282]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:10 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:10 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:10 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:30:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:10 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:11.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:11 smithi149 ceph-mon[66490]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T08:30:11.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:11 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.2 on smithi090 2024-04-24T08:30:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:11 smithi090 ceph-mon[85282]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T08:30:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:11 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.2 on smithi090 2024-04-24T08:30:12.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:12 smithi090 ceph-mon[85282]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:12 smithi149 ceph-mon[66490]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:13.370 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:13.370 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:13.370 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:13.370 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:14 smithi090 ceph-mon[85282]: Reconfiguring mds.cephfs.smithi090.qbyvpv (monmap changed)... 2024-04-24T08:30:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:14 smithi090 ceph-mon[85282]: Reconfiguring daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:30:14.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:14 smithi090 ceph-mon[85282]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:14 smithi149 ceph-mon[66490]: Reconfiguring mds.cephfs.smithi090.qbyvpv (monmap changed)... 2024-04-24T08:30:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:14 smithi149 ceph-mon[66490]: Reconfiguring daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:30:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:14 smithi149 ceph-mon[66490]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: Reconfiguring mds.cephfs.smithi090.lumztb (monmap changed)... 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: Reconfiguring daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:30:15.650 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:15.651 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:15.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:15.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: Reconfiguring mds.cephfs.smithi090.lumztb (monmap changed)... 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: Reconfiguring daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:15.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:18.170 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:18.170 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: Reconfiguring ceph-exporter.smithi149 (monmap changed)... 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: Unable to update caps for client.ceph-exporter.smithi149 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi149"}]: dispatch 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: Reconfiguring daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:30:18.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:17 smithi149 ceph-mon[66490]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: Reconfiguring ceph-exporter.smithi149 (monmap changed)... 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:30:18.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-24T08:30:18.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: Unable to update caps for client.ceph-exporter.smithi149 2024-04-24T08:30:18.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi149"}]: dispatch 2024-04-24T08:30:18.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:18.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: Reconfiguring daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:30:18.295 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:17 smithi090 ceph-mon[85282]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:20.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:20 smithi149 ceph-mon[66490]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:20.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:20.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:20.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:20.475 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:20 smithi090 ceph-mon[85282]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:20.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:20.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:20.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:20.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:21 smithi090 ceph-mon[85282]: Reconfiguring crash.smithi149 (monmap changed)... 2024-04-24T08:30:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:21 smithi090 ceph-mon[85282]: Reconfiguring daemon crash.smithi149 on smithi149 2024-04-24T08:30:21.504 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:21 smithi149 ceph-mon[66490]: Reconfiguring crash.smithi149 (monmap changed)... 2024-04-24T08:30:21.505 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:21 smithi149 ceph-mon[66490]: Reconfiguring daemon crash.smithi149 on smithi149 2024-04-24T08:30:22.476 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:22.476 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:22.476 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:22.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:30:22.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:30:22.477 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.vavftl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:30:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:23 smithi149 ceph-mon[66490]: Reconfiguring mgr.smithi149.vavftl (monmap changed)... 2024-04-24T08:30:23.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:23 smithi149 ceph-mon[66490]: Reconfiguring daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:30:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:23 smithi090 ceph-mon[85282]: Reconfiguring mgr.smithi149.vavftl (monmap changed)... 2024-04-24T08:30:23.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:23 smithi090 ceph-mon[85282]: Reconfiguring daemon mgr.smithi149.vavftl on smithi149 2024-04-24T08:30:24.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:24 smithi149 ceph-mon[66490]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:24.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:24 smithi090 ceph-mon[85282]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:25.401 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: Reconfiguring mon.smithi149 (monmap changed)... 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: Reconfiguring daemon mon.smithi149 on smithi149 2024-04-24T08:30:25.402 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:25 smithi149 ceph-mon[66490]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:25.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: Reconfiguring mon.smithi149 (monmap changed)... 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: Reconfiguring daemon mon.smithi149 on smithi149 2024-04-24T08:30:25.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:25 smithi090 ceph-mon[85282]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:27.845 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:27.845 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:27.846 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T08:30:27.846 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:30:27.846 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:27.846 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.3 on smithi149 2024-04-24T08:30:27.846 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:27 smithi149 ceph-mon[66490]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:28.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.3 on smithi149 2024-04-24T08:30:28.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:27 smithi090 ceph-mon[85282]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:30 smithi149 ceph-mon[66490]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:30:30.454 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:30 smithi090 ceph-mon[85282]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:30.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:30.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:30.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:30:30.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:31 smithi149 ceph-mon[66490]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T08:30:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:31 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.4 on smithi149 2024-04-24T08:30:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:31 smithi090 ceph-mon[85282]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T08:30:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:31 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.4 on smithi149 2024-04-24T08:30:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:32 smithi149 ceph-mon[66490]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:32 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:32 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:32.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:32 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:30:32.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:32 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:32 smithi090 ceph-mon[85282]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:32 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:32 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:32 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:30:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:32 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:33.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:33 smithi090 ceph-mon[85282]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T08:30:33.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:33 smithi090 ceph-mon[85282]: Reconfiguring daemon osd.5 on smithi149 2024-04-24T08:30:33.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:33 smithi149 ceph-mon[66490]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T08:30:33.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:33 smithi149 ceph-mon[66490]: Reconfiguring daemon osd.5 on smithi149 2024-04-24T08:30:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:34 smithi149 ceph-mon[66490]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:34 smithi090 ceph-mon[85282]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: Reconfiguring mds.cephfs.smithi149.eazcmc (monmap changed)... 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:35.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: Reconfiguring daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:30:35.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:35 smithi149 ceph-mon[66490]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:35.614 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: Reconfiguring mds.cephfs.smithi149.eazcmc (monmap changed)... 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: Reconfiguring daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:30:35.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:35 smithi090 ceph-mon[85282]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 -- 172.21.15.90:0/4241188661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 msgr2=0x7fb00c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 --2- 172.21.15.90:0/4241188661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c10ccc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faffc009910 tx=0x7faffc009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 -- 172.21.15.90:0/4241188661 shutdown_connections 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 --2- 172.21.15.90:0/4241188661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 0x7fb00c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 --2- 172.21.15.90:0/4241188661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c10ccc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 -- 172.21.15.90:0/4241188661 >> 172.21.15.90:0/4241188661 conn(0x7fb00c076020 msgr2=0x7fb00c078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 -- 172.21.15.90:0/4241188661 shutdown_connections 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.612+0000 7fb011600700 1 -- 172.21.15.90:0/4241188661 wait complete. 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.613+0000 7fb011600700 1 Processor -- start 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb011600700 1 -- start start 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb011600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 0x7fb00c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb011600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb011600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c19cff0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb011600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a0d80 con 0x7fb00c106b80 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb00a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb00a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54636/0 (socket says 172.21.15.90:54636) 2024-04-24T08:30:35.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.614+0000 7fb00a000700 1 -- 172.21.15.90:0/3569988123 learned_addr learned my addr 172.21.15.90:0/3569988123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.615+0000 7fb00aa00700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 0x7fb00c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.615+0000 7fb00a000700 1 -- 172.21.15.90:0/3569988123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 msgr2=0x7fb00c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.615+0000 7fb00a000700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 0x7fb00c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.615+0000 7fb00a000700 1 -- 172.21.15.90:0/3569988123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0095f0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.615+0000 7fb00a000700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faff800d8d0 tx=0x7faff800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.616+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff8009940 con 0x7fb00c10c8f0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.616+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faff8010460 con 0x7fb00c10c8f0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.616+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff800f5f0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.616+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c1a1060 con 0x7fb00c10c8f0 2024-04-24T08:30:35.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.616+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1a15b0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.617+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c04f2a0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.619+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7faff80105d0 con 0x7fb00c10c8f0 2024-04-24T08:30:35.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.619+0000 7fb003400700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 0x7faff00712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:35.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.620+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7faff808f060 con 0x7fb00c10c8f0 2024-04-24T08:30:35.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.620+0000 7fb00aa00700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 0x7faff00712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:35.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.621+0000 7fb00aa00700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 0x7faff00712a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faffc000c00 tx=0x7faffc005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:35.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.628+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faff8057c30 con 0x7fb00c10c8f0 2024-04-24T08:30:35.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.879+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb00c1a1a00 con 0x7faff006edf0 2024-04-24T08:30:35.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.882+0000 7fb003400700 1 -- 172.21.15.90:0/3569988123 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb00c1a1a00 con 0x7faff006edf0 2024-04-24T08:30:35.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 msgr2=0x7faff00712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 0x7faff00712a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faffc000c00 tx=0x7faffc005fd0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 msgr2=0x7fb00c19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faff800d8d0 tx=0x7faff800dc90 comp rx=0 tx=0).stop 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 shutdown_connections 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7faff006edf0 0x7faff00712a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb00c106b80 0x7fb00c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 --2- 172.21.15.90:0/3569988123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb00c10c8f0 0x7fb00c19c910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 >> 172.21.15.90:0/3569988123 conn(0x7fb00c076020 msgr2=0x7fb00c077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 shutdown_connections 2024-04-24T08:30:35.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:35.885+0000 7fb011600700 1 -- 172.21.15.90:0/3569988123 wait complete. 2024-04-24T08:30:35.904 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 -- 172.21.15.90:0/2073191621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48108470 msgr2=0x7fbc48108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2073191621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48108470 0x7fbc48108840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009910 tx=0x7fbc3c009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 -- 172.21.15.90:0/2073191621 shutdown_connections 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2073191621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48102470 0x7fbc481028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2073191621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48108470 0x7fbc48108840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 -- 172.21.15.90:0/2073191621 >> 172.21.15.90:0/2073191621 conn(0x7fbc480fdf90 msgr2=0x7fbc481003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.045+0000 7fbc4f400700 1 -- 172.21.15.90:0/2073191621 shutdown_connections 2024-04-24T08:30:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.046+0000 7fbc4f400700 1 -- 172.21.15.90:0/2073191621 wait complete. 2024-04-24T08:30:36.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 Processor -- start 2024-04-24T08:30:36.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 -- start start 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48108470 0x7fbc48198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48198e50 con 0x7fbc48102470 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.047+0000 7fbc4f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819cbe0 con 0x7fbc48108470 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54652/0 (socket says 172.21.15.90:54652) 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 -- 172.21.15.90:0/2532470997 learned_addr learned my addr 172.21.15.90:0/2532470997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc47e00700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48108470 0x7fbc48198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 -- 172.21.15.90:0/2532470997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48108470 msgr2=0x7fbc48198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48108470 0x7fbc48198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 -- 172.21.15.90:0/2532470997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0095f0 con 0x7fbc48102470 2024-04-24T08:30:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.048+0000 7fbc4ca00700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.049+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c01d070 con 0x7fbc48102470 2024-04-24T08:30:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.049+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc3c005dc0 con 0x7fbc48102470 2024-04-24T08:30:36.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.049+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c017400 con 0x7fbc48102470 2024-04-24T08:30:36.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.049+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4819ce60 con 0x7fbc48102470 2024-04-24T08:30:36.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.049+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4819d350 con 0x7fbc48102470 2024-04-24T08:30:36.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.050+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4810a9e0 con 0x7fbc48102470 2024-04-24T08:30:36.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.051+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fbc3c003a60 con 0x7fbc48102470 2024-04-24T08:30:36.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.052+0000 7fbc45600700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 0x7fbc300712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.052+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbc3c090420 con 0x7fbc48102470 2024-04-24T08:30:36.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.052+0000 7fbc47e00700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 0x7fbc300712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.053+0000 7fbc47e00700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 0x7fbc300712b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc48199850 tx=0x7fbc3400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.057+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc3c059090 con 0x7fbc48102470 2024-04-24T08:30:36.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.306+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc4819d5c0 con 0x7fbc3006ee00 2024-04-24T08:30:36.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.309+0000 7fbc45600700 1 -- 172.21.15.90:0/2532470997 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbc4819d5c0 con 0x7fbc3006ee00 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.311+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 msgr2=0x7fbc300712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.311+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 0x7fbc300712b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc48199850 tx=0x7fbc3400b410 comp rx=0 tx=0).stop 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.311+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 msgr2=0x7fbc48198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.311+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c00bd70 comp rx=0 tx=0).stop 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 shutdown_connections 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbc3006ee00 0x7fbc300712b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc48102470 0x7fbc48198230 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 --2- 172.21.15.90:0/2532470997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48108470 0x7fbc48198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 >> 172.21.15.90:0/2532470997 conn(0x7fbc480fdf90 msgr2=0x7fbc480ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:36.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 shutdown_connections 2024-04-24T08:30:36.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.312+0000 7fbc4f400700 1 -- 172.21.15.90:0/2532470997 wait complete. 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.467+0000 7f68a0200700 1 -- 172.21.15.90:0/4077686416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 msgr2=0x7f68981028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.467+0000 7f68a0200700 1 --2- 172.21.15.90:0/4077686416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981028e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6884009910 tx=0x7f6884009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.467+0000 7f68a0200700 1 -- 172.21.15.90:0/4077686416 shutdown_connections 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.468+0000 7f68a0200700 1 --2- 172.21.15.90:0/4077686416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981028e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.468+0000 7f68a0200700 1 --2- 172.21.15.90:0/4077686416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 0x7f6898108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.468+0000 7f68a0200700 1 -- 172.21.15.90:0/4077686416 >> 172.21.15.90:0/4077686416 conn(0x7f68980fdf90 msgr2=0x7f68981003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.468+0000 7f68a0200700 1 -- 172.21.15.90:0/4077686416 shutdown_connections 2024-04-24T08:30:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.468+0000 7f68a0200700 1 -- 172.21.15.90:0/4077686416 wait complete. 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 Processor -- start 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 -- start start 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 0x7f6898198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898198ef0 con 0x7f6898102470 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.469+0000 7f68a0200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819cc80 con 0x7f6898108470 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54680/0 (socket says 172.21.15.90:54680) 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 -- 172.21.15.90:0/176748951 learned_addr learned my addr 172.21.15.90:0/176748951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689ce00700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 0x7f6898198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 -- 172.21.15.90:0/176748951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 msgr2=0x7f6898198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 0x7f6898198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.470+0000 7f689d800700 1 -- 172.21.15.90:0/176748951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840095f0 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f689d800700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d900 tx=0x7f688c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c004a00 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f688c0054e0 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c009d00 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689819cf60 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.471+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689819d480 con 0x7f6898102470 2024-04-24T08:30:36.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.472+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689810a970 con 0x7f6898102470 2024-04-24T08:30:36.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.473+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f688c005020 con 0x7f6898102470 2024-04-24T08:30:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.474+0000 7f6896000700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 0x7f68800712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.474+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f688c08efa0 con 0x7f6898102470 2024-04-24T08:30:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.474+0000 7f689ce00700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 0x7f68800712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.475+0000 7f689ce00700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 0x7f68800712a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f688400b5c0 tx=0x7f688401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.479+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f688c057b70 con 0x7f6898102470 2024-04-24T08:30:36.554 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:36 smithi149 ceph-mon[66490]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:36.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.708+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6898199710 con 0x7f688006edf0 2024-04-24T08:30:36.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:36 smithi090 ceph-mon[85282]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:36.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.724+0000 7f6896000700 1 -- 172.21.15.90:0/176748951 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6898199710 con 0x7f688006edf0 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (3m) 73s ago 17m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (17m) 73s ago 17m 8232k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (16m) 49s ago 16m 8182k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (17m) 73s ago 17m 6665k - 18.2.0 dc2bc1663786 a1d4249a1401 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (16m) 49s ago 16m 6665k - 18.2.0 dc2bc1663786 8b28d20fb26d 2024-04-24T08:30:36.726 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (2m) 73s ago 17m 73.9M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (10m) 73s ago 10m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (10m) 73s ago 10m 16.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (10m) 49s ago 10m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (10m) 49s ago 10m 16.5M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (6m) 73s ago 19m 480M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (5m) 49s ago 16m 449M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (91s) 73s ago 19m 28.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (55s) 49s ago 16m 19.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (4m) 73s ago 17m 7889k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (4m) 49s ago 16m 8220k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (15m) 73s ago 15m 217M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (14m) 73s ago 14m 225M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (13m) 73s ago 13m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (13m) 49s ago 13m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:30:36.727 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (12m) 49s ago 12m 220M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:30:36.728 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (12m) 49s ago 12m 202M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:30:36.728 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (4m) 73s ago 16m 62.3M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 msgr2=0x7f68800712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 0x7f68800712a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f688400b5c0 tx=0x7f688401a040 comp rx=0 tx=0).stop 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 msgr2=0x7f68981982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d900 tx=0x7f688c00dc10 comp rx=0 tx=0).stop 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 shutdown_connections 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f688006edf0 0x7f68800712a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6898102470 0x7f68981982d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 --2- 172.21.15.90:0/176748951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6898108470 0x7f6898198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 >> 172.21.15.90:0/176748951 conn(0x7f68980fdf90 msgr2=0x7f68980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:36.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 shutdown_connections 2024-04-24T08:30:36.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.728+0000 7f68a0200700 1 -- 172.21.15.90:0/176748951 wait complete. 2024-04-24T08:30:36.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 -- 172.21.15.90:0/1351067614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 msgr2=0x7fe55810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 --2- 172.21.15.90:0/1351067614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe55810c580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe540009910 tx=0x7fe540009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:36.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 -- 172.21.15.90:0/1351067614 shutdown_connections 2024-04-24T08:30:36.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 --2- 172.21.15.90:0/1351067614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe55810c580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 --2- 172.21.15.90:0/1351067614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 0x7fe5580734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 -- 172.21.15.90:0/1351067614 >> 172.21.15.90:0/1351067614 conn(0x7fe5580fbda0 msgr2=0x7fe5580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:36.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 -- 172.21.15.90:0/1351067614 shutdown_connections 2024-04-24T08:30:36.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.885+0000 7fe55d800700 1 -- 172.21.15.90:0/1351067614 wait complete. 2024-04-24T08:30:36.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.886+0000 7fe55d800700 1 Processor -- start 2024-04-24T08:30:36.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.886+0000 7fe55d800700 1 -- start start 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe55d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 0x7fe558197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe55d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe55d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55819c970 con 0x7fe558073a00 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe55d800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55819cae0 con 0x7fe5580730f0 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe54fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe54fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54706/0 (socket says 172.21.15.90:54706) 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe54fe00700 1 -- 172.21.15.90:0/2732746263 learned_addr learned my addr 172.21.15.90:0/2732746263 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.887+0000 7fe556a00700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 0x7fe558197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54fe00700 1 -- 172.21.15.90:0/2732746263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 msgr2=0x7fe558197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54fe00700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 0x7fe558197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:36.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54fe00700 1 -- 172.21.15.90:0/2732746263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400095f0 con 0x7fe558073a00 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54fe00700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b5c0 tx=0x7fe54000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe54001d070 con 0x7fe558073a00 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe54000f460 con 0x7fe558073a00 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55819cdc0 con 0x7fe558073a00 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.888+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe540017630 con 0x7fe558073a00 2024-04-24T08:30:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.889+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55819d310 con 0x7fe558073a00 2024-04-24T08:30:36.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.890+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558109c80 con 0x7fe558073a00 2024-04-24T08:30:36.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.891+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fe54000f5d0 con 0x7fe558073a00 2024-04-24T08:30:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.891+0000 7fe54f400700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 0x7fe53c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.892+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe540090610 con 0x7fe558073a00 2024-04-24T08:30:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.892+0000 7fe556a00700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 0x7fe53c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.893+0000 7fe556a00700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 0x7fe53c0712a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe548005fd0 tx=0x7fe54800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:36.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:36.896+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe540059200 con 0x7fe558073a00 2024-04-24T08:30:37.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.213+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe558199320 con 0x7fe558073a00 2024-04-24T08:30:37.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.214+0000 7fe54f400700 1 -- 172.21.15.90:0/2732746263 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fe54005c8c0 con 0x7fe558073a00 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:30:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:30:37.217 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-04-24T08:30:37.217 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:30:37.217 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:30:37.217 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 msgr2=0x7fe53c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 0x7fe53c0712a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe548005fd0 tx=0x7fe54800a560 comp rx=0 tx=0).stop 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 msgr2=0x7fe558198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b5c0 tx=0x7fe54000be00 comp rx=0 tx=0).stop 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 shutdown_connections 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe53c06edf0 0x7fe53c0712a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5580730f0 0x7fe558197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 --2- 172.21.15.90:0/2732746263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe558073a00 0x7fe558198630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.217+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 >> 172.21.15.90:0/2732746263 conn(0x7fe5580fbda0 msgr2=0x7fe558106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.218+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 shutdown_connections 2024-04-24T08:30:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.218+0000 7fe55d800700 1 -- 172.21.15.90:0/2732746263 wait complete. 2024-04-24T08:30:37.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 -- 172.21.15.90:0/3544491183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a0100e80 msgr2=0x7f00a0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 --2- 172.21.15.90:0/3544491183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a0100e80 0x7f00a0104e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f008c009910 tx=0x7f008c009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 -- 172.21.15.90:0/3544491183 shutdown_connections 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 --2- 172.21.15.90:0/3544491183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a0100e80 0x7f00a0104e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 --2- 172.21.15.90:0/3544491183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a01004e0 0x7f00a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 -- 172.21.15.90:0/3544491183 >> 172.21.15.90:0/3544491183 conn(0x7f00a00fbda0 msgr2=0x7f00a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 -- 172.21.15.90:0/3544491183 shutdown_connections 2024-04-24T08:30:37.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.373+0000 7f00a5e00700 1 -- 172.21.15.90:0/3544491183 wait complete. 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.374+0000 7f00a5e00700 1 Processor -- start 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.374+0000 7f00a5e00700 1 -- start start 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f00a5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f00a5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a0100e80 0x7f00a01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f00a5e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0198ca0 con 0x7f00a01004e0 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f00a5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a019ca30 con 0x7f00a0100e80 2024-04-24T08:30:37.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54716/0 (socket says 172.21.15.90:54716) 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009ea00700 1 -- 172.21.15.90:0/78406639 learned_addr learned my addr 172.21.15.90:0/78406639 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009e000700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a0100e80 0x7f00a01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009ea00700 1 -- 172.21.15.90:0/78406639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a0100e80 msgr2=0x7f00a01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.375+0000 7f009ea00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a0100e80 0x7f00a01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f009ea00700 1 -- 172.21.15.90:0/78406639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0095f0 con 0x7f00a01004e0 2024-04-24T08:30:37.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f009ea00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f009000d8d0 tx=0x7f009000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0090009940 con 0x7f00a01004e0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0090010460 con 0x7f00a01004e0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009000f5f0 con 0x7f00a01004e0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a019cd10 con 0x7f00a01004e0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.376+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a019d260 con 0x7f00a01004e0 2024-04-24T08:30:37.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.378+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a01045d0 con 0x7f00a01004e0 2024-04-24T08:30:37.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.379+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f00900105d0 con 0x7f00a01004e0 2024-04-24T08:30:37.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.380+0000 7f00a4a00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 0x7f00840712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.380+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f009008efc0 con 0x7f00a01004e0 2024-04-24T08:30:37.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.380+0000 7f009e000700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 0x7f00840712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.381+0000 7f009e000700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 0x7f00840712a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f00a01996a0 tx=0x7f008c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:37.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.385+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0090057c30 con 0x7f00a01004e0 2024-04-24T08:30:37.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.644+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f00a004f2a0 con 0x7f00a01004e0 2024-04-24T08:30:37.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.645+0000 7f00a4a00700 1 -- 172.21.15.90:0/78406639 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f0090016780 con 0x7f00a01004e0 2024-04-24T08:30:37.647 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:30:37.647 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:30:37.647 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:30:37.647 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:30:37.647 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:30:37.648 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:30:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:37.650 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:30:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 msgr2=0x7f00840712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 0x7f00840712a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f00a01996a0 tx=0x7f008c01a040 comp rx=0 tx=0).stop 2024-04-24T08:30:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 msgr2=0x7f00a0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f009000d8d0 tx=0x7f009000dc90 comp rx=0 tx=0).stop 2024-04-24T08:30:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 shutdown_connections 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f008406edf0 0x7f00840712a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00a01004e0 0x7f00a0198080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 --2- 172.21.15.90:0/78406639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a0100e80 0x7f00a01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 >> 172.21.15.90:0/78406639 conn(0x7f00a00fbda0 msgr2=0x7f00a00fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 shutdown_connections 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.648+0000 7f00a5e00700 1 -- 172.21.15.90:0/78406639 wait complete. 2024-04-24T08:30:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 -- 172.21.15.90:0/120442766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 msgr2=0x7f94e8102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 --2- 172.21.15.90:0/120442766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8102a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009910 tx=0x7f94d8009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 -- 172.21.15.90:0/120442766 shutdown_connections 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 --2- 172.21.15.90:0/120442766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 0x7f94e8107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 --2- 172.21.15.90:0/120442766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8102a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 -- 172.21.15.90:0/120442766 >> 172.21.15.90:0/120442766 conn(0x7f94e80fdf40 msgr2=0x7f94e8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.797+0000 7f94ee800700 1 -- 172.21.15.90:0/120442766 shutdown_connections 2024-04-24T08:30:37.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.798+0000 7f94ee800700 1 -- 172.21.15.90:0/120442766 wait complete. 2024-04-24T08:30:37.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 Processor -- start 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 -- start start 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 0x7f94e8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8198d50 con 0x7f94e81026a0 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94ee800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e819cae0 con 0x7f94e8103040 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94e7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94e7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54722/0 (socket says 172.21.15.90:54722) 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.799+0000 7f94e7400700 1 -- 172.21.15.90:0/2692698976 learned_addr learned my addr 172.21.15.90:0/2692698976 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94e7400700 1 -- 172.21.15.90:0/2692698976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 msgr2=0x7f94e8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94e6a00700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 0x7f94e8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94e7400700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 0x7f94e8198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:37.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94e7400700 1 -- 172.21.15.90:0/2692698976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80095f0 con 0x7f94e81026a0 2024-04-24T08:30:37.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94e7400700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006010 tx=0x7f94d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:37.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d801d070 con 0x7f94e81026a0 2024-04-24T08:30:37.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d800f460 con 0x7f94e81026a0 2024-04-24T08:30:37.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e819cd60 con 0x7f94e81026a0 2024-04-24T08:30:37.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.800+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d8017630 con 0x7f94e81026a0 2024-04-24T08:30:37.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.801+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e819d250 con 0x7f94e81026a0 2024-04-24T08:30:37.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.802+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e810a9b0 con 0x7f94e81026a0 2024-04-24T08:30:37.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.803+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f94d800f5d0 con 0x7f94e81026a0 2024-04-24T08:30:37.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.804+0000 7f94dfe00700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 0x7f94cc0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:37.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.804+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f94d8090570 con 0x7f94e81026a0 2024-04-24T08:30:37.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.804+0000 7f94e6a00700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 0x7f94cc0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:37.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.805+0000 7f94e6a00700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 0x7f94cc0712b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94d4005950 tx=0x7f94d40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:37.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:37.809+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f94d8059140 con 0x7f94e81026a0 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: Reconfiguring mds.cephfs.smithi149.zwjtmw (monmap changed)... 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: Reconfiguring daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2732746263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:37.887 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:37 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/78406639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:30:38.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.038+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94e81995a0 con 0x7f94cc06ee00 2024-04-24T08:30:38.039 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: Reconfiguring mds.cephfs.smithi149.zwjtmw (monmap changed)... 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: Reconfiguring daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2732746263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:38.040 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:37 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/78406639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:30:38.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.040+0000 7f94dfe00700 1 -- 172.21.15.90:0/2692698976 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f94e81995a0 con 0x7f94cc06ee00 2024-04-24T08:30:38.041 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:30:38.041 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "4/23 daemons upgraded", 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:30:38.042 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 msgr2=0x7f94cc0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 0x7f94cc0712b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94d4005950 tx=0x7f94d40058e0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 msgr2=0x7f94e8198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006010 tx=0x7f94d800be20 comp rx=0 tx=0).stop 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 shutdown_connections 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f94cc06ee00 0x7f94cc0712b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94e81026a0 0x7f94e8198130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 --2- 172.21.15.90:0/2692698976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94e8103040 0x7f94e8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 >> 172.21.15.90:0/2692698976 conn(0x7f94e80fdf40 msgr2=0x7f94e80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:38.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 shutdown_connections 2024-04-24T08:30:38.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.043+0000 7f94ee800700 1 -- 172.21.15.90:0/2692698976 wait complete. 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 -- 172.21.15.90:0/1575594145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 msgr2=0x7f262c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 --2- 172.21.15.90:0/1575594145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c106ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2618009910 tx=0x7f2618009c20 comp rx=0 tx=0).stop 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 -- 172.21.15.90:0/1575594145 shutdown_connections 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 --2- 172.21.15.90:0/1575594145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c106ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 --2- 172.21.15.90:0/1575594145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 0x7f262c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 -- 172.21.15.90:0/1575594145 >> 172.21.15.90:0/1575594145 conn(0x7f262c076060 msgr2=0x7f262c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:38.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 -- 172.21.15.90:0/1575594145 shutdown_connections 2024-04-24T08:30:38.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.187+0000 7f2632800700 1 -- 172.21.15.90:0/1575594145 wait complete. 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 Processor -- start 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 -- start start 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 0x7f262c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c19d100 con 0x7f262c106b80 2024-04-24T08:30:38.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.188+0000 7f2632800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c1a0e90 con 0x7f262c10c8f0 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54748/0 (socket says 172.21.15.90:54748) 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 -- 172.21.15.90:0/3824421432 learned_addr learned my addr 172.21.15.90:0/3824421432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262aa00700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 0x7f262c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 -- 172.21.15.90:0/3824421432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 msgr2=0x7f262c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 0x7f262c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.189+0000 7f262b400700 1 -- 172.21.15.90:0/3824421432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180095f0 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f262b400700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b4b0 tx=0x7f261c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261c011800 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f261c011e20 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c1a1170 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261c00f2e0 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.190+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1a16c0 con 0x7f262c106b80 2024-04-24T08:30:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.191+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c109b20 con 0x7f262c106b80 2024-04-24T08:30:38.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.192+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f261c010380 con 0x7f262c106b80 2024-04-24T08:30:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.193+0000 7f2623e00700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 0x7f26100712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:30:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.193+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f261c08eb40 con 0x7f262c106b80 2024-04-24T08:30:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.193+0000 7f262aa00700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 0x7f26100712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:30:38.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.194+0000 7f262aa00700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 0x7f26100712b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2618005f50 tx=0x7f261801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:30:38.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.198+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f261c0578d0 con 0x7f262c106b80 2024-04-24T08:30:38.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.519+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f262c1a1990 con 0x7f262c106b80 2024-04-24T08:30:38.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.520+0000 7f2623e00700 1 -- 172.21.15.90:0/3824421432 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f261c0150e0 con 0x7f262c106b80 2024-04-24T08:30:38.521 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:38.521 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:30:38.521 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:30:38.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 msgr2=0x7f26100712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 0x7f26100712b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2618005f50 tx=0x7f261801a040 comp rx=0 tx=0).stop 2024-04-24T08:30:38.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 msgr2=0x7f262c19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:30:38.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b4b0 tx=0x7f261c00b870 comp rx=0 tx=0).stop 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 shutdown_connections 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f261006ee00 0x7f26100712b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f262c106b80 0x7f262c19c4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 --2- 172.21.15.90:0/3824421432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f262c10c8f0 0x7f262c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 >> 172.21.15.90:0/3824421432 conn(0x7f262c076060 msgr2=0x7f262c077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 shutdown_connections 2024-04-24T08:30:38.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:30:38.523+0000 7f2632800700 1 -- 172.21.15.90:0/3824421432 wait complete. 2024-04-24T08:30:38.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:38 smithi090 ceph-mon[85282]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:38.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:38 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3824421432' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:30:38.961 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:38 smithi149 ceph-mon[66490]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:30:38.961 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:38 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3824421432' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all mon 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.375 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]: dispatch 2024-04-24T08:30:40.376 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]': finished 2024-04-24T08:30:40.376 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]: dispatch 2024-04-24T08:30:40.376 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:40 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]': finished 2024-04-24T08:30:40.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all mon 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]: dispatch 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]': finished 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]: dispatch 2024-04-24T08:30:40.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:40 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]': finished 2024-04-24T08:30:42.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:42 smithi090 ceph-mon[85282]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:42.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:42.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:42.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:42 smithi149 ceph-mon[66490]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:42.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:43.167 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:43 smithi090 ceph-mon[85282]: Upgrade: Updating crash.smithi090 (1/2) 2024-04-24T08:30:43.167 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:43 smithi090 ceph-mon[85282]: Deploying daemon crash.smithi090 on smithi090 2024-04-24T08:30:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:43 smithi149 ceph-mon[66490]: Upgrade: Updating crash.smithi090 (1/2) 2024-04-24T08:30:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:43 smithi149 ceph-mon[66490]: Deploying daemon crash.smithi090 on smithi090 2024-04-24T08:30:44.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:44 smithi090 ceph-mon[85282]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:44 smithi149 ceph-mon[66490]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:30:46.467 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:46 smithi090 ceph-mon[85282]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:46.468 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:46 smithi149 ceph-mon[66490]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:30:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:30:48.209 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:47 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:48.209 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:47 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:48.209 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:47 smithi149 ceph-mon[66490]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:30:48.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:47 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:48.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:47 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:48.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:47 smithi090 ceph-mon[85282]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:30:49.234 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:48 smithi149 ceph-mon[66490]: mgrmap e39: smithi090.jsrfxo(active, since 92s), standbys: smithi149.vavftl 2024-04-24T08:30:49.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:48 smithi090 ceph-mon[85282]: mgrmap e39: smithi090.jsrfxo(active, since 92s), standbys: smithi149.vavftl 2024-04-24T08:30:50.240 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:49 smithi149 ceph-mon[66490]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:50.240 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:49 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:50.240 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:49 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:50.240 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:49 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:50.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:49 smithi090 ceph-mon[85282]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:50.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:49 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:50.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:49 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T08:30:50.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:49 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:30:51.098 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:50 smithi149 ceph-mon[66490]: Upgrade: Updating crash.smithi149 (2/2) 2024-04-24T08:30:51.099 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:50 smithi149 ceph-mon[66490]: Deploying daemon crash.smithi149 on smithi149 2024-04-24T08:30:51.213 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:50 smithi090 ceph-mon[85282]: Upgrade: Updating crash.smithi149 (2/2) 2024-04-24T08:30:51.213 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:50 smithi090 ceph-mon[85282]: Deploying daemon crash.smithi149 on smithi149 2024-04-24T08:30:52.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:51 smithi149 ceph-mon[66490]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:52.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:51 smithi090 ceph-mon[85282]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:54 smithi149 ceph-mon[66490]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:30:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:54 smithi090 ceph-mon[85282]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:30:56.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:56.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:56.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:30:56.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:55 smithi149 ceph-mon[66490]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:56.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:56.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:30:56.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:30:56.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:55 smithi090 ceph-mon[85282]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-24T08:30:58.297 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:30:58 smithi149 ceph-mon[66490]: pgmap v54: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:30:58.389 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:30:58 smithi090 ceph-mon[85282]: pgmap v54: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-24T08:31:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:00 smithi149 ceph-mon[66490]: pgmap v55: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:31:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:00.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:00 smithi090 ceph-mon[85282]: pgmap v55: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:31:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:00.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:02.416 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:02 smithi090 ceph-mon[85282]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:31:02.416 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:02.416 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:02.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:02 smithi149 ceph-mon[66490]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:31:02.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:02.474 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:03.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:03.677 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:03.678 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:03 smithi090 ceph-mon[85282]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:03.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:03 smithi149 ceph-mon[66490]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:05.716 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:05.716 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:05.716 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:05 smithi090 ceph-mon[85282]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:05 smithi149 ceph-mon[66490]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:08 smithi149 ceph-mon[66490]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:08.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:08 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:08.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:08 smithi090 ceph-mon[85282]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:08.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:08.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:08 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.701+0000 7fe2f7a00700 1 -- 172.21.15.90:0/890895759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f00739a0 msgr2=0x7fe2f010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.701+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/890895759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f00739a0 0x7fe2f010c5e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009940 tx=0x7fe2dc009c50 comp rx=0 tx=0).stop 2024-04-24T08:31:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 -- 172.21.15.90:0/890895759 shutdown_connections 2024-04-24T08:31:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/890895759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f00739a0 0x7fe2f010c5e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/890895759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0073090 0x7fe2f0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 -- 172.21.15.90:0/890895759 >> 172.21.15.90:0/890895759 conn(0x7fe2f00fbda0 msgr2=0x7fe2f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 -- 172.21.15.90:0/890895759 shutdown_connections 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.702+0000 7fe2f7a00700 1 -- 172.21.15.90:0/890895759 wait complete. 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.703+0000 7fe2f7a00700 1 Processor -- start 2024-04-24T08:31:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.703+0000 7fe2f7a00700 1 -- start start 2024-04-24T08:31:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f7a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f7a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00739a0 0x7fe2f01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f7a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f0198dd0 con 0x7fe2f0073090 2024-04-24T08:31:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f7a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f019cb60 con 0x7fe2f00739a0 2024-04-24T08:31:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:08.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44696/0 (socket says 172.21.15.90:44696) 2024-04-24T08:31:08.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.704+0000 7fe2f5000700 1 -- 172.21.15.90:0/1620421820 learned_addr learned my addr 172.21.15.90:0/1620421820 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:08.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2efe00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00739a0 0x7fe2f01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2f5000700 1 -- 172.21.15.90:0/1620421820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00739a0 msgr2=0x7fe2f01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2f5000700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00739a0 0x7fe2f01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2f5000700 1 -- 172.21.15.90:0/1620421820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0095f0 con 0x7fe2f0073090 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2f5000700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c8a0 tx=0x7fe2e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e400cea0 con 0x7fe2f0073090 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f019ce40 con 0x7fe2f0073090 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.705+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2e4004830 con 0x7fe2f0073090 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.706+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f019d390 con 0x7fe2f0073090 2024-04-24T08:31:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.706+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e40055b0 con 0x7fe2f0073090 2024-04-24T08:31:08.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.707+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d0005320 con 0x7fe2f0073090 2024-04-24T08:31:08.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.708+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe2e4007ad0 con 0x7fe2f0073090 2024-04-24T08:31:08.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.709+0000 7fe2ed600700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 0x7fe2d8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:08.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.709+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe2e408ebc0 con 0x7fe2f0073090 2024-04-24T08:31:08.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.709+0000 7fe2efe00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 0x7fe2d8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:08.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.710+0000 7fe2efe00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 0x7fe2d8071170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01997d0 tx=0x7fe2dc005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:08.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.717+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe2e4057930 con 0x7fe2f0073090 2024-04-24T08:31:08.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.972+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2d0000bf0 con 0x7fe2d806ecc0 2024-04-24T08:31:08.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.974+0000 7fe2ed600700 1 -- 172.21.15.90:0/1620421820 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fe2d0000bf0 con 0x7fe2d806ecc0 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 msgr2=0x7fe2d8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 0x7fe2d8071170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01997d0 tx=0x7fe2dc005a90 comp rx=0 tx=0).stop 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 msgr2=0x7fe2f01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c8a0 tx=0x7fe2e400cc60 comp rx=0 tx=0).stop 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 shutdown_connections 2024-04-24T08:31:08.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fe2d806ecc0 0x7fe2d8071170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2f0073090 0x7fe2f01981b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 --2- 172.21.15.90:0/1620421820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f00739a0 0x7fe2f01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:08.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 >> 172.21.15.90:0/1620421820 conn(0x7fe2f00fbda0 msgr2=0x7fe2f0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:08.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 shutdown_connections 2024-04-24T08:31:08.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:08.978+0000 7fe2f7a00700 1 -- 172.21.15.90:0/1620421820 wait complete. 2024-04-24T08:31:08.996 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.141+0000 7fc652800700 1 -- 172.21.15.90:0/2750579767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068540 msgr2=0x7fc64c068910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.141+0000 7fc652800700 1 --2- 172.21.15.90:0/2750579767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068540 0x7fc64c068910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009940 tx=0x7fc63c009c50 comp rx=0 tx=0).stop 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 -- 172.21.15.90:0/2750579767 shutdown_connections 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 --2- 172.21.15.90:0/2750579767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068ee0 0x7fc64c110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 --2- 172.21.15.90:0/2750579767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068540 0x7fc64c068910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 -- 172.21.15.90:0/2750579767 >> 172.21.15.90:0/2750579767 conn(0x7fc64c075130 msgr2=0x7fc64c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 -- 172.21.15.90:0/2750579767 shutdown_connections 2024-04-24T08:31:09.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.142+0000 7fc652800700 1 -- 172.21.15.90:0/2750579767 wait complete. 2024-04-24T08:31:09.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.143+0000 7fc652800700 1 Processor -- start 2024-04-24T08:31:09.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.143+0000 7fc652800700 1 -- start start 2024-04-24T08:31:09.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc652800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068540 0x7fc64c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc652800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc652800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c19d070 con 0x7fc64c068ee0 2024-04-24T08:31:09.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc652800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c1a0e00 con 0x7fc64c068540 2024-04-24T08:31:09.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc64aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc64aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44714/0 (socket says 172.21.15.90:44714) 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc64aa00700 1 -- 172.21.15.90:0/3866240354 learned_addr learned my addr 172.21.15.90:0/3866240354 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.144+0000 7fc64b400700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068540 0x7fc64c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.145+0000 7fc64aa00700 1 -- 172.21.15.90:0/3866240354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068540 msgr2=0x7fc64c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.145+0000 7fc64aa00700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068540 0x7fc64c19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.145+0000 7fc64aa00700 1 -- 172.21.15.90:0/3866240354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0095f0 con 0x7fc64c068ee0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.145+0000 7fc64aa00700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c8a0 tx=0x7fc63800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.146+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6380078e0 con 0x7fc64c068ee0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.146+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c1a10e0 con 0x7fc64c068ee0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.146+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc63800f450 con 0x7fc64c068ee0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.146+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc63800e5e0 con 0x7fc64c068ee0 2024-04-24T08:31:09.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.146+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c1a1630 con 0x7fc64c068ee0 2024-04-24T08:31:09.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.147+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c101410 con 0x7fc64c068ee0 2024-04-24T08:31:09.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.149+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc63800f5c0 con 0x7fc64c068ee0 2024-04-24T08:31:09.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.150+0000 7fc643e00700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 0x7fc630071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.150+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc63808ecc0 con 0x7fc64c068ee0 2024-04-24T08:31:09.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.150+0000 7fc64b400700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 0x7fc630071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.151+0000 7fc64b400700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 0x7fc630071170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006010 tx=0x7fc63c00b750 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:09.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.155+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc638057af0 con 0x7fc64c068ee0 2024-04-24T08:31:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.409+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc64c1a1970 con 0x7fc63006ecc0 2024-04-24T08:31:09.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.412+0000 7fc643e00700 1 -- 172.21.15.90:0/3866240354 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fc64c1a1970 con 0x7fc63006ecc0 2024-04-24T08:31:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 msgr2=0x7fc630071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 0x7fc630071170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006010 tx=0x7fc63c00b750 comp rx=0 tx=0).stop 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 msgr2=0x7fc64c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c8a0 tx=0x7fc63800cbb0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 shutdown_connections 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc63006ecc0 0x7fc630071170 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc64c068540 0x7fc64c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 --2- 172.21.15.90:0/3866240354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc64c068ee0 0x7fc64c19c990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 >> 172.21.15.90:0/3866240354 conn(0x7fc64c075130 msgr2=0x7fc64c10eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 shutdown_connections 2024-04-24T08:31:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.416+0000 7fc652800700 1 -- 172.21.15.90:0/3866240354 wait complete. 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 -- 172.21.15.90:0/795459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168102470 msgr2=0x7f41681028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 --2- 172.21.15.90:0/795459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168102470 0x7f41681028e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4154009910 tx=0x7f4154009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 -- 172.21.15.90:0/795459749 shutdown_connections 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 --2- 172.21.15.90:0/795459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168102470 0x7f41681028e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 --2- 172.21.15.90:0/795459749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168108470 0x7f4168108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.576+0000 7f416da00700 1 -- 172.21.15.90:0/795459749 >> 172.21.15.90:0/795459749 conn(0x7f41680fdf90 msgr2=0x7f41681003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.577+0000 7f416da00700 1 -- 172.21.15.90:0/795459749 shutdown_connections 2024-04-24T08:31:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.577+0000 7f416da00700 1 -- 172.21.15.90:0/795459749 wait complete. 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 Processor -- start 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 -- start start 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168102470 0x7f4168198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168198eb0 con 0x7f4168108470 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f416da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416819cc40 con 0x7f4168102470 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.578+0000 7f4166000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44736/0 (socket says 172.21.15.90:44736) 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 -- 172.21.15.90:0/3012260996 learned_addr learned my addr 172.21.15.90:0/3012260996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166a00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168102470 0x7f4168198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 -- 172.21.15.90:0/3012260996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168102470 msgr2=0x7f4168198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168102470 0x7f4168198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 -- 172.21.15.90:0/3012260996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540095f0 con 0x7f4168108470 2024-04-24T08:31:09.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.579+0000 7f4166000700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f415400b5c0 tx=0x7f415400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:09.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.580+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f415401d070 con 0x7f4168108470 2024-04-24T08:31:09.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.580+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4154005e00 con 0x7f4168108470 2024-04-24T08:31:09.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.580+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416819cec0 con 0x7f4168108470 2024-04-24T08:31:09.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.580+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4154017400 con 0x7f4168108470 2024-04-24T08:31:09.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.580+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416819d3b0 con 0x7f4168108470 2024-04-24T08:31:09.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.581+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416810a970 con 0x7f4168108470 2024-04-24T08:31:09.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.582+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f415400f460 con 0x7f4168108470 2024-04-24T08:31:09.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.583+0000 7f415f400700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 0x7f414c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:09.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.583+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4154090270 con 0x7f4168108470 2024-04-24T08:31:09.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.583+0000 7f4166a00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 0x7f414c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:09.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.584+0000 7f4166a00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 0x7f414c071170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4158005950 tx=0x7f41580058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:09.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.588+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4154059100 con 0x7f4168108470 2024-04-24T08:31:09.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.837+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41681996d0 con 0x7f414c06ecc0 2024-04-24T08:31:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.856+0000 7f415f400700 1 -- 172.21.15.90:0/3012260996 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f41681996d0 con 0x7f414c06ecc0 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (4m) 7s ago 18m 14.2M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (18m) 7s ago 18m 8368k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (17m) 10s ago 17m 8284k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (23s) 7s ago 18m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (15s) 10s ago 16m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:31:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (3m) 7s ago 17m 72.6M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (10m) 7s ago 10m 86.7M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (10m) 7s ago 10m 17.3M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (10m) 10s ago 10m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (10m) 10s ago 10m 17.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (7m) 7s ago 19m 500M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (5m) 10s ago 16m 450M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (2m) 7s ago 20m 46.0M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (88s) 10s ago 16m 37.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (5m) 7s ago 18m 7894k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (5m) 10s ago 16m 7948k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (15m) 7s ago 15m 218M 1216M 18.2.0 dc2bc1663786 2b3731ff64ab 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (15m) 7s ago 15m 226M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:31:09.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (14m) 7s ago 14m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:31:09.860 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (13m) 10s ago 13m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:31:09.860 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (13m) 10s ago 13m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:31:09.860 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (12m) 10s ago 12m 204M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:31:09.860 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (4m) 7s ago 17m 66.1M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:31:09.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 msgr2=0x7f414c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 0x7f414c071170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4158005950 tx=0x7f41580058e0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 msgr2=0x7f41681987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f415400b5c0 tx=0x7f415400be20 comp rx=0 tx=0).stop 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 shutdown_connections 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f414c06ecc0 0x7f414c071170 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4168102470 0x7f4168198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 --2- 172.21.15.90:0/3012260996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168108470 0x7f41681987d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 >> 172.21.15.90:0/3012260996 conn(0x7f41680fdf90 msgr2=0x7f41680ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 shutdown_connections 2024-04-24T08:31:09.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:09.859+0000 7f416da00700 1 -- 172.21.15.90:0/3012260996 wait complete. 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.021+0000 7f12e7c00700 1 -- 172.21.15.90:0/2317370373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 msgr2=0x7f12e00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.021+0000 7f12e7c00700 1 --2- 172.21.15.90:0/2317370373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e00689b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009910 tx=0x7f12cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 -- 172.21.15.90:0/2317370373 shutdown_connections 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 --2- 172.21.15.90:0/2317370373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e00689b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 --2- 172.21.15.90:0/2317370373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 0x7f12e01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 -- 172.21.15.90:0/2317370373 >> 172.21.15.90:0/2317370373 conn(0x7f12e0075130 msgr2=0x7f12e0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:10.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 -- 172.21.15.90:0/2317370373 shutdown_connections 2024-04-24T08:31:10.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.022+0000 7f12e7c00700 1 -- 172.21.15.90:0/2317370373 wait complete. 2024-04-24T08:31:10.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.023+0000 7f12e7c00700 1 Processor -- start 2024-04-24T08:31:10.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.023+0000 7f12e7c00700 1 -- start start 2024-04-24T08:31:10.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e7c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e7c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 0x7f12e01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e7c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0198c90 con 0x7f12e0068540 2024-04-24T08:31:10.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e7c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e019ca20 con 0x7f12e0101000 2024-04-24T08:31:10.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46626/0 (socket says 172.21.15.90:46626) 2024-04-24T08:31:10.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 -- 172.21.15.90:0/3475667215 learned_addr learned my addr 172.21.15.90:0/3475667215 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:10.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 -- 172.21.15.90:0/3475667215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 msgr2=0x7f12e01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12dfe00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 0x7f12e01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 0x7f12e01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.024+0000 7f12e5200700 1 -- 172.21.15.90:0/3475667215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0095f0 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12e5200700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12d400d8d0 tx=0x7f12d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d4009940 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d4010460 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d400f5f0 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e019cd00 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.025+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e019d250 con 0x7f12e0068540 2024-04-24T08:31:10.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.027+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e004f2a0 con 0x7f12e0068540 2024-04-24T08:31:10.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.028+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f12d40105d0 con 0x7f12e0068540 2024-04-24T08:31:10.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.029+0000 7f12dd600700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 0x7f12c8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.029+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f12d408ed10 con 0x7f12e0068540 2024-04-24T08:31:10.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.029+0000 7f12dfe00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 0x7f12c8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.030+0000 7f12dfe00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 0x7f12c8071170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006010 tx=0x7f12cc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.033+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f12d4057ba0 con 0x7f12e0068540 2024-04-24T08:31:10.185 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:10 smithi090 ceph-mon[85282]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:10.186 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:10 smithi090 ceph-mon[85282]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.358+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f12e019d560 con 0x7f12e0068540 2024-04-24T08:31:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.358+0000 7f12dd600700 1 -- 172.21.15.90:0/3475667215 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f12d405b260 con 0x7f12e0068540 2024-04-24T08:31:10.359 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:31:10.359 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-04-24T08:31:10.360 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:31:10.361 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:31:10.361 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 msgr2=0x7f12c8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 0x7f12c8071170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006010 tx=0x7f12cc00b540 comp rx=0 tx=0).stop 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 msgr2=0x7f12e0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12d400d8d0 tx=0x7f12d400dc90 comp rx=0 tx=0).stop 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 shutdown_connections 2024-04-24T08:31:10.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f12c806ecc0 0x7f12c8071170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12e0068540 0x7f12e0198070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 --2- 172.21.15.90:0/3475667215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e0101000 0x7f12e01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 >> 172.21.15.90:0/3475667215 conn(0x7f12e0075130 msgr2=0x7f12e00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:10.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 shutdown_connections 2024-04-24T08:31:10.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.361+0000 7f12e7c00700 1 -- 172.21.15.90:0/3475667215 wait complete. 2024-04-24T08:31:10.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:10 smithi149 ceph-mon[66490]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:10.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:10 smithi149 ceph-mon[66490]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 -- 172.21.15.90:0/2217776909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 msgr2=0x7efc4c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 --2- 172.21.15.90:0/2217776909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c10c5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc38009940 tx=0x7efc38009c50 comp rx=0 tx=0).stop 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 -- 172.21.15.90:0/2217776909 shutdown_connections 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 --2- 172.21.15.90:0/2217776909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c10c5c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 --2- 172.21.15.90:0/2217776909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 0x7efc4c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.513+0000 7efc53c00700 1 -- 172.21.15.90:0/2217776909 >> 172.21.15.90:0/2217776909 conn(0x7efc4c0fbda0 msgr2=0x7efc4c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.514+0000 7efc53c00700 1 -- 172.21.15.90:0/2217776909 shutdown_connections 2024-04-24T08:31:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.514+0000 7efc53c00700 1 -- 172.21.15.90:0/2217776909 wait complete. 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 Processor -- start 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 -- start start 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 0x7efc4c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c198e50 con 0x7efc4c0739c0 2024-04-24T08:31:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.515+0000 7efc53c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c19cbe0 con 0x7efc4c0730b0 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46646/0 (socket says 172.21.15.90:46646) 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 -- 172.21.15.90:0/2942995862 learned_addr learned my addr 172.21.15.90:0/2942995862 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc51200700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 0x7efc4c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 -- 172.21.15.90:0/2942995862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 msgr2=0x7efc4c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 0x7efc4c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.516+0000 7efc4be00700 1 -- 172.21.15.90:0/2942995862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380095f0 con 0x7efc4c0739c0 2024-04-24T08:31:10.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc4be00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc3800b580 tx=0x7efc380055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3801d070 con 0x7efc4c0739c0 2024-04-24T08:31:10.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc38005dc0 con 0x7efc4c0739c0 2024-04-24T08:31:10.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc38021410 con 0x7efc4c0739c0 2024-04-24T08:31:10.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c19ce60 con 0x7efc4c0739c0 2024-04-24T08:31:10.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.517+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c19d350 con 0x7efc4c0739c0 2024-04-24T08:31:10.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.519+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c109cc0 con 0x7efc4c0739c0 2024-04-24T08:31:10.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.520+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efc3800f7f0 con 0x7efc4c0739c0 2024-04-24T08:31:10.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.521+0000 7efc49600700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 0x7efc34071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.521+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efc380900f0 con 0x7efc4c0739c0 2024-04-24T08:31:10.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.521+0000 7efc51200700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 0x7efc34071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.522+0000 7efc51200700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 0x7efc34071170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc40005950 tx=0x7efc4000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.526+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc38058e60 con 0x7efc4c0739c0 2024-04-24T08:31:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.814+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efc4c04f2a0 con 0x7efc4c0739c0 2024-04-24T08:31:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.814+0000 7efc49600700 1 -- 172.21.15.90:0/2942995862 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7efc3805c520 con 0x7efc4c0739c0 2024-04-24T08:31:10.816 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:31:10.817 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:31:10.818 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:31:10.819 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:10.820 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 msgr2=0x7efc34071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 0x7efc34071170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc40005950 tx=0x7efc4000b410 comp rx=0 tx=0).stop 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 msgr2=0x7efc4c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc3800b580 tx=0x7efc380055f0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 shutdown_connections 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efc3406ecc0 0x7efc34071170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc4c0730b0 0x7efc4c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 --2- 172.21.15.90:0/2942995862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc4c0739c0 0x7efc4c198770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.817+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 >> 172.21.15.90:0/2942995862 conn(0x7efc4c0fbda0 msgr2=0x7efc4c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.818+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 shutdown_connections 2024-04-24T08:31:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.818+0000 7efc53c00700 1 -- 172.21.15.90:0/2942995862 wait complete. 2024-04-24T08:31:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:31:10.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 -- 172.21.15.90:0/1012824890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d0100e80 msgr2=0x7f47d0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1012824890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d0100e80 0x7f47d0104e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009910 tx=0x7f47bc009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 -- 172.21.15.90:0/1012824890 shutdown_connections 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1012824890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d0100e80 0x7f47d0104e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1012824890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d01004e0 0x7f47d01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 -- 172.21.15.90:0/1012824890 >> 172.21.15.90:0/1012824890 conn(0x7f47d00fbda0 msgr2=0x7f47d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 -- 172.21.15.90:0/1012824890 shutdown_connections 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.967+0000 7f47d5a00700 1 -- 172.21.15.90:0/1012824890 wait complete. 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.968+0000 7f47d5a00700 1 Processor -- start 2024-04-24T08:31:10.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47d5a00700 1 -- start start 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47d5a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47d5a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d0100e80 0x7f47d0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47d5a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0073ad0 con 0x7f47d01004e0 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47d5a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0073c10 con 0x7f47d0100e80 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46670/0 (socket says 172.21.15.90:46670) 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.969+0000 7f47cea00700 1 -- 172.21.15.90:0/1101079073 learned_addr learned my addr 172.21.15.90:0/1101079073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47ce000700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d0100e80 0x7f47d0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47cea00700 1 -- 172.21.15.90:0/1101079073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d0100e80 msgr2=0x7f47d0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47cea00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d0100e80 0x7f47d0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47cea00700 1 -- 172.21.15.90:0/1101079073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0095f0 con 0x7f47d01004e0 2024-04-24T08:31:10.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47cea00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47c000d8d0 tx=0x7f47c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c0009940 con 0x7f47d01004e0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47c0010460 con 0x7f47d01004e0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d01a2100 con 0x7f47d01004e0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.970+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c000f5f0 con 0x7f47d01004e0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.971+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d01a2510 con 0x7f47d01004e0 2024-04-24T08:31:10.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.972+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0108850 con 0x7f47d01004e0 2024-04-24T08:31:10.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.973+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f47c00105d0 con 0x7f47d01004e0 2024-04-24T08:31:10.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.973+0000 7f47c7400700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 0x7f47b4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:10.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.973+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f47c008edb0 con 0x7f47d01004e0 2024-04-24T08:31:10.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.974+0000 7f47ce000700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 0x7f47b4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:10.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.974+0000 7f47ce000700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 0x7f47b4071170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47d0074840 tx=0x7f47bc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:10.978+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f47c0057ba0 con 0x7f47d01004e0 2024-04-24T08:31:11.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.270+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47d0061190 con 0x7f47b406ecc0 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.272+0000 7f47c7400700 1 -- 172.21.15.90:0/1101079073 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f47d0061190 con 0x7f47b406ecc0 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:31:11.273 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "6/23 daemons upgraded", 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:31:11.274 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:31:11.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.275+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 msgr2=0x7f47b4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.275+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 0x7f47b4071170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47d0074840 tx=0x7f47bc01a040 comp rx=0 tx=0).stop 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.275+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 msgr2=0x7f47d0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47c000d8d0 tx=0x7f47c000dc90 comp rx=0 tx=0).stop 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 shutdown_connections 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f47b406ecc0 0x7f47b4071170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47d01004e0 0x7f47d0073050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 --2- 172.21.15.90:0/1101079073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d0100e80 0x7f47d0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 >> 172.21.15.90:0/1101079073 conn(0x7f47d00fbda0 msgr2=0x7f47d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:11.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 shutdown_connections 2024-04-24T08:31:11.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.276+0000 7f47d5a00700 1 -- 172.21.15.90:0/1101079073 wait complete. 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.413 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3475667215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]': finished 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]': finished 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T08:31:11.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:11 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2942995862' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 -- 172.21.15.90:0/1521636743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 msgr2=0x7fd6e80ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 --2- 172.21.15.90:0/1521636743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e80ff720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800b0a0 tx=0x7fd6d800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 -- 172.21.15.90:0/1521636743 shutdown_connections 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 --2- 172.21.15.90:0/1521636743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 --2- 172.21.15.90:0/1521636743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e80ff720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 -- 172.21.15.90:0/1521636743 >> 172.21.15.90:0/1521636743 conn(0x7fd6e80748b0 msgr2=0x7fd6e8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 -- 172.21.15.90:0/1521636743 shutdown_connections 2024-04-24T08:31:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.432+0000 7fd6ee000700 1 -- 172.21.15.90:0/1521636743 wait complete. 2024-04-24T08:31:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.433+0000 7fd6ee000700 1 Processor -- start 2024-04-24T08:31:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6ee000700 1 -- start start 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6ee000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6ee000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6ee000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8196ac0 con 0x7fd6e80ff350 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6ee000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e819a850 con 0x7fd6e80ffc60 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6e7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6e7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46688/0 (socket says 172.21.15.90:46688) 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.434+0000 7fd6e7400700 1 -- 172.21.15.90:0/2414278724 learned_addr learned my addr 172.21.15.90:0/2414278724 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e6a00700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e7400700 1 -- 172.21.15.90:0/2414278724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 msgr2=0x7fd6e81963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e7400700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e81963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e7400700 1 -- 172.21.15.90:0/2414278724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8009d00 con 0x7fd6e80ff350 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e6a00700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e81963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6e7400700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800ccc0 tx=0x7fd6d800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d800e040 con 0x7fd6e80ff350 2024-04-24T08:31:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6d8003de0 con 0x7fd6e80ff350 2024-04-24T08:31:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d801d970 con 0x7fd6e80ff350 2024-04-24T08:31:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.435+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e819aad0 con 0x7fd6e80ff350 2024-04-24T08:31:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.436+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e819b020 con 0x7fd6e80ff350 2024-04-24T08:31:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.437+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8107dd0 con 0x7fd6e80ff350 2024-04-24T08:31:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.438+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd6d8004040 con 0x7fd6e80ff350 2024-04-24T08:31:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.438+0000 7fd6dfe00700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 0x7fd6cc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.439+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd6d80907c0 con 0x7fd6e80ff350 2024-04-24T08:31:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.439+0000 7fd6e6a00700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 0x7fd6cc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:11.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.439+0000 7fd6e6a00700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 0x7fd6cc071170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005fd0 tx=0x7fd6d4005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:11.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.443+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd6d8059530 con 0x7fd6e80ff350 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3475667215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]: dispatch 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]': finished 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]: dispatch 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]': finished 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T08:31:11.529 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:11 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2942995862' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:31:11.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.771+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6e81077b0 con 0x7fd6e80ff350 2024-04-24T08:31:11.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.772+0000 7fd6dfe00700 1 -- 172.21.15.90:0/2414278724 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd6d805cbf0 con 0x7fd6e80ff350 2024-04-24T08:31:11.773 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:31:11.773 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:31:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.774+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 msgr2=0x7fd6cc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.774+0000 7fd6ee000700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 0x7fd6cc071170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005fd0 tx=0x7fd6d4005ee0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.774+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 msgr2=0x7fd6e8195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.774+0000 7fd6ee000700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800ccc0 tx=0x7fd6d800cda0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 shutdown_connections 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd6cc06ecc0 0x7fd6cc071170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6e80ff350 0x7fd6e8195ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 --2- 172.21.15.90:0/2414278724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e80ffc60 0x7fd6e81963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 >> 172.21.15.90:0/2414278724 conn(0x7fd6e80748b0 msgr2=0x7fd6e8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 shutdown_connections 2024-04-24T08:31:11.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:11.775+0000 7fd6ee000700 1 -- 172.21.15.90:0/2414278724 wait complete. 2024-04-24T08:31:12.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:12 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all crash 2024-04-24T08:31:12.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:12 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T08:31:12.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:12 smithi090 ceph-mon[85282]: Upgrade: osd.0 is safe to restart 2024-04-24T08:31:12.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:12 smithi090 ceph-mon[85282]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:12.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:12 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2414278724' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:31:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:12 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all crash 2024-04-24T08:31:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:12 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T08:31:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:12 smithi149 ceph-mon[66490]: Upgrade: osd.0 is safe to restart 2024-04-24T08:31:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:12 smithi149 ceph-mon[66490]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:12 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2414278724' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:31:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:13 smithi090 ceph-mon[85282]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:31:13.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:13 smithi149 ceph-mon[66490]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:31:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:14.242 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:14 smithi090 ceph-mon[85282]: Upgrade: Updating osd.0 2024-04-24T08:31:14.242 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:14 smithi090 ceph-mon[85282]: Deploying daemon osd.0 on smithi090 2024-04-24T08:31:14.242 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:14 smithi090 ceph-mon[85282]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:14 smithi149 ceph-mon[66490]: Upgrade: Updating osd.0 2024-04-24T08:31:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:14 smithi149 ceph-mon[66490]: Deploying daemon osd.0 on smithi090 2024-04-24T08:31:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:14 smithi149 ceph-mon[66490]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:16 smithi090 ceph-mon[85282]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:16.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:16 smithi149 ceph-mon[66490]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:18 smithi149 ceph-mon[66490]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:18.530 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:18 smithi090 ceph-mon[85282]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:18.530 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:18 smithi090 systemd[1]: Stopping Ceph osd.0 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:31:18.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:18 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[39906]: 2024-04-24T08:31:18.527+0000 7f922b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:31:18.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:18 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[39906]: 2024-04-24T08:31:18.527+0000 7f922b800700 -1 osd.0 50 *** Got signal Terminated *** 2024-04-24T08:31:18.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:18 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[39906]: 2024-04-24T08:31:18.527+0000 7f922b800700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:31:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:19 smithi149 ceph-mon[66490]: osd.0 marked itself down and dead 2024-04-24T08:31:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:19 smithi090 ceph-mon[85282]: osd.0 marked itself down and dead 2024-04-24T08:31:19.982 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:19 smithi090 podman[96625]: 2024-04-24 08:31:19.710495655 +0000 UTC m=+1.306944460 container died 2b3731ff64ab97d046ef9b489a0bcc2f1ad43e4e54f06431c83981a06aaf858d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-04-24T08:31:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:20 smithi090 ceph-mon[85282]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:20 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:31:20.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:20 smithi090 ceph-mon[85282]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T08:31:20.294 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:20 smithi090 podman[96625]: 2024-04-24 08:31:20.143007246 +0000 UTC m=+1.739456099 container cleanup 2b3731ff64ab97d046ef9b489a0bcc2f1ad43e4e54f06431c83981a06aaf858d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T08:31:20.294 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:20 smithi090 bash[96625]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0 2024-04-24T08:31:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:20 smithi149 ceph-mon[66490]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:31:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:20 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:31:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:20 smithi149 ceph-mon[66490]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T08:31:20.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:20 smithi090 podman[96645]: 2024-04-24 08:31:20.425450442 +0000 UTC m=+0.702480978 container remove 2b3731ff64ab97d046ef9b489a0bcc2f1ad43e4e54f06431c83981a06aaf858d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-24T08:31:21.520 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:21 smithi090 ceph-mon[85282]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T08:31:21.520 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:21 smithi090 podman[96730]: 2024-04-24 08:31:21.181573117 +0000 UTC m=+0.035541927 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:31:21.521 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:21 smithi090 podman[96730]: 2024-04-24 08:31:21.368419433 +0000 UTC m=+0.222388281 container create 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T08:31:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:21 smithi149 ceph-mon[66490]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T08:31:21.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:21 smithi090 podman[96730]: 2024-04-24 08:31:21.641358237 +0000 UTC m=+0.495327072 container init 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T08:31:21.794 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:21 smithi090 podman[96730]: 2024-04-24 08:31:21.652679418 +0000 UTC m=+0.506648273 container start 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:31:21.794 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:21 smithi090 podman[96730]: 2024-04-24 08:31:21.73936451 +0000 UTC m=+0.593333330 container attach 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T08:31:22.513 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:22 smithi090 podman[96730]: 2024-04-24 08:31:22.109471956 +0000 UTC m=+0.963440788 container died 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:31:22.513 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:22 smithi090 ceph-mon[85282]: pgmap v68: 65 pgs: 3 active+undersized+degraded, 3 active+undersized, 54 active+clean, 5 stale+active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7/267 objects degraded (2.622%) 2024-04-24T08:31:22.513 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:22 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:22 smithi149 ceph-mon[66490]: pgmap v68: 65 pgs: 3 active+undersized+degraded, 3 active+undersized, 54 active+clean, 5 stale+active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7/267 objects degraded (2.622%) 2024-04-24T08:31:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:22 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:23.267 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:22 smithi090 podman[96756]: 2024-04-24 08:31:22.835376375 +0000 UTC m=+0.705432043 container remove 551f810a50e0e657a861e7df01ff72e46cde3783925006834b985b2910539b8d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T08:31:23.268 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:22 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-24T08:31:23.268 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:22 smithi090 systemd[1]: Stopped Ceph osd.0 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:31:23.268 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:22 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.0.service: Consumed 1min 47.303s CPU time. 2024-04-24T08:31:23.543 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:23 smithi090 systemd[1]: Starting Ceph osd.0 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:31:23.935 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:23 smithi090 podman[96862]: 2024-04-24 08:31:23.525600511 +0000 UTC m=+0.036363025 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:31:23.935 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:23 smithi090 podman[96862]: 2024-04-24 08:31:23.636855329 +0000 UTC m=+0.147617873 container create 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T08:31:23.935 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:23 smithi090 podman[96862]: 2024-04-24 08:31:23.933831844 +0000 UTC m=+0.444594395 container init 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:31:24.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-mon[85282]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:24.293 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:23 smithi090 podman[96862]: 2024-04-24 08:31:23.94682858 +0000 UTC m=+0.457591113 container start 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T08:31:24.293 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 podman[96862]: 2024-04-24 08:31:24.0328855 +0000 UTC m=+0.543648074 container attach 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T08:31:24.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:24 smithi149 ceph-mon[66490]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:25.294 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:31:25.294 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:31:25.294 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 bash[96862]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T08:31:25.295 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T08:31:25.296 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate[96876]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T08:31:25.296 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:24 smithi090 podman[96862]: 2024-04-24 08:31:24.925431236 +0000 UTC m=+1.436193792 container died 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS) 2024-04-24T08:31:26.043 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:25 smithi090 podman[97006]: 2024-04-24 08:31:25.737854831 +0000 UTC m=+0.789256150 container remove 5c70b0e787d84308719a2fe5b4e29dafaddda4ebd2fb4e7dbe217d662b3752aa (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:31:26.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:26 smithi090 ceph-mon[85282]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:26.388 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 podman[97073]: 2024-04-24 08:31:26.005257593 +0000 UTC m=+0.034546155 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:31:26.388 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 podman[97073]: 2024-04-24 08:31:26.114130606 +0000 UTC m=+0.143419190 container create a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:31:26.388 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 podman[97073]: 2024-04-24 08:31:26.387115087 +0000 UTC m=+0.416403698 container init a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:31:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:26 smithi149 ceph-mon[66490]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:26.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 podman[97073]: 2024-04-24 08:31:26.398001579 +0000 UTC m=+0.427290170 container start a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:31:26.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 ceph-osd[97092]: -- 172.21.15.90:0/4202087795 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55834c499680 con 0x55834c463000 2024-04-24T08:31:26.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 bash[97073]: a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 2024-04-24T08:31:26.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 systemd[1]: Started Ceph osd.0 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:31:27.293 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:26 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:31:26.974+0000 7fb4fa9a77c0 -1 Falling back to public interface 2024-04-24T08:31:27.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:27.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:27.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:27.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:27 smithi090 ceph-mon[85282]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:28.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:27 smithi149 ceph-mon[66490]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:30 smithi149 ceph-mon[66490]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:30 smithi090 ceph-mon[85282]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:31.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:31 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:31.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:31 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:32.293 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:32 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:31:32.111+0000 7fb4fa9a77c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-24T08:31:32.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:32 smithi090 ceph-mon[85282]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:32 smithi149 ceph-mon[66490]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-24T08:31:32.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:32 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:31:32.505+0000 7fb4fa9a77c0 -1 osd.0 50 log_to_monitors true 2024-04-24T08:31:33.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:33 smithi149 ceph-mon[66490]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:31:33.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:33 smithi090 ceph-mon[85282]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:31:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: pgmap v74: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 51/267 objects degraded (19.101%) 2024-04-24T08:31:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:31:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: osdmap e53: 6 total, 5 up, 6 in 2024-04-24T08:31:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:31:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:34.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:34 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:34.543 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:31:34.271+0000 7fb4e4e00700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: pgmap v74: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 51/267 objects degraded (19.101%) 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: osdmap e53: 6 total, 5 up, 6 in 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:34.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:34 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:35.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:35 smithi090 ceph-mon[85282]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' 2024-04-24T08:31:35.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:35 smithi090 ceph-mon[85282]: pgmap v76: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 51/267 objects degraded (19.101%) 2024-04-24T08:31:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:35 smithi149 ceph-mon[66490]: from='osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826]' entity='osd.0' 2024-04-24T08:31:35.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:35 smithi149 ceph-mon[66490]: pgmap v76: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 51/267 objects degraded (19.101%) 2024-04-24T08:31:36.302 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:31:36.302 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826] boot 2024-04-24T08:31:36.302 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T08:31:36.302 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:31:36.303 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:36.303 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: osd.0 [v2:172.21.15.90:6802/4221900826,v1:172.21.15.90:6803/4221900826] boot 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:37 smithi090 ceph-mon[85282]: osdmap e55: 6 total, 6 up, 6 in 2024-04-24T08:31:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:37 smithi090 ceph-mon[85282]: pgmap v79: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s; 7/267 objects degraded (2.622%) 2024-04-24T08:31:37.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:37 smithi149 ceph-mon[66490]: osdmap e55: 6 total, 6 up, 6 in 2024-04-24T08:31:37.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:37 smithi149 ceph-mon[66490]: pgmap v79: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s; 7/267 objects degraded (2.622%) 2024-04-24T08:31:38.507 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:38 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:38.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:38 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:31:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:39 smithi149 ceph-mon[66490]: pgmap v80: 65 pgs: 25 peering, 40 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:31:39.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:39 smithi090 ceph-mon[85282]: pgmap v80: 65 pgs: 25 peering, 40 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:31:40.671 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:40 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-24T08:31:40.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:40 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-24T08:31:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:41 smithi149 ceph-mon[66490]: pgmap v81: 65 pgs: 13 peering, 52 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:31:41.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:41 smithi090 ceph-mon[85282]: pgmap v81: 65 pgs: 13 peering, 52 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T08:31:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 -- 172.21.15.90:0/1838273208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed40730f0 msgr2=0x7f5ed40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 --2- 172.21.15.90:0/1838273208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed40730f0 0x7f5ed40734c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b0a0 tx=0x7f5ec400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:31:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 -- 172.21.15.90:0/1838273208 shutdown_connections 2024-04-24T08:31:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 --2- 172.21.15.90:0/1838273208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed4073a00 0x7f5ed410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:41.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 --2- 172.21.15.90:0/1838273208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed40730f0 0x7f5ed40734c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:41.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 -- 172.21.15.90:0/1838273208 >> 172.21.15.90:0/1838273208 conn(0x7f5ed40fbda0 msgr2=0x7f5ed40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:41.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 -- 172.21.15.90:0/1838273208 shutdown_connections 2024-04-24T08:31:41.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.948+0000 7f5edba00700 1 -- 172.21.15.90:0/1838273208 wait complete. 2024-04-24T08:31:41.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.949+0000 7f5edba00700 1 Processor -- start 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.949+0000 7f5edba00700 1 -- start start 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5edba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed40730f0 0x7f5ed4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5edba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5edba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed419c990 con 0x7f5ed4073a00 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5edba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed419cb00 con 0x7f5ed40730f0 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37546/0 (socket says 172.21.15.90:37546) 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 -- 172.21.15.90:0/2891632628 learned_addr learned my addr 172.21.15.90:0/2891632628 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ed9000700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed40730f0 0x7f5ed4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 -- 172.21.15.90:0/2891632628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed40730f0 msgr2=0x7f5ed4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed40730f0 0x7f5ed4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.950+0000 7f5ecbe00700 1 -- 172.21.15.90:0/2891632628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec4009d00 con 0x7f5ed4073a00 2024-04-24T08:31:41.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5ecbe00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d8d0 tx=0x7f5ebc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:41.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ebc0098a0 con 0x7f5ed4073a00 2024-04-24T08:31:41.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ebc010460 con 0x7f5ed4073a00 2024-04-24T08:31:41.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ebc00f5f0 con 0x7f5ed4073a00 2024-04-24T08:31:41.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed419cde0 con 0x7f5ed4073a00 2024-04-24T08:31:41.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.951+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed419d330 con 0x7f5ed4073a00 2024-04-24T08:31:41.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.953+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed4109c80 con 0x7f5ed4073a00 2024-04-24T08:31:41.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.954+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5ebc009a00 con 0x7f5ed4073a00 2024-04-24T08:31:41.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.955+0000 7f5ed2000700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 0x7f5eb8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:41.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.955+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5ebc08ed70 con 0x7f5ed4073a00 2024-04-24T08:31:41.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.955+0000 7f5ed9000700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 0x7f5eb8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:41.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.956+0000 7f5ed9000700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 0x7f5eb8071170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009480 tx=0x7f5ec400b7b0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:41.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:41.960+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5ebc057ae0 con 0x7f5ed4073a00 2024-04-24T08:31:42.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.214+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed4199420 con 0x7f5eb806ecc0 2024-04-24T08:31:42.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.217+0000 7f5ed2000700 1 -- 172.21.15.90:0/2891632628 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5ed4199420 con 0x7f5eb806ecc0 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 msgr2=0x7f5eb8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 0x7f5eb8071170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009480 tx=0x7f5ec400b7b0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 msgr2=0x7f5ed4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d8d0 tx=0x7f5ebc00dbe0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 shutdown_connections 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5eb806ecc0 0x7f5eb8071170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ed40730f0 0x7f5ed4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 --2- 172.21.15.90:0/2891632628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed4073a00 0x7f5ed4198650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.219+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 >> 172.21.15.90:0/2891632628 conn(0x7f5ed40fbda0 msgr2=0x7f5ed4106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.220+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 shutdown_connections 2024-04-24T08:31:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.220+0000 7f5edba00700 1 -- 172.21.15.90:0/2891632628 wait complete. 2024-04-24T08:31:42.238 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 -- 172.21.15.90:0/3346655816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 msgr2=0x7f43f010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 --2- 172.21.15.90:0/3346655816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f010ccc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009910 tx=0x7f43e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 -- 172.21.15.90:0/3346655816 shutdown_connections 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 --2- 172.21.15.90:0/3346655816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 0x7f43f0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 --2- 172.21.15.90:0/3346655816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f010ccc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 -- 172.21.15.90:0/3346655816 >> 172.21.15.90:0/3346655816 conn(0x7f43f0076060 msgr2=0x7f43f0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:42.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.382+0000 7f43f5e00700 1 -- 172.21.15.90:0/3346655816 shutdown_connections 2024-04-24T08:31:42.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.383+0000 7f43f5e00700 1 -- 172.21.15.90:0/3346655816 wait complete. 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 Processor -- start 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 -- start start 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 0x7f43f019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f019d120 con 0x7f43f010c8f0 2024-04-24T08:31:42.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43f5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f01a0eb0 con 0x7f43f0106b80 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.384+0000 7f43ee000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37554/0 (socket says 172.21.15.90:37554) 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 -- 172.21.15.90:0/1793845883 learned_addr learned my addr 172.21.15.90:0/1793845883 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43eea00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 0x7f43f019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 -- 172.21.15.90:0/1793845883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 msgr2=0x7f43f019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 0x7f43f019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 -- 172.21.15.90:0/1793845883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00095f0 con 0x7f43f010c8f0 2024-04-24T08:31:42.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.385+0000 7f43ee000700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d8d0 tx=0x7f43dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:42.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.386+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43dc009940 con 0x7f43f010c8f0 2024-04-24T08:31:42.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.386+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43dc010460 con 0x7f43f010c8f0 2024-04-24T08:31:42.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.386+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43dc00f5f0 con 0x7f43f010c8f0 2024-04-24T08:31:42.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.386+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f01a1190 con 0x7f43f010c8f0 2024-04-24T08:31:42.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.386+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f01a16e0 con 0x7f43f010c8f0 2024-04-24T08:31:42.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.387+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f0109b20 con 0x7f43f010c8f0 2024-04-24T08:31:42.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.388+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f43dc0105d0 con 0x7f43f010c8f0 2024-04-24T08:31:42.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.389+0000 7f43f4a00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 0x7f43d4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.389+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f43dc08edb0 con 0x7f43f010c8f0 2024-04-24T08:31:42.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.389+0000 7f43eea00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 0x7f43d4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.390+0000 7f43eea00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 0x7f43d4071170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006010 tx=0x7f43e001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:42.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.394+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f43dc057ba0 con 0x7f43f010c8f0 2024-04-24T08:31:42.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.640+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43f01a1950 con 0x7f43d406ecc0 2024-04-24T08:31:42.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.642+0000 7f43f4a00700 1 -- 172.21.15.90:0/1793845883 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f43f01a1950 con 0x7f43d406ecc0 2024-04-24T08:31:42.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 msgr2=0x7f43d4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 0x7f43d4071170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006010 tx=0x7f43e001a040 comp rx=0 tx=0).stop 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 msgr2=0x7f43f019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d8d0 tx=0x7f43dc00dc90 comp rx=0 tx=0).stop 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 shutdown_connections 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f43d406ecc0 0x7f43d4071170 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f0106b80 0x7f43f019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 --2- 172.21.15.90:0/1793845883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010c8f0 0x7f43f019ca40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 >> 172.21.15.90:0/1793845883 conn(0x7f43f0076060 msgr2=0x7f43f0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:42.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 shutdown_connections 2024-04-24T08:31:42.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.646+0000 7f43f5e00700 1 -- 172.21.15.90:0/1793845883 wait complete. 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.806+0000 7f7d36200700 1 -- 172.21.15.90:0/1572118117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 msgr2=0x7f7d300734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.806+0000 7f7d36200700 1 --2- 172.21.15.90:0/1572118117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d300734c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009910 tx=0x7f7d20009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 -- 172.21.15.90:0/1572118117 shutdown_connections 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 --2- 172.21.15.90:0/1572118117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 0x7f7d30110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 --2- 172.21.15.90:0/1572118117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d300734c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 -- 172.21.15.90:0/1572118117 >> 172.21.15.90:0/1572118117 conn(0x7f7d300fbda0 msgr2=0x7f7d300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 -- 172.21.15.90:0/1572118117 shutdown_connections 2024-04-24T08:31:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.807+0000 7f7d36200700 1 -- 172.21.15.90:0/1572118117 wait complete. 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 Processor -- start 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 -- start start 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 0x7f7d3019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3019d000 con 0x7f7d300730f0 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d36200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a0d90 con 0x7f7d30073a00 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.808+0000 7f7d2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37574/0 (socket says 172.21.15.90:37574) 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 -- 172.21.15.90:0/123733678 learned_addr learned my addr 172.21.15.90:0/123733678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d27e00700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 0x7f7d3019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 -- 172.21.15.90:0/123733678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 msgr2=0x7f7d3019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 0x7f7d3019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 -- 172.21.15.90:0/123733678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200095f0 con 0x7f7d300730f0 2024-04-24T08:31:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.809+0000 7f7d2f400700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.810+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d2001d070 con 0x7f7d300730f0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.810+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d2000f460 con 0x7f7d300730f0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.810+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d20017630 con 0x7f7d300730f0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.810+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d301a1070 con 0x7f7d300730f0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.810+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301a1570 con 0x7f7d300730f0 2024-04-24T08:31:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.811+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3010e4d0 con 0x7f7d300730f0 2024-04-24T08:31:42.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.812+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7d2000f5d0 con 0x7f7d300730f0 2024-04-24T08:31:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.813+0000 7f7d2cc00700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 0x7f7d14071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.813+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d200901c0 con 0x7f7d300730f0 2024-04-24T08:31:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.814+0000 7f7d27e00700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 0x7f7d14071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.814+0000 7f7d27e00700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 0x7f7d14071170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d18005950 tx=0x7f7d180058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:42.818+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7d20059050 con 0x7f7d300730f0 2024-04-24T08:31:43.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:43.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:42 smithi149 ceph-mon[66490]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-04-24T08:31:43.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:43.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:42 smithi090 ceph-mon[85282]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-04-24T08:31:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.090+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d301a1850 con 0x7f7d1406ecc0 2024-04-24T08:31:43.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.110+0000 7f7d2cc00700 1 -- 172.21.15.90:0/123733678 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7d301a1850 con 0x7f7d1406ecc0 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (4m) 9s ago 18m 13.7M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (19m) 9s ago 19m 8404k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (17m) 43s ago 17m 8284k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (56s) 9s ago 18m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (48s) 43s ago 17m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:31:43.112 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (4m) 9s ago 18m 73.6M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (11m) 9s ago 11m 87.0M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (11m) 9s ago 11m 17.5M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (11m) 43s ago 11m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (11m) 43s ago 11m 17.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (7m) 9s ago 20m 502M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (6m) 43s ago 17m 450M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (2m) 9s ago 20m 47.1M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (2m) 43s ago 17m 37.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (5m) 9s ago 18m 7684k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (5m) 43s ago 17m 7948k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (17s) 9s ago 16m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (15m) 9s ago 15m 227M 1216M 18.2.0 dc2bc1663786 e9ba17510726 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (15m) 9s ago 15m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (14m) 43s ago 14m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:31:43.113 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (13m) 43s ago 13m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:31:43.114 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (13m) 43s ago 13m 204M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:31:43.114 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 9s ago 18m 66.1M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.114+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 msgr2=0x7f7d14071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.114+0000 7f7d36200700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 0x7f7d14071170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d18005950 tx=0x7f7d180058e0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.114+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 msgr2=0x7f7d3019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2000be20 comp rx=0 tx=0).stop 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 shutdown_connections 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7d1406ecc0 0x7f7d14071170 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d300730f0 0x7f7d3019c3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 --2- 172.21.15.90:0/123733678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d30073a00 0x7f7d3019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 >> 172.21.15.90:0/123733678 conn(0x7f7d300fbda0 msgr2=0x7f7d30102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 shutdown_connections 2024-04-24T08:31:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.115+0000 7f7d36200700 1 -- 172.21.15.90:0/123733678 wait complete. 2024-04-24T08:31:43.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 -- 172.21.15.90:0/3453931671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac1004e0 msgr2=0x7f60ac1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 --2- 172.21.15.90:0/3453931671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac1004e0 0x7f60ac1008b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f609c009910 tx=0x7f609c009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 -- 172.21.15.90:0/3453931671 shutdown_connections 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 --2- 172.21.15.90:0/3453931671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100e80 0x7f60ac104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 --2- 172.21.15.90:0/3453931671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac1004e0 0x7f60ac1008b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 -- 172.21.15.90:0/3453931671 >> 172.21.15.90:0/3453931671 conn(0x7f60ac0fbda0 msgr2=0x7f60ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.274+0000 7f60b1800700 1 -- 172.21.15.90:0/3453931671 shutdown_connections 2024-04-24T08:31:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.275+0000 7f60b1800700 1 -- 172.21.15.90:0/3453931671 wait complete. 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 Processor -- start 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 -- start start 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac1004e0 0x7f60ac195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac196b10 con 0x7f60ac100e80 2024-04-24T08:31:43.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.276+0000 7f60b1800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac19a8a0 con 0x7f60ac1004e0 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37580/0 (socket says 172.21.15.90:37580) 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 -- 172.21.15.90:0/3804277319 learned_addr learned my addr 172.21.15.90:0/3804277319 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60aaa00700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac1004e0 0x7f60ac195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 -- 172.21.15.90:0/3804277319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac1004e0 msgr2=0x7f60ac195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac1004e0 0x7f60ac195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 -- 172.21.15.90:0/3804277319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0095f0 con 0x7f60ac100e80 2024-04-24T08:31:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.277+0000 7f60a3e00700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f609400d900 tx=0x7f609400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.278+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6094009900 con 0x7f60ac100e80 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.278+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6094010460 con 0x7f60ac100e80 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.278+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac19ab80 con 0x7f60ac100e80 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.278+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609400f5f0 con 0x7f60ac100e80 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.278+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac19b0d0 con 0x7f60ac100e80 2024-04-24T08:31:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.279+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac1087f0 con 0x7f60ac100e80 2024-04-24T08:31:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.280+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f60940105d0 con 0x7f60ac100e80 2024-04-24T08:31:43.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.281+0000 7f60a3400700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 0x7f6090071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.281+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f609408ed00 con 0x7f60ac100e80 2024-04-24T08:31:43.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.281+0000 7f60aaa00700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 0x7f6090071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:43.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.282+0000 7f60aaa00700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 0x7f6090071170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f609c006010 tx=0x7f609c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:43.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.286+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6094057a70 con 0x7f60ac100e80 2024-04-24T08:31:43.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.615+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60ac19b2f0 con 0x7f60ac100e80 2024-04-24T08:31:43.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.616+0000 7f60a3400700 1 -- 172.21.15.90:0/3804277319 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f609405b130 con 0x7f60ac100e80 2024-04-24T08:31:43.617 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:31:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 5 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:31:43.619 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 msgr2=0x7f6090071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 0x7f6090071170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f609c006010 tx=0x7f609c01a040 comp rx=0 tx=0).stop 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 msgr2=0x7f60ac196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f609400d900 tx=0x7f609400dcc0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 shutdown_connections 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f609006ecc0 0x7f6090071170 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac1004e0 0x7f60ac195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 --2- 172.21.15.90:0/3804277319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac100e80 0x7f60ac196430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.619+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 >> 172.21.15.90:0/3804277319 conn(0x7f60ac0fbda0 msgr2=0x7f60ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.620+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 shutdown_connections 2024-04-24T08:31:43.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.620+0000 7f60b1800700 1 -- 172.21.15.90:0/3804277319 wait complete. 2024-04-24T08:31:43.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 -- 172.21.15.90:0/2686399517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 msgr2=0x7fc5d4110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 --2- 172.21.15.90:0/2686399517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d4110bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009910 tx=0x7fc5c0009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:43.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 -- 172.21.15.90:0/2686399517 shutdown_connections 2024-04-24T08:31:43.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 --2- 172.21.15.90:0/2686399517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d4110bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 --2- 172.21.15.90:0/2686399517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4075280 0x7fc5d4075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 -- 172.21.15.90:0/2686399517 >> 172.21.15.90:0/2686399517 conn(0x7fc5d40fdf90 msgr2=0x7fc5d41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:43.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 -- 172.21.15.90:0/2686399517 shutdown_connections 2024-04-24T08:31:43.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.780+0000 7fc5daa00700 1 -- 172.21.15.90:0/2686399517 wait complete. 2024-04-24T08:31:43.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.781+0000 7fc5daa00700 1 Processor -- start 2024-04-24T08:31:43.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.781+0000 7fc5daa00700 1 -- start start 2024-04-24T08:31:43.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5daa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4075280 0x7fc5d419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5daa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419d100 con 0x7fc5d4075b90 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5daa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41a0e90 con 0x7fc5d4075280 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37606/0 (socket says 172.21.15.90:37606) 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 -- 172.21.15.90:0/640227987 learned_addr learned my addr 172.21.15.90:0/640227987 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 -- 172.21.15.90:0/640227987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4075280 msgr2=0x7fc5d419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4075280 0x7fc5d419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.782+0000 7fc5d3400700 1 -- 172.21.15.90:0/640227987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00095f0 con 0x7fc5d4075b90 2024-04-24T08:31:43.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5d3400700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00092e0 tx=0x7fc5c00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c001d070 con 0x7fc5d4075b90 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5c0005a00 con 0x7fc5d4075b90 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d41a1110 con 0x7fc5d4075b90 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c0021410 con 0x7fc5d4075b90 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.783+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d41a1660 con 0x7fc5d4075b90 2024-04-24T08:31:43.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.784+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d410e340 con 0x7fc5d4075b90 2024-04-24T08:31:43.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.785+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc5c000f460 con 0x7fc5d4075b90 2024-04-24T08:31:43.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.786+0000 7fc5d0c00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 0x7fc5b8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:43.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.786+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc5c00901e0 con 0x7fc5d4075b90 2024-04-24T08:31:43.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.786+0000 7fc5d3e00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 0x7fc5b8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:43.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.787+0000 7fc5d3e00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 0x7fc5b8071170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400a990 tx=0x7fc5c4005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:43.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:43.791+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc5c0058f50 con 0x7fc5d4075b90 2024-04-24T08:31:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:43 smithi149 ceph-mon[66490]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:43 smithi149 ceph-mon[66490]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:43 smithi149 ceph-mon[66490]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-24T08:31:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:43 smithi149 ceph-mon[66490]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:43 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3804277319' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:43 smithi090 ceph-mon[85282]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:43 smithi090 ceph-mon[85282]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:43 smithi090 ceph-mon[85282]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-24T08:31:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:43 smithi090 ceph-mon[85282]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:44.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:43 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3804277319' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:31:44.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.073+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc5d404f2a0 con 0x7fc5d4075b90 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.074+0000 7fc5d0c00700 1 -- 172.21.15.90:0/640227987 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7fc5c005c610 con 0x7fc5d4075b90 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:31:44.075 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:31:44.076 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:31:44.077 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:44.078 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:31:44.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 msgr2=0x7fc5b8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 0x7fc5b8071170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400a990 tx=0x7fc5c4005c80 comp rx=0 tx=0).stop 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 msgr2=0x7fc5d419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00092e0 tx=0x7fc5c00056a0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 shutdown_connections 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc5b806ecc0 0x7fc5b8071170 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4075280 0x7fc5d419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 --2- 172.21.15.90:0/640227987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4075b90 0x7fc5d419ca20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 >> 172.21.15.90:0/640227987 conn(0x7fc5d40fdf90 msgr2=0x7fc5d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 shutdown_connections 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.077+0000 7fc5daa00700 1 -- 172.21.15.90:0/640227987 wait complete. 2024-04-24T08:31:44.080 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:31:44.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 -- 172.21.15.90:0/290079429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a0100b50 msgr2=0x7f26a0104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 --2- 172.21.15.90:0/290079429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a0100b50 0x7f26a0104b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f268c009910 tx=0x7f268c009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 -- 172.21.15.90:0/290079429 shutdown_connections 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 --2- 172.21.15.90:0/290079429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a0100b50 0x7f26a0104b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 --2- 172.21.15.90:0/290079429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a01001b0 0x7f26a0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 -- 172.21.15.90:0/290079429 >> 172.21.15.90:0/290079429 conn(0x7f26a00fba70 msgr2=0x7f26a00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 -- 172.21.15.90:0/290079429 shutdown_connections 2024-04-24T08:31:44.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.230+0000 7f26a6800700 1 -- 172.21.15.90:0/290079429 wait complete. 2024-04-24T08:31:44.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.231+0000 7f26a6800700 1 Processor -- start 2024-04-24T08:31:44.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.231+0000 7f26a6800700 1 -- start start 2024-04-24T08:31:44.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 0x7f26a0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a6800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0198930 con 0x7f26a01001b0 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a019c6c0 con 0x7f26a0100b50 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37624/0 (socket says 172.21.15.90:37624) 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.232+0000 7f26a5400700 1 -- 172.21.15.90:0/3411980172 learned_addr learned my addr 172.21.15.90:0/3411980172 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a4a00700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 0x7f26a0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a5400700 1 -- 172.21.15.90:0/3411980172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 msgr2=0x7f26a0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a5400700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 0x7f26a0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a5400700 1 -- 172.21.15.90:0/3411980172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0095f0 con 0x7f26a01001b0 2024-04-24T08:31:44.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a4a00700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 0x7f26a0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:31:44.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a5400700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f269400d900 tx=0x7f269400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:44.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2694004a00 con 0x7f26a01001b0 2024-04-24T08:31:44.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26940054e0 con 0x7f26a01001b0 2024-04-24T08:31:44.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a019c9a0 con 0x7f26a01001b0 2024-04-24T08:31:44.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2694009d60 con 0x7f26a01001b0 2024-04-24T08:31:44.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.233+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a019cef0 con 0x7f26a01001b0 2024-04-24T08:31:44.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.234+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a01049b0 con 0x7f26a01001b0 2024-04-24T08:31:44.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.235+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2694005020 con 0x7f26a01001b0 2024-04-24T08:31:44.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.236+0000 7f269e000700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 0x7f2688071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.236+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f269408f070 con 0x7f26a01001b0 2024-04-24T08:31:44.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.237+0000 7f26a4a00700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 0x7f2688071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.237+0000 7f26a4a00700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 0x7f2688071170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f268c00b5c0 tx=0x7f268c0053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:44.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.241+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2694057de0 con 0x7f26a01001b0 2024-04-24T08:31:44.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.484+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26a019d220 con 0x7f268806ecc0 2024-04-24T08:31:44.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.487+0000 7f269e000700 1 -- 172.21.15.90:0/3411980172 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f26a019d220 con 0x7f268806ecc0 2024-04-24T08:31:44.488 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:31:44.488 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:31:44.488 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:31:44.489 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:31:44.490 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "7/23 daemons upgraded", 2024-04-24T08:31:44.490 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:31:44.490 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:31:44.490 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:31:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 msgr2=0x7f2688071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 0x7f2688071170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f268c00b5c0 tx=0x7f268c0053a0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 msgr2=0x7f26a0197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f269400d900 tx=0x7f269400dcc0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 shutdown_connections 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f268806ecc0 0x7f2688071170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26a01001b0 0x7f26a0197d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 --2- 172.21.15.90:0/3411980172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a0100b50 0x7f26a0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 >> 172.21.15.90:0/3411980172 conn(0x7f26a00fba70 msgr2=0x7f26a0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 shutdown_connections 2024-04-24T08:31:44.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.490+0000 7f26a6800700 1 -- 172.21.15.90:0/3411980172 wait complete. 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 -- 172.21.15.90:0/830830504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 msgr2=0x7f745c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 --2- 172.21.15.90:0/830830504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c0ff790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c009c20 comp rx=0 tx=0).stop 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 -- 172.21.15.90:0/830830504 shutdown_connections 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 --2- 172.21.15.90:0/830830504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 --2- 172.21.15.90:0/830830504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c0ff790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.644+0000 7f7461400700 1 -- 172.21.15.90:0/830830504 >> 172.21.15.90:0/830830504 conn(0x7f745c0fac60 msgr2=0x7f745c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:44.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.645+0000 7f7461400700 1 -- 172.21.15.90:0/830830504 shutdown_connections 2024-04-24T08:31:44.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.645+0000 7f7461400700 1 -- 172.21.15.90:0/830830504 wait complete. 2024-04-24T08:31:44.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 Processor -- start 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 -- start start 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c196b90 con 0x7f745c0ff3c0 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f7461400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c19a920 con 0x7f745c0ffd60 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.646+0000 7f745a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37640/0 (socket says 172.21.15.90:37640) 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 -- 172.21.15.90:0/1383987828 learned_addr learned my addr 172.21.15.90:0/1383987828 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 -- 172.21.15.90:0/1383987828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 msgr2=0x7f745c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f7459e00700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 -- 172.21.15.90:0/1383987828 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0095f0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f7459e00700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.647+0000 7f745a800700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:44.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.648+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c01d070 con 0x7f745c0ff3c0 2024-04-24T08:31:44.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.648+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f744c004fb0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.648+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c19aba0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.648+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c022ea0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.648+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c19b090 con 0x7f745c0ff3c0 2024-04-24T08:31:44.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.649+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c1086e0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.650+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f744c022470 con 0x7f745c0ff3c0 2024-04-24T08:31:44.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.651+0000 7f7453400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 0x7f7440071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:31:44.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.651+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f744c090390 con 0x7f745c0ff3c0 2024-04-24T08:31:44.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.651+0000 7f7459e00700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 0x7f7440071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:31:44.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.652+0000 7f7459e00700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 0x7f7440071170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f745c197590 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:31:44.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.656+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f744c059250 con 0x7f745c0ff3c0 2024-04-24T08:31:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.957+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f745c04f2a0 con 0x7f745c0ff3c0 2024-04-24T08:31:44.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.957+0000 7f7453400700 1 -- 172.21.15.90:0/1383987828 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f744c022720 con 0x7f745c0ff3c0 2024-04-24T08:31:44.958 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:31:44.959 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:31:44.959 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:31:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 msgr2=0x7f7440071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 0x7f7440071170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f745c197590 tx=0x7f7448008040 comp rx=0 tx=0).stop 2024-04-24T08:31:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 msgr2=0x7f745c195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c00be20 comp rx=0 tx=0).stop 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 shutdown_connections 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f744006ecc0 0x7f7440071170 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f745c0ff3c0 0x7f745c195f70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 --2- 172.21.15.90:0/1383987828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745c0ffd60 0x7f745c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.960+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 >> 172.21.15.90:0/1383987828 conn(0x7f745c0fac60 msgr2=0x7f745c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.961+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 shutdown_connections 2024-04-24T08:31:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:31:44.961+0000 7f7461400700 1 -- 172.21.15.90:0/1383987828 wait complete. 2024-04-24T08:31:45.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:44 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/640227987' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:31:45.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:44 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/640227987' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:31:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:45 smithi149 ceph-mon[66490]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:45 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1383987828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:31:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:45 smithi149 ceph-mon[66490]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-04-24T08:31:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:46.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:45 smithi090 ceph-mon[85282]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:31:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:45 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1383987828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:31:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:45 smithi090 ceph-mon[85282]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-04-24T08:31:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:46.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:31:48.476 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:48 smithi090 ceph-mon[85282]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 947 B/s rd, 0 op/s 2024-04-24T08:31:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:48 smithi149 ceph-mon[66490]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 947 B/s rd, 0 op/s 2024-04-24T08:31:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:50 smithi149 ceph-mon[66490]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-04-24T08:31:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:50 smithi090 ceph-mon[85282]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-04-24T08:31:51.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:51 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:51.487 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:51 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:31:52.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:52 smithi090 ceph-mon[85282]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:52 smithi149 ceph-mon[66490]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:54 smithi149 ceph-mon[66490]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:54 smithi090 ceph-mon[85282]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:56.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:56 smithi149 ceph-mon[66490]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:31:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:56 smithi090 ceph-mon[85282]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:31:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:57 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:57 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:58.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:58 smithi090 ceph-mon[85282]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:58.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:58 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:58.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:58 smithi090 ceph-mon[85282]: Upgrade: osd.1 is safe to restart 2024-04-24T08:31:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:58 smithi149 ceph-mon[66490]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:31:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:58 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T08:31:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:58 smithi149 ceph-mon[66490]: Upgrade: osd.1 is safe to restart 2024-04-24T08:31:59.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:31:59 smithi090 ceph-mon[85282]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:31:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:31:59 smithi149 ceph-mon[66490]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:32:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: Upgrade: Updating osd.1 2024-04-24T08:32:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:32:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: Deploying daemon osd.1 on smithi090 2024-04-24T08:32:00.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: Upgrade: Updating osd.1 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: Deploying daemon osd.1 on smithi090 2024-04-24T08:32:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:02.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:01 smithi149 ceph-mon[66490]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:32:02.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:01 smithi090 ceph-mon[85282]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:32:04.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:04 smithi149 ceph-mon[66490]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:32:04.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:04 smithi090 ceph-mon[85282]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:32:05.043 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:04 smithi090 systemd[1]: Stopping Ceph osd.1 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:32:05.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:05 smithi149 ceph-mon[66490]: osd.1 marked itself down and dead 2024-04-24T08:32:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:05 smithi090 ceph-mon[85282]: osd.1 marked itself down and dead 2024-04-24T08:32:05.543 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:05 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[45479]: 2024-04-24T08:32:05.066+0000 7f58fbc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:32:05.544 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:05 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[45479]: 2024-04-24T08:32:05.066+0000 7f58fbc00700 -1 osd.1 55 *** Got signal Terminated *** 2024-04-24T08:32:05.544 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:05 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[45479]: 2024-04-24T08:32:05.066+0000 7f58fbc00700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:32:06.044 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:05 smithi090 podman[102183]: 2024-04-24 08:32:05.755830649 +0000 UTC m=+0.823694951 container died e9ba17510726fff0ab44f57ef221fdd22974e0f118061eab83d2e54147100d12 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212) 2024-04-24T08:32:06.428 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:06 smithi149 ceph-mon[66490]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:32:06.429 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:06 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:32:06.429 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:06 smithi149 ceph-mon[66490]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T08:32:06.448 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:06 smithi090 ceph-mon[85282]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:32:06.448 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:06 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:32:06.448 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:06 smithi090 ceph-mon[85282]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T08:32:06.449 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:06 smithi090 podman[102183]: 2024-04-24 08:32:06.178792472 +0000 UTC m=+1.246656778 container cleanup e9ba17510726fff0ab44f57ef221fdd22974e0f118061eab83d2e54147100d12 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T08:32:06.449 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:06 smithi090 bash[102183]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1 2024-04-24T08:32:06.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:06 smithi090 podman[102201]: 2024-04-24 08:32:06.44613819 +0000 UTC m=+0.675066584 container remove e9ba17510726fff0ab44f57ef221fdd22974e0f118061eab83d2e54147100d12 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T08:32:07.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:07 smithi090 ceph-mon[85282]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T08:32:07.515 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:07 smithi090 podman[102289]: 2024-04-24 08:32:07.181960268 +0000 UTC m=+0.032581064 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:07.515 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:07 smithi090 podman[102289]: 2024-04-24 08:32:07.305856221 +0000 UTC m=+0.156477043 container create eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T08:32:07.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:07 smithi149 ceph-mon[66490]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T08:32:07.778 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:07 smithi090 podman[102289]: 2024-04-24 08:32:07.635393105 +0000 UTC m=+0.486013945 container init eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:32:07.778 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:07 smithi090 podman[102289]: 2024-04-24 08:32:07.64656368 +0000 UTC m=+0.497184517 container start eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T08:32:07.778 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:07 smithi090 podman[102289]: 2024-04-24 08:32:07.775912264 +0000 UTC m=+0.626533086 container attach eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:32:08.521 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:08 smithi090 ceph-mon[85282]: pgmap v96: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-24T08:32:08.521 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:08 smithi090 ceph-mon[85282]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-24T08:32:08.521 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:08 smithi090 podman[102289]: 2024-04-24 08:32:08.104415147 +0000 UTC m=+0.955035987 container died eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T08:32:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:08 smithi149 ceph-mon[66490]: pgmap v96: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-24T08:32:08.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:08 smithi149 ceph-mon[66490]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-24T08:32:09.235 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:08 smithi090 podman[102314]: 2024-04-24 08:32:08.840023368 +0000 UTC m=+0.714852576 container remove eebabc96bb15d275c015254aca73ca4ebf7626bfbf60c1fb03f2646f406d456e (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:32:09.235 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:08 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-24T08:32:09.235 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:08 smithi090 systemd[1]: Stopped Ceph osd.1 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:32:09.235 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:08 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 52.568s CPU time. 2024-04-24T08:32:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:09 smithi149 ceph-mon[66490]: pgmap v97: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 2/267 objects degraded (0.749%) 2024-04-24T08:32:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:09 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 2/267 objects degraded (0.749%), 2 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:09.542 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:09 smithi090 systemd[1]: Starting Ceph osd.1 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:32:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:09 smithi090 ceph-mon[85282]: pgmap v97: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 2/267 objects degraded (0.749%) 2024-04-24T08:32:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:09 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 2/267 objects degraded (0.749%), 2 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:09.922 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:09 smithi090 podman[102420]: 2024-04-24 08:32:09.560141009 +0000 UTC m=+0.033072104 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:09.922 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:09 smithi090 podman[102420]: 2024-04-24 08:32:09.657999608 +0000 UTC m=+0.130930733 container create c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T08:32:09.922 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:09 smithi090 podman[102420]: 2024-04-24 08:32:09.920309753 +0000 UTC m=+0.393240888 container init c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T08:32:10.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:09 smithi090 podman[102420]: 2024-04-24 08:32:09.932045611 +0000 UTC m=+0.404976742 container start c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T08:32:10.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 podman[102420]: 2024-04-24 08:32:10.028836185 +0000 UTC m=+0.501767340 container attach c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T08:32:11.044 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:11.044 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 bash[102420]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T08:32:11.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T08:32:11.046 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T08:32:11.046 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate[102435]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T08:32:11.046 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:10 smithi090 podman[102420]: 2024-04-24 08:32:10.930611778 +0000 UTC m=+1.403542940 container died c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T08:32:11.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:11 smithi149 ceph-mon[66490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-24T08:32:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:11 smithi090 ceph-mon[85282]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-24T08:32:12.007 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:11 smithi090 podman[102565]: 2024-04-24 08:32:11.658746283 +0000 UTC m=+0.710212189 container remove c929c6a2b066da985b84598b233d83e78c32d47e7d8c487e888996befa049226 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2) 2024-04-24T08:32:12.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:12 smithi090 ceph-mon[85282]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:12.294 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 podman[102628]: 2024-04-24 08:32:11.912226791 +0000 UTC m=+0.034491332 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:12.294 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 podman[102628]: 2024-04-24 08:32:12.009563185 +0000 UTC m=+0.131827762 container create 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T08:32:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:12 smithi149 ceph-mon[66490]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:12.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 podman[102628]: 2024-04-24 08:32:12.315036132 +0000 UTC m=+0.437300779 container init 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T08:32:12.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 podman[102628]: 2024-04-24 08:32:12.327422113 +0000 UTC m=+0.449686671 container start 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T08:32:12.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 ceph-osd[102648]: -- 172.21.15.90:0/3329836146 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5442f3680 con 0x55a5442bcc00 2024-04-24T08:32:12.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 bash[102628]: 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc 2024-04-24T08:32:12.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 systemd[1]: Started Ceph osd.1 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:32:13.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:12 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:32:12.916+0000 7f5bc9d6e7c0 -1 Falling back to public interface 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: pgmap v99: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:13.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: pgmap v99: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:14.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:14.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.134+0000 7fcfe7e00700 1 -- 172.21.15.90:0/3422065651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 msgr2=0x7fcfe00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.134+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/3422065651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe00734c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009910 tx=0x7fcfd4009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 -- 172.21.15.90:0/3422065651 shutdown_connections 2024-04-24T08:32:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/3422065651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/3422065651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe00734c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 -- 172.21.15.90:0/3422065651 >> 172.21.15.90:0/3422065651 conn(0x7fcfe00fbda0 msgr2=0x7fcfe00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 -- 172.21.15.90:0/3422065651 shutdown_connections 2024-04-24T08:32:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.135+0000 7fcfe7e00700 1 -- 172.21.15.90:0/3422065651 wait complete. 2024-04-24T08:32:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.136+0000 7fcfe7e00700 1 Processor -- start 2024-04-24T08:32:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.136+0000 7fcfe7e00700 1 -- start start 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.137+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe01001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.137+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.137+0000 7fcfe7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0100c60 con 0x7fcfe00730f0 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.137+0000 7fcfe7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0100dd0 con 0x7fcfe0073a00 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.138+0000 7fcfe4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.138+0000 7fcfe4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:46844/0 (socket says 172.21.15.90:46844) 2024-04-24T08:32:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.138+0000 7fcfe4a00700 1 -- 172.21.15.90:0/289562109 learned_addr learned my addr 172.21.15.90:0/289562109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.138+0000 7fcfe5400700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe01001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.139+0000 7fcfe4a00700 1 -- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 msgr2=0x7fcfe01001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.139+0000 7fcfe4a00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe01001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.139+0000 7fcfe4a00700 1 -- 172.21.15.90:0/289562109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd40095f0 con 0x7fcfe0073a00 2024-04-24T08:32:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.139+0000 7fcfe5400700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:32:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfe4a00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b4b0 tx=0x7fcfcc00b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfcc011800 con 0x7fcfe0073a00 2024-04-24T08:32:15.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe0100f10 con 0x7fcfe0073a00 2024-04-24T08:32:15.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfcc011e20 con 0x7fcfe0073a00 2024-04-24T08:32:15.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfcc00f2e0 con 0x7fcfe0073a00 2024-04-24T08:32:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.140+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0198190 con 0x7fcfe0073a00 2024-04-24T08:32:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.142+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0109ce0 con 0x7fcfe0073a00 2024-04-24T08:32:15.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.145+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcfcc010380 con 0x7fcfe0073a00 2024-04-24T08:32:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.146+0000 7fcfde000700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 0x7fcfc8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.146+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcfcc08ed50 con 0x7fcfe0073a00 2024-04-24T08:32:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.147+0000 7fcfe5400700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 0x7fcfc8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.149+0000 7fcfe5400700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 0x7fcfc8071170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005f50 tx=0x7fcfd401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.161+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcfcc057be0 con 0x7fcfe0073a00 2024-04-24T08:32:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.441+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfe01017d0 con 0x7fcfc806ecc0 2024-04-24T08:32:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.443+0000 7fcfde000700 1 -- 172.21.15.90:0/289562109 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcfe01017d0 con 0x7fcfc806ecc0 2024-04-24T08:32:15.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 msgr2=0x7fcfc8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 0x7fcfc8071170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005f50 tx=0x7fcfd401a040 comp rx=0 tx=0).stop 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 msgr2=0x7fcfe0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b4b0 tx=0x7fcfcc00b7c0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 shutdown_connections 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcfc806ecc0 0x7fcfc8071170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfe00730f0 0x7fcfe01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 --2- 172.21.15.90:0/289562109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfe0073a00 0x7fcfe0100720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.445+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 >> 172.21.15.90:0/289562109 conn(0x7fcfe00fbda0 msgr2=0x7fcfe0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.446+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 shutdown_connections 2024-04-24T08:32:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.446+0000 7fcfe7e00700 1 -- 172.21.15.90:0/289562109 wait complete. 2024-04-24T08:32:15.463 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.626+0000 7fc793e00700 1 -- 172.21.15.90:0/2812103267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 msgr2=0x7fc78c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.626+0000 7fc793e00700 1 --2- 172.21.15.90:0/2812103267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c073480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc780009940 tx=0x7fc780009c50 comp rx=0 tx=0).stop 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 -- 172.21.15.90:0/2812103267 shutdown_connections 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 --2- 172.21.15.90:0/2812103267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 0x7fc78c10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 --2- 172.21.15.90:0/2812103267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c073480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 -- 172.21.15.90:0/2812103267 >> 172.21.15.90:0/2812103267 conn(0x7fc78c0fbda0 msgr2=0x7fc78c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:15.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 -- 172.21.15.90:0/2812103267 shutdown_connections 2024-04-24T08:32:15.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.627+0000 7fc793e00700 1 -- 172.21.15.90:0/2812103267 wait complete. 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.628+0000 7fc793e00700 1 Processor -- start 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.628+0000 7fc793e00700 1 -- start start 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.628+0000 7fc793e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc793e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 0x7fc78c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc793e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c072d00 con 0x7fc78c0730b0 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc793e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c06e5e0 con 0x7fc78c0739c0 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc791400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc791400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50676/0 (socket says 172.21.15.90:50676) 2024-04-24T08:32:15.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc791400700 1 -- 172.21.15.90:0/85849274 learned_addr learned my addr 172.21.15.90:0/85849274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:15.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc790a00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 0x7fc78c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc791400700 1 -- 172.21.15.90:0/85849274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 msgr2=0x7fc78c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.629+0000 7fc791400700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 0x7fc78c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc791400700 1 -- 172.21.15.90:0/85849274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800095f0 con 0x7fc78c0730b0 2024-04-24T08:32:15.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc791400700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc780005620 tx=0x7fc780005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:15.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78001d070 con 0x7fc78c0730b0 2024-04-24T08:32:15.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc78000f460 con 0x7fc78c0730b0 2024-04-24T08:32:15.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c06e860 con 0x7fc78c0730b0 2024-04-24T08:32:15.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.630+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc780021640 con 0x7fc78c0730b0 2024-04-24T08:32:15.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.631+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c1aae60 con 0x7fc78c0730b0 2024-04-24T08:32:15.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.632+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c109cc0 con 0x7fc78c0730b0 2024-04-24T08:32:15.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.634+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc78000fa40 con 0x7fc78c0730b0 2024-04-24T08:32:15.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.635+0000 7fc78a000700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 0x7fc774071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:15.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.635+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc780090120 con 0x7fc78c0730b0 2024-04-24T08:32:15.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.636+0000 7fc790a00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 0x7fc774071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:15.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.636+0000 7fc790a00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 0x7fc774071170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc778005fd0 tx=0x7fc77800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:15.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.641+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc780058f10 con 0x7fc78c0730b0 2024-04-24T08:32:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.913+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc78c06f170 con 0x7fc77406ecc0 2024-04-24T08:32:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.915+0000 7fc78a000700 1 -- 172.21.15.90:0/85849274 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc78c06f170 con 0x7fc77406ecc0 2024-04-24T08:32:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 msgr2=0x7fc774071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 0x7fc774071170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc778005fd0 tx=0x7fc77800a560 comp rx=0 tx=0).stop 2024-04-24T08:32:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 msgr2=0x7fc78c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc780005620 tx=0x7fc780005700 comp rx=0 tx=0).stop 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 shutdown_connections 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc77406ecc0 0x7fc774071170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78c0730b0 0x7fc78c06db60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 --2- 172.21.15.90:0/85849274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc78c0739c0 0x7fc78c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 >> 172.21.15.90:0/85849274 conn(0x7fc78c0fbda0 msgr2=0x7fc78c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 shutdown_connections 2024-04-24T08:32:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:15.918+0000 7fc793e00700 1 -- 172.21.15.90:0/85849274 wait complete. 2024-04-24T08:32:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 -- 172.21.15.90:0/2024554817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc0687a0 msgr2=0x7f80fc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 --2- 172.21.15.90:0/2024554817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc0687a0 0x7f80fc068b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009910 tx=0x7f80f0009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 -- 172.21.15.90:0/2024554817 shutdown_connections 2024-04-24T08:32:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 --2- 172.21.15.90:0/2024554817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc069140 0x7f80fc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 --2- 172.21.15.90:0/2024554817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc0687a0 0x7f80fc068b70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 -- 172.21.15.90:0/2024554817 >> 172.21.15.90:0/2024554817 conn(0x7f80fc075130 msgr2=0x7f80fc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 -- 172.21.15.90:0/2024554817 shutdown_connections 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.062+0000 7f8104800700 1 -- 172.21.15.90:0/2024554817 wait complete. 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.063+0000 7f8104800700 1 Processor -- start 2024-04-24T08:32:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8104800700 1 -- start start 2024-04-24T08:32:16.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8104800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc0687a0 0x7f80fc195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8104800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8104800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc196b80 con 0x7f80fc069140 2024-04-24T08:32:16.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8104800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc19a910 con 0x7f80fc0687a0 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.064+0000 7f8101400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50690/0 (socket says 172.21.15.90:50690) 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 -- 172.21.15.90:0/1827683074 learned_addr learned my addr 172.21.15.90:0/1827683074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101e00700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc0687a0 0x7f80fc195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 -- 172.21.15.90:0/1827683074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc0687a0 msgr2=0x7f80fc195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc0687a0 0x7f80fc195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 -- 172.21.15.90:0/1827683074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00095f0 con 0x7f80fc069140 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.065+0000 7f8101400700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00b480 tx=0x7f80ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.066+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80ec0118c0 con 0x7f80fc069140 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.066+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80ec010380 con 0x7f80fc069140 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.066+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80ec00f4f0 con 0x7f80fc069140 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.066+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc19abf0 con 0x7f80fc069140 2024-04-24T08:32:16.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.066+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc19b140 con 0x7f80fc069140 2024-04-24T08:32:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.067+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc108770 con 0x7f80fc069140 2024-04-24T08:32:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.068+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80ec0104f0 con 0x7f80fc069140 2024-04-24T08:32:16.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.069+0000 7f80f6a00700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 0x7f80e4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.069+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f80ec08ea10 con 0x7f80fc069140 2024-04-24T08:32:16.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.069+0000 7f8101e00700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 0x7f80e4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.070+0000 7f8101e00700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 0x7f80e4071170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006010 tx=0x7f80f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:16.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.074+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f80ec057800 con 0x7f80fc069140 2024-04-24T08:32:16.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.384+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80fc19b410 con 0x7f80e406ecc0 2024-04-24T08:32:16.386 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:16 smithi090 ceph-mon[85282]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:16.386 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:16.386 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:16.386 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:16 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:16.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.406+0000 7f80f6a00700 1 -- 172.21.15.90:0/1827683074 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f80fc19b410 con 0x7f80e406ecc0 2024-04-24T08:32:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:32:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 42s ago 19m 13.7M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (19m) 42s ago 19m 8404k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (18m) 77s ago 18m 8284k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (90s) 42s ago 19m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (82s) 77s ago 18m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (4m) 42s ago 18m 73.6M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (11m) 42s ago 11m 87.0M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (12m) 42s ago 12m 17.5M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (12m) 77s ago 12m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (11m) 77s ago 11m 17.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (8m) 42s ago 21m 502M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (6m) 77s ago 17m 450M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (3m) 42s ago 21m 47.1M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (2m) 77s ago 17m 37.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (6m) 42s ago 19m 7684k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:32:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 77s ago 17m 7948k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (50s) 42s ago 16m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 starting - - - 1216M 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (15m) 42s ago 15m 190M 1216M 18.2.0 dc2bc1663786 60c6cb43f002 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (15m) 77s ago 15m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (14m) 77s ago 14m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (13m) 77s ago 13m 204M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:32:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 42s ago 18m 66.1M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 msgr2=0x7f80e4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 0x7f80e4071170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006010 tx=0x7f80f001a040 comp rx=0 tx=0).stop 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 msgr2=0x7f80fc1964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00b480 tx=0x7f80ec00b840 comp rx=0 tx=0).stop 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 shutdown_connections 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f80e406ecc0 0x7f80e4071170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80fc0687a0 0x7f80fc195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 --2- 172.21.15.90:0/1827683074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80fc069140 0x7f80fc1964a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 >> 172.21.15.90:0/1827683074 conn(0x7f80fc075130 msgr2=0x7f80fc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:16.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 shutdown_connections 2024-04-24T08:32:16.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.409+0000 7f8104800700 1 -- 172.21.15.90:0/1827683074 wait complete. 2024-04-24T08:32:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:16 smithi149 ceph-mon[66490]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:16 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.558+0000 7f20d6000700 1 -- 172.21.15.90:0/1322736050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 msgr2=0x7f20d0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 --2- 172.21.15.90:0/1322736050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0068b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009910 tx=0x7f20c0009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 -- 172.21.15.90:0/1322736050 shutdown_connections 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 --2- 172.21.15.90:0/1322736050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 --2- 172.21.15.90:0/1322736050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0068b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 -- 172.21.15.90:0/1322736050 >> 172.21.15.90:0/1322736050 conn(0x7f20d0075130 msgr2=0x7f20d0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 -- 172.21.15.90:0/1322736050 shutdown_connections 2024-04-24T08:32:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.559+0000 7f20d6000700 1 -- 172.21.15.90:0/1322736050 wait complete. 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 Processor -- start 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 -- start start 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d01964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0196b80 con 0x7f20d00687a0 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.560+0000 7f20d6000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d019a910 con 0x7f20d0069140 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50708/0 (socket says 172.21.15.90:50708) 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 -- 172.21.15.90:0/2120385658 learned_addr learned my addr 172.21.15.90:0/2120385658 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cea00700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d01964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 -- 172.21.15.90:0/2120385658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 msgr2=0x7f20d01964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d01964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 -- 172.21.15.90:0/2120385658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00095f0 con 0x7f20d00687a0 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cea00700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d01964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:32:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.561+0000 7f20cf400700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.562+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c001d070 con 0x7f20d00687a0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.562+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20c0004fb0 con 0x7f20d00687a0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.562+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c0022ea0 con 0x7f20d00687a0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.562+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d019ab90 con 0x7f20d00687a0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.562+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d019b080 con 0x7f20d00687a0 2024-04-24T08:32:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.563+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d0108790 con 0x7f20d00687a0 2024-04-24T08:32:16.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.564+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f20c0022470 con 0x7f20d00687a0 2024-04-24T08:32:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.565+0000 7f20c7e00700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 0x7f20b4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.565+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f20c0090360 con 0x7f20d00687a0 2024-04-24T08:32:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.565+0000 7f20cea00700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 0x7f20b4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.566+0000 7f20cea00700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 0x7f20b4071170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f20d0197580 tx=0x7f20bc008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:16.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.570+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f20c0059200 con 0x7f20d00687a0 2024-04-24T08:32:16.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.887+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20d019b3b0 con 0x7f20d00687a0 2024-04-24T08:32:16.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.888+0000 7f20c7e00700 1 -- 172.21.15.90:0/2120385658 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f20c005c8c0 con 0x7f20d00687a0 2024-04-24T08:32:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:32:16.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:32:16.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:32:16.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:32:16.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-04-24T08:32:16.891 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 5 2024-04-24T08:32:16.891 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:32:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 msgr2=0x7f20b4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 0x7f20b4071170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f20d0197580 tx=0x7f20bc008040 comp rx=0 tx=0).stop 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 msgr2=0x7f20d0195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c000be20 comp rx=0 tx=0).stop 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 shutdown_connections 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f20b406ecc0 0x7f20b4071170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00687a0 0x7f20d0195f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 --2- 172.21.15.90:0/2120385658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20d0069140 0x7f20d01964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 >> 172.21.15.90:0/2120385658 conn(0x7f20d0075130 msgr2=0x7f20d00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 shutdown_connections 2024-04-24T08:32:16.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:16.891+0000 7f20d6000700 1 -- 172.21.15.90:0/2120385658 wait complete. 2024-04-24T08:32:17.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.039+0000 7f741bc00700 1 -- 172.21.15.90:0/679567086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 msgr2=0x7f74140734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.039+0000 7f741bc00700 1 --2- 172.21.15.90:0/679567086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74140734c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7408009910 tx=0x7f7408009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:17.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 -- 172.21.15.90:0/679567086 shutdown_connections 2024-04-24T08:32:17.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 --2- 172.21.15.90:0/679567086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 0x7f741410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 --2- 172.21.15.90:0/679567086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74140734c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 -- 172.21.15.90:0/679567086 >> 172.21.15.90:0/679567086 conn(0x7f74140fbda0 msgr2=0x7f74140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 -- 172.21.15.90:0/679567086 shutdown_connections 2024-04-24T08:32:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.040+0000 7f741bc00700 1 -- 172.21.15.90:0/679567086 wait complete. 2024-04-24T08:32:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.041+0000 7f741bc00700 1 Processor -- start 2024-04-24T08:32:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f741bc00700 1 -- start start 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f741bc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f741bc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 0x7f7414198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f741bc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741419cb10 con 0x7f74140730f0 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f741bc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741419cc80 con 0x7f7414073a00 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f7419200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f7419200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50714/0 (socket says 172.21.15.90:50714) 2024-04-24T08:32:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.042+0000 7f7419200700 1 -- 172.21.15.90:0/606493466 learned_addr learned my addr 172.21.15.90:0/606493466 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7413e00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 0x7f7414198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7419200700 1 -- 172.21.15.90:0/606493466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 msgr2=0x7f7414198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7419200700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 0x7f7414198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7419200700 1 -- 172.21.15.90:0/606493466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080095f0 con 0x7f74140730f0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7419200700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f740800b5c0 tx=0x7f740800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.043+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740801d070 con 0x7f74140730f0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.044+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f740800f460 con 0x7f74140730f0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.044+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741419cf60 con 0x7f74140730f0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.044+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7408017630 con 0x7f74140730f0 2024-04-24T08:32:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.044+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741419d4b0 con 0x7f74140730f0 2024-04-24T08:32:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.045+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7414109ce0 con 0x7f74140730f0 2024-04-24T08:32:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.046+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f740800f5d0 con 0x7f74140730f0 2024-04-24T08:32:17.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.047+0000 7f7411600700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 0x7f73fc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.047+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7408090430 con 0x7f74140730f0 2024-04-24T08:32:17.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.047+0000 7f7413e00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 0x7f73fc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.048+0000 7f7413e00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 0x7f73fc071170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7400006fd0 tx=0x7f7400008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.052+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f74080591a0 con 0x7f74140730f0 2024-04-24T08:32:17.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:17 smithi090 ceph-mon[85282]: from='client.44171 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:17.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:17 smithi090 ceph-mon[85282]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:17.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:17 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2120385658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:17.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.356+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74141994c0 con 0x7f74140730f0 2024-04-24T08:32:17.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.357+0000 7f7411600700 1 -- 172.21.15.90:0/606493466 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f740805c860 con 0x7f74140730f0 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:17.358 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:32:17.359 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:32:17.360 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:17.361 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 msgr2=0x7f73fc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 0x7f73fc071170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7400006fd0 tx=0x7f7400008040 comp rx=0 tx=0).stop 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 msgr2=0x7f74141981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f740800b5c0 tx=0x7f740800be00 comp rx=0 tx=0).stop 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 shutdown_connections 2024-04-24T08:32:17.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f73fc06ecc0 0x7f73fc071170 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74140730f0 0x7f74141981b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 --2- 172.21.15.90:0/606493466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7414073a00 0x7f7414198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 >> 172.21.15.90:0/606493466 conn(0x7f74140fbda0 msgr2=0x7f7414106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:17.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 shutdown_connections 2024-04-24T08:32:17.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.361+0000 7f741bc00700 1 -- 172.21.15.90:0/606493466 wait complete. 2024-04-24T08:32:17.364 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:32:17.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:17 smithi149 ceph-mon[66490]: from='client.44171 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:17.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:17 smithi149 ceph-mon[66490]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:17.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:17 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2120385658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 -- 172.21.15.90:0/2881307683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868100e80 msgr2=0x7fd868104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 --2- 172.21.15.90:0/2881307683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868100e80 0x7fd868104e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd854009910 tx=0x7fd854009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 -- 172.21.15.90:0/2881307683 shutdown_connections 2024-04-24T08:32:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 --2- 172.21.15.90:0/2881307683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868100e80 0x7fd868104e50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 --2- 172.21.15.90:0/2881307683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8681004e0 0x7fd8681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 -- 172.21.15.90:0/2881307683 >> 172.21.15.90:0/2881307683 conn(0x7fd8680fbda0 msgr2=0x7fd8680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 -- 172.21.15.90:0/2881307683 shutdown_connections 2024-04-24T08:32:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.537+0000 7fd86d200700 1 -- 172.21.15.90:0/2881307683 wait complete. 2024-04-24T08:32:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 Processor -- start 2024-04-24T08:32:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 -- start start 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 0x7fd868073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868074d70 con 0x7fd8681004e0 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.539+0000 7fd86d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868074ee0 con 0x7fd868100e80 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.540+0000 7fd865e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.540+0000 7fd865e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50732/0 (socket says 172.21.15.90:50732) 2024-04-24T08:32:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.540+0000 7fd865e00700 1 -- 172.21.15.90:0/2338557723 learned_addr learned my addr 172.21.15.90:0/2338557723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.540+0000 7fd865400700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 0x7fd868073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd865e00700 1 -- 172.21.15.90:0/2338557723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 msgr2=0x7fd868073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd865e00700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 0x7fd868073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd865e00700 1 -- 172.21.15.90:0/2338557723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540095f0 con 0x7fd8681004e0 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd865400700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 0x7fd868073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd865e00700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c930 tx=0x7fd85800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd858007ad0 con 0x7fd8681004e0 2024-04-24T08:32:17.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd858007c30 con 0x7fd8681004e0 2024-04-24T08:32:17.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd868073b30 con 0x7fd8681004e0 2024-04-24T08:32:17.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.541+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd858018610 con 0x7fd8681004e0 2024-04-24T08:32:17.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.542+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8681a68b0 con 0x7fd8681004e0 2024-04-24T08:32:17.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.543+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd868108850 con 0x7fd8681004e0 2024-04-24T08:32:17.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.544+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd85800f450 con 0x7fd8681004e0 2024-04-24T08:32:17.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.545+0000 7fd85ea00700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 0x7fd84c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.545+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd85808f020 con 0x7fd8681004e0 2024-04-24T08:32:17.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.545+0000 7fd865400700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 0x7fd84c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.546+0000 7fd865400700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 0x7fd84c071170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8680fc560 tx=0x7fd85401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.550+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd858057e30 con 0x7fd8681004e0 2024-04-24T08:32:17.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.812+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8680746c0 con 0x7fd84c06ecc0 2024-04-24T08:32:17.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.814+0000 7fd85ea00700 1 -- 172.21.15.90:0/2338557723 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd8680746c0 con 0x7fd84c06ecc0 2024-04-24T08:32:17.815 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:32:17.815 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "7/23 daemons upgraded", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:32:17.816 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:32:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 msgr2=0x7fd84c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 0x7fd84c071170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8680fc560 tx=0x7fd85401a040 comp rx=0 tx=0).stop 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 msgr2=0x7fd868073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c930 tx=0x7fd85800ccf0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 shutdown_connections 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd84c06ecc0 0x7fd84c071170 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8681004e0 0x7fd868073050 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 --2- 172.21.15.90:0/2338557723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd868100e80 0x7fd868073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.817+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 >> 172.21.15.90:0/2338557723 conn(0x7fd8680fbda0 msgr2=0x7fd8680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.818+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 shutdown_connections 2024-04-24T08:32:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.818+0000 7fd86d200700 1 -- 172.21.15.90:0/2338557723 wait complete. 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 -- 172.21.15.90:0/2450881887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 msgr2=0x7f71b010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 --2- 172.21.15.90:0/2450881887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b010a6d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f719c009910 tx=0x7f719c009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 -- 172.21.15.90:0/2450881887 shutdown_connections 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 --2- 172.21.15.90:0/2450881887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b010a6d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 --2- 172.21.15.90:0/2450881887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b00ff350 0x7f71b00ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 -- 172.21.15.90:0/2450881887 >> 172.21.15.90:0/2450881887 conn(0x7f71b00748b0 msgr2=0x7f71b0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:17.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.970+0000 7f71b6200700 1 -- 172.21.15.90:0/2450881887 shutdown_connections 2024-04-24T08:32:17.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.971+0000 7f71b6200700 1 -- 172.21.15.90:0/2450881887 wait complete. 2024-04-24T08:32:17.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.972+0000 7f71b6200700 1 Processor -- start 2024-04-24T08:32:17.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.972+0000 7f71b6200700 1 -- start start 2024-04-24T08:32:17.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71b6200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b00ff350 0x7f71b0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71b6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71b6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0198c80 con 0x7f71b00ffc60 2024-04-24T08:32:17.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71b6200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019ca10 con 0x7f71b00ff350 2024-04-24T08:32:17.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50754/0 (socket says 172.21.15.90:50754) 2024-04-24T08:32:17.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.973+0000 7f71aea00700 1 -- 172.21.15.90:0/2791889128 learned_addr learned my addr 172.21.15.90:0/2791889128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:17.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.974+0000 7f71aea00700 1 -- 172.21.15.90:0/2791889128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b00ff350 msgr2=0x7f71b0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:17.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.974+0000 7f71aea00700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b00ff350 0x7f71b0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:17.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.974+0000 7f71aea00700 1 -- 172.21.15.90:0/2791889128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0095f0 con 0x7f71b00ffc60 2024-04-24T08:32:17.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.974+0000 7f71aea00700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f719c006010 tx=0x7f719c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.975+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719c01d070 con 0x7f71b00ffc60 2024-04-24T08:32:17.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.975+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f719c00f460 con 0x7f71b00ffc60 2024-04-24T08:32:17.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.975+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719c017630 con 0x7f71b00ffc60 2024-04-24T08:32:17.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.975+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b019cc90 con 0x7f71b00ffc60 2024-04-24T08:32:17.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.975+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b019d130 con 0x7f71b00ffc60 2024-04-24T08:32:17.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.976+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b010a230 con 0x7f71b00ffc60 2024-04-24T08:32:17.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.978+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f719c00f5d0 con 0x7f71b00ffc60 2024-04-24T08:32:17.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.979+0000 7f71a7e00700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 0x7f7194071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:17.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.979+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f719c090350 con 0x7f71b00ffc60 2024-04-24T08:32:17.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.979+0000 7f71af400700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 0x7f7194071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:17.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.980+0000 7f71af400700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 0x7f7194071170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71b006a640 tx=0x7f71a0009470 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:17.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:17.989+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f719c059140 con 0x7f71b00ffc60 2024-04-24T08:32:18.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:18 smithi090 ceph-mon[85282]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:18.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:18 smithi090 ceph-mon[85282]: pgmap v101: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:18.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:18 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/606493466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:32:18.293 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:18 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:32:18.105+0000 7f5bc9d6e7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-24T08:32:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.355+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f71b004f2a0 con 0x7f71b00ffc60 2024-04-24T08:32:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.356+0000 7f71a7e00700 1 -- 172.21.15.90:0/2791889128 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1182 (secure 0 0 0) 0x7f719c05c800 con 0x7f71b00ffc60 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: osd.1 (root=default,host=smithi090) is down 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-24T08:32:18.357 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-24T08:32:18.358 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 msgr2=0x7f7194071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 0x7f7194071170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71b006a640 tx=0x7f71a0009470 comp rx=0 tx=0).stop 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 msgr2=0x7f71b01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f719c006010 tx=0x7f719c00be20 comp rx=0 tx=0).stop 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 shutdown_connections 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f719406ecc0 0x7f7194071170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b00ff350 0x7f71b0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 --2- 172.21.15.90:0/2791889128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71b00ffc60 0x7f71b01985a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 >> 172.21.15.90:0/2791889128 conn(0x7f71b00748b0 msgr2=0x7f71b0100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 shutdown_connections 2024-04-24T08:32:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:18.359+0000 7f71b6200700 1 -- 172.21.15.90:0/2791889128 wait complete. 2024-04-24T08:32:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:18 smithi149 ceph-mon[66490]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:18 smithi149 ceph-mon[66490]: pgmap v101: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:18.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:18 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/606493466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:32:18.793 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:18 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:32:18.505+0000 7f5bc9d6e7c0 -1 osd.1 55 log_to_monitors true 2024-04-24T08:32:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:19 smithi149 ceph-mon[66490]: from='client.34298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:19 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2791889128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:32:19.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:19 smithi149 ceph-mon[66490]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:32:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:19 smithi090 ceph-mon[85282]: from='client.34298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:19.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:19 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2791889128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:32:19.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:19 smithi090 ceph-mon[85282]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:32:19.544 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:32:19 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:32:19.178+0000 7f5bb9000700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:32:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:20 smithi149 ceph-mon[66490]: pgmap v102: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:20 smithi149 ceph-mon[66490]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:32:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:20 smithi149 ceph-mon[66490]: osdmap e58: 6 total, 5 up, 6 in 2024-04-24T08:32:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:20 smithi149 ceph-mon[66490]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:32:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:20 smithi090 ceph-mon[85282]: pgmap v102: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:20 smithi090 ceph-mon[85282]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:32:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:20 smithi090 ceph-mon[85282]: osdmap e58: 6 total, 5 up, 6 in 2024-04-24T08:32:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:20 smithi090 ceph-mon[85282]: from='osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546] boot 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:21.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:21 smithi090 ceph-mon[85282]: osdmap e60: 6 total, 6 up, 6 in 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: osd.1 [v2:172.21.15.90:6810/1858244546,v1:172.21.15.90:6811/1858244546] boot 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:21 smithi149 ceph-mon[66490]: osdmap e60: 6 total, 6 up, 6 in 2024-04-24T08:32:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:22 smithi149 ceph-mon[66490]: pgmap v105: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 18/267 objects degraded (6.742%) 2024-04-24T08:32:22.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:22 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:22.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:22 smithi090 ceph-mon[85282]: pgmap v105: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 18/267 objects degraded (6.742%) 2024-04-24T08:32:22.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:22 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:23.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:23.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:23.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:23 smithi090 ceph-mon[85282]: pgmap v107: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/267 objects degraded (6.742%) 2024-04-24T08:32:24.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:23 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:24.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:23 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:23 smithi149 ceph-mon[66490]: pgmap v107: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/267 objects degraded (6.742%) 2024-04-24T08:32:25.488 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:25 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-24T08:32:25.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:25 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-24T08:32:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:26 smithi149 ceph-mon[66490]: pgmap v108: 65 pgs: 20 peering, 45 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:32:26.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:26 smithi090 ceph-mon[85282]: pgmap v108: 65 pgs: 20 peering, 45 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:32:28.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:28 smithi149 ceph-mon[66490]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-24T08:32:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:28 smithi090 ceph-mon[85282]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-24T08:32:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: Upgrade: osd.2 is safe to restart 2024-04-24T08:32:29.528 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:29 smithi149 ceph-mon[66490]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-24T08:32:29.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:29.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:32:29.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: Upgrade: osd.2 is safe to restart 2024-04-24T08:32:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:29 smithi090 ceph-mon[85282]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-24T08:32:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:32:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:32:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:32:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:32 smithi149 ceph-mon[66490]: Upgrade: Updating osd.2 2024-04-24T08:32:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:32 smithi149 ceph-mon[66490]: Deploying daemon osd.2 on smithi090 2024-04-24T08:32:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:32 smithi149 ceph-mon[66490]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:32:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:32 smithi090 ceph-mon[85282]: Upgrade: Updating osd.2 2024-04-24T08:32:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:32 smithi090 ceph-mon[85282]: Deploying daemon osd.2 on smithi090 2024-04-24T08:32:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:32 smithi090 ceph-mon[85282]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:32:34.462 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:34 smithi090 ceph-mon[85282]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 857 B/s rd, 1 op/s 2024-04-24T08:32:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:34 smithi149 ceph-mon[66490]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 857 B/s rd, 1 op/s 2024-04-24T08:32:36.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:35 smithi090 systemd[1]: Stopping Ceph osd.2 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:32:36.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:36 smithi149 ceph-mon[66490]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:32:36.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:36.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:36 smithi149 ceph-mon[66490]: osd.2 marked itself down and dead 2024-04-24T08:32:36.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[50927]: 2024-04-24T08:32:36.170+0000 7f77b1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:32:36.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[50927]: 2024-04-24T08:32:36.170+0000 7f77b1400700 -1 osd.2 60 *** Got signal Terminated *** 2024-04-24T08:32:36.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[50927]: 2024-04-24T08:32:36.170+0000 7f77b1400700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:32:36.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-mon[85282]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:32:36.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:36.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:36 smithi090 ceph-mon[85282]: osd.2 marked itself down and dead 2024-04-24T08:32:37.241 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:36 smithi090 podman[107639]: 2024-04-24 08:32:36.872135387 +0000 UTC m=+0.849495690 container died 60c6cb43f002c10b0597167be52877de798d56689f37c7a494f83e3c4ea8f33b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T08:32:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:37 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:32:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:37 smithi149 ceph-mon[66490]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T08:32:37.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:37 smithi149 ceph-mon[66490]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T08:32:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:37 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:32:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:37 smithi090 ceph-mon[85282]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T08:32:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:37 smithi090 ceph-mon[85282]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T08:32:38.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:37 smithi090 podman[107639]: 2024-04-24 08:32:37.641859282 +0000 UTC m=+1.619219607 container cleanup 60c6cb43f002c10b0597167be52877de798d56689f37c7a494f83e3c4ea8f33b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-24T08:32:38.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:37 smithi090 bash[107639]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2 2024-04-24T08:32:38.043 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:37 smithi090 podman[107660]: 2024-04-24 08:32:37.851884957 +0000 UTC m=+0.939302408 container remove 60c6cb43f002c10b0597167be52877de798d56689f37c7a494f83e3c4ea8f33b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-24T08:32:38.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:38 smithi149 ceph-mon[66490]: pgmap v115: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:32:38.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:38 smithi090 ceph-mon[85282]: pgmap v115: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:32:39.001 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:38 smithi090 podman[107751]: 2024-04-24 08:32:38.571529066 +0000 UTC m=+0.033227141 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:39.001 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:38 smithi090 podman[107751]: 2024-04-24 08:32:38.694986769 +0000 UTC m=+0.156684877 container create 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T08:32:39.001 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:39 smithi090 podman[107751]: 2024-04-24 08:32:38.999584692 +0000 UTC m=+0.461282807 container init 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T08:32:39.254 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:39 smithi090 podman[107751]: 2024-04-24 08:32:39.011641762 +0000 UTC m=+0.473339877 container start 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T08:32:39.254 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:39 smithi090 podman[107751]: 2024-04-24 08:32:39.114813646 +0000 UTC m=+0.576511738 container attach 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T08:32:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:39 smithi149 ceph-mon[66490]: pgmap v117: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 4 active+undersized, 51 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 10/267 objects degraded (3.745%) 2024-04-24T08:32:39.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:39 smithi090 podman[107751]: 2024-04-24 08:32:39.461054455 +0000 UTC m=+0.922752527 container died 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T08:32:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:39 smithi090 ceph-mon[85282]: pgmap v117: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 4 active+undersized, 51 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 10/267 objects degraded (3.745%) 2024-04-24T08:32:40.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:40 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 4 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:40.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:40 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 4 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:40.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 podman[107776]: 2024-04-24 08:32:40.162243688 +0000 UTC m=+0.677341589 container remove 028d2a5e69c44dfb40b84e8c1a15020757a089a0521cd5eac2478f2ea9de4636 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T08:32:40.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-24T08:32:40.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 systemd[1]: Stopped Ceph osd.2 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:32:40.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.2.service: Consumed 1min 36.443s CPU time. 2024-04-24T08:32:40.956 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 systemd[1]: Starting Ceph osd.2 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:32:41.211 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 podman[107877]: 2024-04-24 08:32:40.955125699 +0000 UTC m=+0.124052383 container create 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T08:32:41.211 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:40 smithi090 podman[107877]: 2024-04-24 08:32:40.863235719 +0000 UTC m=+0.032162378 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:41.211 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:41 smithi090 podman[107877]: 2024-04-24 08:32:41.209516922 +0000 UTC m=+0.378443649 container init 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T08:32:41.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:41 smithi149 ceph-mon[66490]: pgmap v118: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:41.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:41 smithi090 ceph-mon[85282]: pgmap v118: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:41.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:41 smithi090 podman[107877]: 2024-04-24 08:32:41.22215211 +0000 UTC m=+0.391078824 container start 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T08:32:41.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:41 smithi090 podman[107877]: 2024-04-24 08:32:41.350026006 +0000 UTC m=+0.518952757 container attach 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:32:42.544 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:42.544 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:42.544 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:42.544 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:32:42.544 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 bash[107877]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate[107891]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T08:32:42.545 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:42 smithi090 podman[107877]: 2024-04-24 08:32:42.194150995 +0000 UTC m=+1.363077701 container died 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T08:32:43.484 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 podman[108036]: 2024-04-24 08:32:43.131055731 +0000 UTC m=+0.915846992 container remove 1390daff03de5a081b949e9f116f1993f4086800f7fef3505ac35b0e74ad7744 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T08:32:43.793 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 podman[108100]: 2024-04-24 08:32:43.388815247 +0000 UTC m=+0.033002651 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:32:43.793 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 podman[108100]: 2024-04-24 08:32:43.498777993 +0000 UTC m=+0.142965462 container create 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:32:44.164 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 podman[108100]: 2024-04-24 08:32:43.812569977 +0000 UTC m=+0.456757410 container init 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T08:32:44.164 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 podman[108100]: 2024-04-24 08:32:43.823151682 +0000 UTC m=+0.467339091 container start 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T08:32:44.164 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 ceph-osd[108119]: -- 172.21.15.90:0/388973507 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56503a061680 con 0x56503a02b000 2024-04-24T08:32:44.164 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 bash[108100]: 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 2024-04-24T08:32:44.164 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:43 smithi090 systemd[1]: Started Ceph osd.2 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:32:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:44 smithi149 ceph-mon[66490]: pgmap v119: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:44.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:44 smithi090 ceph-mon[85282]: pgmap v119: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:32:44.543 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:44 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:32:44.390+0000 7f310055e7c0 -1 Falling back to public interface 2024-04-24T08:32:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:46 smithi149 ceph-mon[66490]: pgmap v120: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 344 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:46 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:46 smithi090 ceph-mon[85282]: pgmap v120: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 344 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:32:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:46 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:48.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:48 smithi149 ceph-mon[66490]: pgmap v121: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:48.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:48 smithi090 ceph-mon[85282]: pgmap v121: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 0 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 -- 172.21.15.90:0/1918299050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 msgr2=0x7f6468104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 --2- 172.21.15.90:0/1918299050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468104e50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6454009910 tx=0x7f6454009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 -- 172.21.15.90:0/1918299050 shutdown_connections 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 --2- 172.21.15.90:0/1918299050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468104e50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 --2- 172.21.15.90:0/1918299050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 0x7f64681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 -- 172.21.15.90:0/1918299050 >> 172.21.15.90:0/1918299050 conn(0x7f64680fbda0 msgr2=0x7f64680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 -- 172.21.15.90:0/1918299050 shutdown_connections 2024-04-24T08:32:48.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.561+0000 7f646ee00700 1 -- 172.21.15.90:0/1918299050 wait complete. 2024-04-24T08:32:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.562+0000 7f646ee00700 1 Processor -- start 2024-04-24T08:32:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f646ee00700 1 -- start start 2024-04-24T08:32:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f646ee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 0x7f6468198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f646ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f646ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468198d50 con 0x7f6468100e80 2024-04-24T08:32:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f646ee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646819cae0 con 0x7f64681004e0 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f6467400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f6467400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50568/0 (socket says 172.21.15.90:50568) 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.563+0000 7f6467400700 1 -- 172.21.15.90:0/2546105873 learned_addr learned my addr 172.21.15.90:0/2546105873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.564+0000 7f6467e00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 0x7f6468198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.564+0000 7f6467400700 1 -- 172.21.15.90:0/2546105873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 msgr2=0x7f6468198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.564+0000 7f6467400700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 0x7f6468198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.564+0000 7f6467400700 1 -- 172.21.15.90:0/2546105873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540095f0 con 0x7f6468100e80 2024-04-24T08:32:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.564+0000 7f6467400700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f645400bd70 tx=0x7f645400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.565+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f645401d070 con 0x7f6468100e80 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.565+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6454005dc0 con 0x7f6468100e80 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.565+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646819cd60 con 0x7f6468100e80 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.565+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6454017400 con 0x7f6468100e80 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.565+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646819d250 con 0x7f6468100e80 2024-04-24T08:32:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.566+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64681045d0 con 0x7f6468100e80 2024-04-24T08:32:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.568+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f645400f460 con 0x7f6468100e80 2024-04-24T08:32:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.569+0000 7f6464c00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 0x7f644c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:48.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.569+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6454090320 con 0x7f6468100e80 2024-04-24T08:32:48.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.569+0000 7f6467e00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 0x7f644c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:48.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.570+0000 7f6467e00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 0x7f644c071170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6458005950 tx=0x7f645800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.575+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6454059090 con 0x7f6468100e80 2024-04-24T08:32:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.839+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f646819d530 con 0x7f644c06ecc0 2024-04-24T08:32:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.841+0000 7f6464c00700 1 -- 172.21.15.90:0/2546105873 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f646819d530 con 0x7f644c06ecc0 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 msgr2=0x7f644c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 0x7f644c071170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6458005950 tx=0x7f645800b410 comp rx=0 tx=0).stop 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 msgr2=0x7f6468198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f645400bd70 tx=0x7f645400bda0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 shutdown_connections 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f644c06ecc0 0x7f644c071170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64681004e0 0x7f6468198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 --2- 172.21.15.90:0/2546105873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468100e80 0x7f6468198670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 >> 172.21.15.90:0/2546105873 conn(0x7f64680fbda0 msgr2=0x7f64680fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 shutdown_connections 2024-04-24T08:32:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.844+0000 7f646ee00700 1 -- 172.21.15.90:0/2546105873 wait complete. 2024-04-24T08:32:48.861 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:32:48.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.994+0000 7fbb3b800700 1 -- 172.21.15.90:0/2247778257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb341004e0 msgr2=0x7fbb341008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.994+0000 7fbb3b800700 1 --2- 172.21.15.90:0/2247778257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb341004e0 0x7fbb341008b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009910 tx=0x7fbb28009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 -- 172.21.15.90:0/2247778257 shutdown_connections 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 --2- 172.21.15.90:0/2247778257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb34100e80 0x7fbb34104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 --2- 172.21.15.90:0/2247778257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb341004e0 0x7fbb341008b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 -- 172.21.15.90:0/2247778257 >> 172.21.15.90:0/2247778257 conn(0x7fbb340fbda0 msgr2=0x7fbb340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 -- 172.21.15.90:0/2247778257 shutdown_connections 2024-04-24T08:32:48.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.995+0000 7fbb3b800700 1 -- 172.21.15.90:0/2247778257 wait complete. 2024-04-24T08:32:48.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 Processor -- start 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 -- start start 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb341004e0 0x7fbb34195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34196b10 con 0x7fbb34100e80 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.996+0000 7fbb3b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419a8a0 con 0x7fbb341004e0 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50588/0 (socket says 172.21.15.90:50588) 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 -- 172.21.15.90:0/1796712031 learned_addr learned my addr 172.21.15.90:0/1796712031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb38e00700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb341004e0 0x7fbb34195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 -- 172.21.15.90:0/1796712031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb341004e0 msgr2=0x7fbb34195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb341004e0 0x7fbb34195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:48.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 -- 172.21.15.90:0/1796712031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280095f0 con 0x7fbb34100e80 2024-04-24T08:32:48.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.997+0000 7fbb33e00700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c8f0 tx=0x7fbb2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:48.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.998+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb20007a50 con 0x7fbb34100e80 2024-04-24T08:32:48.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.998+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3419ab80 con 0x7fbb34100e80 2024-04-24T08:32:48.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.998+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb20007bb0 con 0x7fbb34100e80 2024-04-24T08:32:49.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.998+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb20018630 con 0x7fbb34100e80 2024-04-24T08:32:49.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.998+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3419b0d0 con 0x7fbb34100e80 2024-04-24T08:32:49.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:48.999+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb341087f0 con 0x7fbb34100e80 2024-04-24T08:32:49.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.000+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbb20007d20 con 0x7fbb34100e80 2024-04-24T08:32:49.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.001+0000 7fbb31600700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 0x7fbb1c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.001+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb2008ee90 con 0x7fbb34100e80 2024-04-24T08:32:49.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.002+0000 7fbb38e00700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 0x7fbb1c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.003+0000 7fbb38e00700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 0x7fbb1c071170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006010 tx=0x7fbb2801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:49.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.007+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbb20057c00 con 0x7fbb34100e80 2024-04-24T08:32:49.293 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:48 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:32:48.974+0000 7f310055e7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-24T08:32:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.344+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb3419b3c0 con 0x7fbb1c06ecc0 2024-04-24T08:32:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.345+0000 7fbb31600700 1 -- 172.21.15.90:0/1796712031 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbb3419b3c0 con 0x7fbb1c06ecc0 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 msgr2=0x7fbb1c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 0x7fbb1c071170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006010 tx=0x7fbb2801a040 comp rx=0 tx=0).stop 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 msgr2=0x7fbb34196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c8f0 tx=0x7fbb2000ccb0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 shutdown_connections 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb1c06ecc0 0x7fbb1c071170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb341004e0 0x7fbb34195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 --2- 172.21.15.90:0/1796712031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb34100e80 0x7fbb34196430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 >> 172.21.15.90:0/1796712031 conn(0x7fbb340fbda0 msgr2=0x7fbb340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:49.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 shutdown_connections 2024-04-24T08:32:49.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.348+0000 7fbb3b800700 1 -- 172.21.15.90:0/1796712031 wait complete. 2024-04-24T08:32:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 -- 172.21.15.90:0/3766625797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c069140 msgr2=0x7feb4c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 --2- 172.21.15.90:0/3766625797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c069140 0x7feb4c104eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb34009910 tx=0x7feb34009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:49.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 -- 172.21.15.90:0/3766625797 shutdown_connections 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 --2- 172.21.15.90:0/3766625797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c069140 0x7feb4c104eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 --2- 172.21.15.90:0/3766625797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb4c0687a0 0x7feb4c068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 -- 172.21.15.90:0/3766625797 >> 172.21.15.90:0/3766625797 conn(0x7feb4c075130 msgr2=0x7feb4c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 -- 172.21.15.90:0/3766625797 shutdown_connections 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.495+0000 7feb52a00700 1 -- 172.21.15.90:0/3766625797 wait complete. 2024-04-24T08:32:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.496+0000 7feb52a00700 1 Processor -- start 2024-04-24T08:32:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.496+0000 7feb52a00700 1 -- start start 2024-04-24T08:32:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb52a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb52a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb4c069140 0x7feb4c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb52a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c198da0 con 0x7feb4c0687a0 2024-04-24T08:32:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb52a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c19cb30 con 0x7feb4c069140 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50626/0 (socket says 172.21.15.90:50626) 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.497+0000 7feb4be00700 1 -- 172.21.15.90:0/1836677195 learned_addr learned my addr 172.21.15.90:0/1836677195 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb4be00700 1 -- 172.21.15.90:0/1836677195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb4c069140 msgr2=0x7feb4c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb4be00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb4c069140 0x7feb4c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb4be00700 1 -- 172.21.15.90:0/1836677195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340095f0 con 0x7feb4c0687a0 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb4be00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d8d0 tx=0x7feb3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3c009940 con 0x7feb4c0687a0 2024-04-24T08:32:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb3c010460 con 0x7feb4c0687a0 2024-04-24T08:32:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c19ce10 con 0x7feb4c0687a0 2024-04-24T08:32:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3c00f5f0 con 0x7feb4c0687a0 2024-04-24T08:32:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.498+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c19d360 con 0x7feb4c0687a0 2024-04-24T08:32:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.500+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c04f2a0 con 0x7feb4c0687a0 2024-04-24T08:32:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.501+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb3c0105d0 con 0x7feb4c0687a0 2024-04-24T08:32:49.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.502+0000 7feb49600700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 0x7feb30071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.502+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feb3c08edb0 con 0x7feb4c0687a0 2024-04-24T08:32:49.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.502+0000 7feb43400700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 0x7feb30071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.503+0000 7feb43400700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 0x7feb30071170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb3400b5c0 tx=0x7feb34005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:49.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.507+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb3c057ba0 con 0x7feb4c0687a0 2024-04-24T08:32:49.717 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:49 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:32:49.332+0000 7f310055e7c0 -1 osd.2 60 log_to_monitors true 2024-04-24T08:32:49.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.761+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb4c19d6b0 con 0x7feb3006ecc0 2024-04-24T08:32:49.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.772+0000 7feb49600700 1 -- 172.21.15.90:0/1836677195 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7feb4c19d6b0 con 0x7feb3006ecc0 2024-04-24T08:32:49.774 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:32:49.774 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 29s ago 19m 13.8M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:32:49.774 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (20m) 29s ago 20m 8711k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:32:49.774 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (18m) 110s ago 18m 8284k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:32:49.774 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (2m) 29s ago 20m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (115s) 110s ago 18m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 29s ago 19m 73.2M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (12m) 29s ago 12m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (12m) 29s ago 12m 17.8M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (12m) 110s ago 12m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:32:49.775 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (12m) 110s ago 12m 17.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (8m) 29s ago 21m 507M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (7m) 110s ago 18m 450M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (3m) 29s ago 21m 48.5M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (3m) 110s ago 18m 37.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (6m) 29s ago 20m 7872k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:32:49.776 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 110s ago 18m 7948k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (83s) 29s ago 17m 86.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (37s) 29s ago 16m 11.2M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 starting - - - 1216M 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (15m) 110s ago 15m 249M 4096M 18.2.0 dc2bc1663786 6868701f3d7d 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (15m) 110s ago 15m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:32:49.777 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (14m) 110s ago 14m 204M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:32:49.778 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 29s ago 19m 70.2M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:32:49.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 msgr2=0x7feb30071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 0x7feb30071170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb3400b5c0 tx=0x7feb34005960 comp rx=0 tx=0).stop 2024-04-24T08:32:49.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 msgr2=0x7feb4c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d8d0 tx=0x7feb3c00dc90 comp rx=0 tx=0).stop 2024-04-24T08:32:49.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 shutdown_connections 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feb3006ecc0 0x7feb30071170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb4c0687a0 0x7feb4c198180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 --2- 172.21.15.90:0/1836677195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb4c069140 0x7feb4c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 >> 172.21.15.90:0/1836677195 conn(0x7feb4c075130 msgr2=0x7feb4c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 shutdown_connections 2024-04-24T08:32:49.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.776+0000 7feb52a00700 1 -- 172.21.15.90:0/1836677195 wait complete. 2024-04-24T08:32:49.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 -- 172.21.15.90:0/407671813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 msgr2=0x7fb6281008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 --2- 172.21.15.90:0/407671813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281008b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb618009910 tx=0x7fb618009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 -- 172.21.15.90:0/407671813 shutdown_connections 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 --2- 172.21.15.90:0/407671813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 0x7fb628104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 --2- 172.21.15.90:0/407671813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281008b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 -- 172.21.15.90:0/407671813 >> 172.21.15.90:0/407671813 conn(0x7fb6280fbda0 msgr2=0x7fb6280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.923+0000 7fb62dc00700 1 -- 172.21.15.90:0/407671813 shutdown_connections 2024-04-24T08:32:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.924+0000 7fb62dc00700 1 -- 172.21.15.90:0/407671813 wait complete. 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 Processor -- start 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 -- start start 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 0x7fb6281985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628198c40 con 0x7fb6281004e0 2024-04-24T08:32:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb62dc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628198d80 con 0x7fb628100e80 2024-04-24T08:32:49.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb626a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb626a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53180/0 (socket says 172.21.15.90:53180) 2024-04-24T08:32:49.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.925+0000 7fb626a00700 1 -- 172.21.15.90:0/1742147054 learned_addr learned my addr 172.21.15.90:0/1742147054 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.926+0000 7fb626000700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 0x7fb6281985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.926+0000 7fb626a00700 1 -- 172.21.15.90:0/1742147054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 msgr2=0x7fb6281985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.926+0000 7fb626a00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 0x7fb6281985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.926+0000 7fb626a00700 1 -- 172.21.15.90:0/1742147054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6180095f0 con 0x7fb6281004e0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.926+0000 7fb626a00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b5c0 tx=0x7fb61800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.927+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61801d070 con 0x7fb6281004e0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.927+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb61800f460 con 0x7fb6281004e0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.927+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62819cb70 con 0x7fb6281004e0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.927+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb618017610 con 0x7fb6281004e0 2024-04-24T08:32:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.927+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62819d060 con 0x7fb6281004e0 2024-04-24T08:32:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.928+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62804f2a0 con 0x7fb6281004e0 2024-04-24T08:32:49.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.929+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb61800f5d0 con 0x7fb6281004e0 2024-04-24T08:32:49.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.930+0000 7fb61f400700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 0x7fb60c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:49.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.930+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb6180904b0 con 0x7fb6281004e0 2024-04-24T08:32:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.931+0000 7fb626000700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 0x7fb60c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.931+0000 7fb626000700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 0x7fb60c071170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6281996d0 tx=0x7fb614005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:49.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:49.939+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb618059220 con 0x7fb6281004e0 2024-04-24T08:32:50.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.289+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb62819d3c0 con 0x7fb6281004e0 2024-04-24T08:32:50.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.290+0000 7fb61f400700 1 -- 172.21.15.90:0/1742147054 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb61805c8e0 con 0x7fb6281004e0 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:50.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:32:50.292 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:32:50.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 msgr2=0x7fb60c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 0x7fb60c071170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6281996d0 tx=0x7fb614005d50 comp rx=0 tx=0).stop 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 msgr2=0x7fb6281980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b5c0 tx=0x7fb61800be00 comp rx=0 tx=0).stop 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 shutdown_connections 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb60c06ecc0 0x7fb60c071170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6281004e0 0x7fb6281980b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 --2- 172.21.15.90:0/1742147054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100e80 0x7fb6281985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.292+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 >> 172.21.15.90:0/1742147054 conn(0x7fb6280fbda0 msgr2=0x7fb628103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.293+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 shutdown_connections 2024-04-24T08:32:50.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.293+0000 7fb62dc00700 1 -- 172.21.15.90:0/1742147054 wait complete. 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 -- 172.21.15.90:0/1391556024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 msgr2=0x7f996c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 --2- 172.21.15.90:0/1391556024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c106ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9958009910 tx=0x7f9958009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 -- 172.21.15.90:0/1391556024 shutdown_connections 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 --2- 172.21.15.90:0/1391556024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c106ff0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 --2- 172.21.15.90:0/1391556024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 0x7f996c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 -- 172.21.15.90:0/1391556024 >> 172.21.15.90:0/1391556024 conn(0x7f996c076060 msgr2=0x7f996c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:50.447 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:50 smithi090 ceph-mon[85282]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:50.447 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:50 smithi090 ceph-mon[85282]: pgmap v122: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 688 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:50.447 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:50 smithi090 ceph-mon[85282]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:32:50.447 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:32:50 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:32:50.156+0000 7f30ef800700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 -- 172.21.15.90:0/1391556024 shutdown_connections 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.445+0000 7f9971c00700 1 -- 172.21.15.90:0/1391556024 wait complete. 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.446+0000 7f9971c00700 1 Processor -- start 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.446+0000 7f9971c00700 1 -- start start 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f9971c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f9971c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 0x7f996c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f9971c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c19d060 con 0x7f996c106b80 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f9971c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1a0df0 con 0x7f996c10c8f0 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f996aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f996aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53194/0 (socket says 172.21.15.90:53194) 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.447+0000 7f996aa00700 1 -- 172.21.15.90:0/2138779937 learned_addr learned my addr 172.21.15.90:0/2138779937 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f996a000700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 0x7f996c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f996aa00700 1 -- 172.21.15.90:0/2138779937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 msgr2=0x7f996c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f996aa00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 0x7f996c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f996aa00700 1 -- 172.21.15.90:0/2138779937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580095f0 con 0x7f996c106b80 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f996aa00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c8a0 tx=0x7f995c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c00cea0 con 0x7f996c106b80 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995c004830 con 0x7f996c106b80 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c1a10d0 con 0x7f996c106b80 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c0055b0 con 0x7f996c106b80 2024-04-24T08:32:50.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.448+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c1a1620 con 0x7f996c106b80 2024-04-24T08:32:50.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.449+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c10ecc0 con 0x7f996c106b80 2024-04-24T08:32:50.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.450+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f995c0049a0 con 0x7f996c106b80 2024-04-24T08:32:50.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.451+0000 7f9963400700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 0x7f9950071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.451+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f995c08eaf0 con 0x7f996c106b80 2024-04-24T08:32:50.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.452+0000 7f996a000700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 0x7f9950071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.452+0000 7f996a000700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 0x7f9950071170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f9958005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:50.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.456+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f995c057900 con 0x7f996c106b80 2024-04-24T08:32:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:50 smithi149 ceph-mon[66490]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:50 smithi149 ceph-mon[66490]: pgmap v122: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 688 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:50 smithi149 ceph-mon[66490]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:32:50.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.747+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f996c04f2a0 con 0x7f996c106b80 2024-04-24T08:32:50.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.748+0000 7f9963400700 1 -- 172.21.15.90:0/2138779937 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f995c01d780 con 0x7f996c106b80 2024-04-24T08:32:50.749 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:32:50.749 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:32:50.749 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:32:50.750 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:32:50.751 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:50.752 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:32:50.753 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:32:50.753 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:50.753 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:32:50.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 msgr2=0x7f9950071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 0x7f9950071170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f9958005fd0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 msgr2=0x7f996c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c8a0 tx=0x7f995c00cc60 comp rx=0 tx=0).stop 2024-04-24T08:32:50.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 shutdown_connections 2024-04-24T08:32:50.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f995006ecc0 0x7f9950071170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c106b80 0x7f996c19c440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 --2- 172.21.15.90:0/2138779937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f996c10c8f0 0x7f996c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 >> 172.21.15.90:0/2138779937 conn(0x7f996c076060 msgr2=0x7f996c077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 shutdown_connections 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.751+0000 7f9971c00700 1 -- 172.21.15.90:0/2138779937 wait complete. 2024-04-24T08:32:50.755 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 -- 172.21.15.90:0/237368442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc41061b0 msgr2=0x7f5cc4106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/237368442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc41061b0 0x7f5cc4106580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009910 tx=0x7f5cb8009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 -- 172.21.15.90:0/237368442 shutdown_connections 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/237368442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc4068540 0x7f5cc40689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/237368442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc41061b0 0x7f5cc4106580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 -- 172.21.15.90:0/237368442 >> 172.21.15.90:0/237368442 conn(0x7f5cc4074fa0 msgr2=0x7f5cc40753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 -- 172.21.15.90:0/237368442 shutdown_connections 2024-04-24T08:32:50.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.916+0000 7f5cc9e00700 1 -- 172.21.15.90:0/237368442 wait complete. 2024-04-24T08:32:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.917+0000 7f5cc9e00700 1 Processor -- start 2024-04-24T08:32:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.917+0000 7f5cc9e00700 1 -- start start 2024-04-24T08:32:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.917+0000 7f5cc9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc41061b0 0x7f5cc41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc9e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4198a80 con 0x7f5cc4068540 2024-04-24T08:32:50.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc9e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc419c810 con 0x7f5cc41061b0 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53206/0 (socket says 172.21.15.90:53206) 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 -- 172.21.15.90:0/1112114241 learned_addr learned my addr 172.21.15.90:0/1112114241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc3e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc41061b0 0x7f5cc41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 -- 172.21.15.90:0/1112114241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc41061b0 msgr2=0x7f5cc41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc41061b0 0x7f5cc41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.918+0000 7f5cc8a00700 1 -- 172.21.15.90:0/1112114241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80095f0 con 0x7f5cc4068540 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc8a00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb801d070 con 0x7f5cc4068540 2024-04-24T08:32:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cb8004fb0 con 0x7f5cc4068540 2024-04-24T08:32:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb8022ea0 con 0x7f5cc4068540 2024-04-24T08:32:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc419ca90 con 0x7f5cc4068540 2024-04-24T08:32:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.919+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc419d000 con 0x7f5cc4068540 2024-04-24T08:32:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.920+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4062380 con 0x7f5cc4068540 2024-04-24T08:32:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.921+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5cb8022470 con 0x7f5cc4068540 2024-04-24T08:32:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.922+0000 7f5cc1600700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 0x7f5cac071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.922+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5cb8090360 con 0x7f5cc4068540 2024-04-24T08:32:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.923+0000 7f5cc3e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 0x7f5cac071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:50.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.923+0000 7f5cc3e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 0x7f5cac071170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:50.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:50.928+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5cb8059200 con 0x7f5cc4068540 2024-04-24T08:32:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.215+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cc419d4b0 con 0x7f5cac06ecc0 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.216+0000 7f5cc1600700 1 -- 172.21.15.90:0/1112114241 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5cc419d4b0 con 0x7f5cac06ecc0 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:32:51.217 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "8/23 daemons upgraded", 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:32:51.218 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:32:51.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 msgr2=0x7f5cac071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 0x7f5cac071170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0008040 comp rx=0 tx=0).stop 2024-04-24T08:32:51.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 msgr2=0x7f5cc4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb800be20 comp rx=0 tx=0).stop 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 shutdown_connections 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f5cac06ecc0 0x7f5cac071170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cc4068540 0x7f5cc4197e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 --2- 172.21.15.90:0/1112114241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cc41061b0 0x7f5cc41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.219+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 >> 172.21.15.90:0/1112114241 conn(0x7f5cc4074fa0 msgr2=0x7f5cc40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.220+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 shutdown_connections 2024-04-24T08:32:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.220+0000 7f5cc9e00700 1 -- 172.21.15.90:0/1112114241 wait complete. 2024-04-24T08:32:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 -- 172.21.15.90:0/1680504571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4101070 msgr2=0x7f51b4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 --2- 172.21.15.90:0/1680504571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4101070 0x7f51b4101440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009910 tx=0x7f51a4009c20 comp rx=0 tx=0).stop 2024-04-24T08:32:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 -- 172.21.15.90:0/1680504571 shutdown_connections 2024-04-24T08:32:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 --2- 172.21.15.90:0/1680504571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4068540 0x7f51b40689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 --2- 172.21.15.90:0/1680504571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4101070 0x7f51b4101440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.381+0000 7f51ba400700 1 -- 172.21.15.90:0/1680504571 >> 172.21.15.90:0/1680504571 conn(0x7f51b4075130 msgr2=0x7f51b4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:51.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:51.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:51.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:32:51.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: osdmap e63: 6 total, 5 up, 6 in 2024-04-24T08:32:51.383 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:32:51.384 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1742147054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:51.384 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:51 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2138779937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.382+0000 7f51ba400700 1 -- 172.21.15.90:0/1680504571 shutdown_connections 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.382+0000 7f51ba400700 1 -- 172.21.15.90:0/1680504571 wait complete. 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.383+0000 7f51ba400700 1 Processor -- start 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.383+0000 7f51ba400700 1 -- start start 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51ba400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51ba400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101070 0x7f51b4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51ba400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b4198c70 con 0x7f51b4068540 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51ba400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419ca00 con 0x7f51b4101070 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51b3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51b3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53220/0 (socket says 172.21.15.90:53220) 2024-04-24T08:32:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51b3400700 1 -- 172.21.15.90:0/2715767672 learned_addr learned my addr 172.21.15.90:0/2715767672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.384+0000 7f51abe00700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101070 0x7f51b4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b3400700 1 -- 172.21.15.90:0/2715767672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101070 msgr2=0x7f51b4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b3400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101070 0x7f51b4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b3400700 1 -- 172.21.15.90:0/2715767672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40095f0 con 0x7f51b4068540 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b3400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a401d070 con 0x7f51b4068540 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a400f460 con 0x7f51b4068540 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a4017630 con 0x7f51b4068540 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.385+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b419cc80 con 0x7f51b4068540 2024-04-24T08:32:51.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.386+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b419d120 con 0x7f51b4068540 2024-04-24T08:32:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.387+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b404f2a0 con 0x7f51b4068540 2024-04-24T08:32:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.388+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f51a400f5d0 con 0x7f51b4068540 2024-04-24T08:32:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.389+0000 7f51b0c00700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 0x7f5198071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:32:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.389+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f51a4090370 con 0x7f51b4068540 2024-04-24T08:32:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.389+0000 7f51abe00700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 0x7f5198071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:32:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.390+0000 7f51abe00700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 0x7f5198071170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f519c005950 tx=0x7f519c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:32:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.394+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f51a40590e0 con 0x7f51b4068540 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: osdmap e63: 6 total, 5 up, 6 in 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1742147054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:32:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:51 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2138779937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:32:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.730+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f51b419d490 con 0x7f51b4068540 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.731+0000 7f51b0c00700 1 -- 172.21.15.90:0/2715767672 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1041 (secure 0 0 0) 0x7f51a405c7a0 con 0x7f51b4068540 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-04-24T08:32:51.732 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-04-24T08:32:51.733 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-04-24T08:32:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.734+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 msgr2=0x7f5198071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.734+0000 7f51ba400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 0x7f5198071170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f519c005950 tx=0x7f519c0058e0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.734+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 msgr2=0x7f51b4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:32:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.734+0000 7f51ba400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a400be20 comp rx=0 tx=0).stop 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.734+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 shutdown_connections 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f519806ecc0 0x7f5198071170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b4068540 0x7f51b4198050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 --2- 172.21.15.90:0/2715767672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101070 0x7f51b4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 >> 172.21.15.90:0/2715767672 conn(0x7f51b4075130 msgr2=0x7f51b40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 shutdown_connections 2024-04-24T08:32:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:32:51.735+0000 7f51ba400700 1 -- 172.21.15.90:0/2715767672 wait complete. 2024-04-24T08:32:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: pgmap v124: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143] boot 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2715767672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:32:52.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:52 smithi149 ceph-mon[66490]: osdmap e65: 6 total, 6 up, 6 in 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: pgmap v124: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: osd.2 [v2:172.21.15.90:6818/701071143,v1:172.21.15.90:6819/701071143] boot 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:32:52.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:52.617 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:52.617 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2715767672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:32:52.617 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:52 smithi090 ceph-mon[85282]: osdmap e65: 6 total, 6 up, 6 in 2024-04-24T08:32:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:54 smithi149 ceph-mon[66490]: pgmap v127: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:54.530 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:54 smithi090 ceph-mon[85282]: pgmap v127: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/267 objects degraded (13.109%) 2024-04-24T08:32:54.530 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:54.530 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:32:55.407 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:55 smithi090 ceph-mon[85282]: pgmap v128: 65 pgs: 9 active+undersized+degraded, 10 active+undersized, 46 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/267 objects degraded (10.112%) 2024-04-24T08:32:55.407 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:55 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:55 smithi149 ceph-mon[66490]: pgmap v128: 65 pgs: 9 active+undersized+degraded, 10 active+undersized, 46 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/267 objects degraded (10.112%) 2024-04-24T08:32:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:55 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T08:32:57.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:57 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/267 objects degraded (10.112%), 9 pgs degraded) 2024-04-24T08:32:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:57 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/267 objects degraded (10.112%), 9 pgs degraded) 2024-04-24T08:32:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:32:58 smithi149 ceph-mon[66490]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-24T08:32:58.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:32:58 smithi090 ceph-mon[85282]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T08:33:00.361 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.362 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.482 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:00.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T08:33:00.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:00.483 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:01.495 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:01 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T08:33:01.495 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:01 smithi149 ceph-mon[66490]: Upgrade: osd.3 is safe to restart 2024-04-24T08:33:01.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:01 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T08:33:01.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:01 smithi090 ceph-mon[85282]: Upgrade: osd.3 is safe to restart 2024-04-24T08:33:02.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:02 smithi149 ceph-mon[66490]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 925 B/s rd, 1 op/s 2024-04-24T08:33:02.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:02.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:33:02.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:02.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:02 smithi090 ceph-mon[85282]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 925 B/s rd, 1 op/s 2024-04-24T08:33:02.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:02.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:33:02.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:03.383 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:03 smithi149 ceph-mon[66490]: Upgrade: Updating osd.3 2024-04-24T08:33:03.383 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:03 smithi149 ceph-mon[66490]: Deploying daemon osd.3 on smithi149 2024-04-24T08:33:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:03 smithi090 ceph-mon[85282]: Upgrade: Updating osd.3 2024-04-24T08:33:03.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:03 smithi090 ceph-mon[85282]: Deploying daemon osd.3 on smithi149 2024-04-24T08:33:04.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:04 smithi149 ceph-mon[66490]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 841 B/s rd, 1 op/s 2024-04-24T08:33:04.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:04 smithi090 ceph-mon[85282]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 841 B/s rd, 1 op/s 2024-04-24T08:33:06.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:06 smithi149 ceph-mon[66490]: pgmap v133: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:33:06.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:06 smithi090 ceph-mon[85282]: pgmap v133: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:33:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:07.275 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:07 smithi149 systemd[1]: Stopping Ceph osd.3 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:33:07.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:07 smithi149 ceph-mon[66490]: pgmap v134: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-24T08:33:07.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:07 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[37321]: 2024-04-24T08:33:07.470+0000 7fa7c1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:33:07.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:07 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[37321]: 2024-04-24T08:33:07.470+0000 7fa7c1c00700 -1 osd.3 65 *** Got signal Terminated *** 2024-04-24T08:33:07.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:07 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[37321]: 2024-04-24T08:33:07.470+0000 7fa7c1c00700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:33:07.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:07 smithi090 ceph-mon[85282]: pgmap v134: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-24T08:33:08.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:08 smithi090 ceph-mon[85282]: osd.3 marked itself down and dead 2024-04-24T08:33:08.645 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:08 smithi149 ceph-mon[66490]: osd.3 marked itself down and dead 2024-04-24T08:33:09.029 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:08 smithi149 podman[75856]: 2024-04-24 08:33:08.888813934 +0000 UTC m=+1.607809605 container died 6868701f3d7d18acc7ce84dfa37fea716c38ca4aefb8fdd13f1c41bc40a96862 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T08:33:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:09 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:33:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:09 smithi149 ceph-mon[66490]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T08:33:09.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:09 smithi149 ceph-mon[66490]: pgmap v136: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:33:09.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:09 smithi149 podman[75856]: 2024-04-24 08:33:09.356122608 +0000 UTC m=+2.075118316 container cleanup 6868701f3d7d18acc7ce84dfa37fea716c38ca4aefb8fdd13f1c41bc40a96862 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T08:33:09.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:09 smithi149 bash[75856]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3 2024-04-24T08:33:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:09 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:33:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:09 smithi090 ceph-mon[85282]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T08:33:09.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:09 smithi090 ceph-mon[85282]: pgmap v136: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:33:10.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:09 smithi149 podman[75883]: 2024-04-24 08:33:09.607629815 +0000 UTC m=+0.706008995 container remove 6868701f3d7d18acc7ce84dfa37fea716c38ca4aefb8fdd13f1c41bc40a96862 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T08:33:10.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:10 smithi149 ceph-mon[66490]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T08:33:10.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:10 smithi149 podman[75972]: 2024-04-24 08:33:10.091704177 +0000 UTC m=+0.032077868 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:10.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:10 smithi149 podman[75972]: 2024-04-24 08:33:10.250331081 +0000 UTC m=+0.190704819 container create 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T08:33:10.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:10 smithi090 ceph-mon[85282]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T08:33:11.005 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:10 smithi149 podman[75972]: 2024-04-24 08:33:10.545692772 +0000 UTC m=+0.486066494 container init 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T08:33:11.005 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:10 smithi149 podman[75972]: 2024-04-24 08:33:10.557597462 +0000 UTC m=+0.497971204 container start 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T08:33:11.005 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:10 smithi149 podman[75972]: 2024-04-24 08:33:10.677822089 +0000 UTC m=+0.618195831 container attach 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:33:11.006 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:11 smithi149 podman[75972]: 2024-04-24 08:33:11.004044184 +0000 UTC m=+0.944417909 container died 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:33:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:11 smithi149 ceph-mon[66490]: pgmap v138: 65 pgs: 10 active+undersized+degraded, 9 peering, 3 stale+active+clean, 12 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-24T08:33:11.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:11 smithi090 ceph-mon[85282]: pgmap v138: 65 pgs: 10 active+undersized+degraded, 9 peering, 3 stale+active+clean, 12 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-24T08:33:12.213 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:11 smithi149 podman[75997]: 2024-04-24 08:33:11.814449541 +0000 UTC m=+0.795182817 container remove 8f8503bb07f8ae15e62ccfc4ebc88960143985c59807b15af00102e239d6cf58 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:33:12.213 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:11 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-24T08:33:12.213 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:11 smithi149 systemd[1]: Stopped Ceph osd.3 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:33:12.213 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:11 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 57.471s CPU time. 2024-04-24T08:33:12.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:12 smithi149 systemd[1]: Starting Ceph osd.3 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:33:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:12 smithi149 ceph-mon[66490]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T08:33:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:12 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 10 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:12.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:12 smithi090 ceph-mon[85282]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T08:33:12.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:12 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 10 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:12.820 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:12 smithi149 podman[76100]: 2024-04-24 08:33:12.469615496 +0000 UTC m=+0.033737466 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:12.820 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:12 smithi149 podman[76100]: 2024-04-24 08:33:12.590907097 +0000 UTC m=+0.155029107 container create 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:33:13.136 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:12 smithi149 podman[76100]: 2024-04-24 08:33:12.953480688 +0000 UTC m=+0.517602741 container init 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:33:13.136 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:12 smithi149 podman[76100]: 2024-04-24 08:33:12.965566458 +0000 UTC m=+0.529688459 container start 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T08:33:13.136 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:13 smithi149 podman[76100]: 2024-04-24 08:33:13.135106852 +0000 UTC m=+0.699228922 container attach 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS) 2024-04-24T08:33:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:13 smithi149 ceph-mon[66490]: pgmap v139: 65 pgs: 10 active+undersized+degraded, 9 peering, 3 stale+active+clean, 12 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-24T08:33:13.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:13 smithi090 ceph-mon[85282]: pgmap v139: 65 pgs: 10 active+undersized+degraded, 9 peering, 3 stale+active+clean, 12 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-24T08:33:14.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:14.527 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 bash[76100]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T08:33:14.528 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T08:33:14.529 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T08:33:14.529 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T08:33:14.529 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate[76114]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T08:33:14.529 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:14 smithi149 podman[76100]: 2024-04-24 08:33:14.21952628 +0000 UTC m=+1.783648300 container died 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T08:33:15.276 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 podman[76239]: 2024-04-24 08:33:15.013640849 +0000 UTC m=+0.775973014 container remove 8770354e268502b3ab8c0e362f78354c709a7bafdebaf5abadbc40899c822255 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T08:33:15.777 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 podman[76301]: 2024-04-24 08:33:15.32816128 +0000 UTC m=+0.036256922 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:15.777 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 podman[76301]: 2024-04-24 08:33:15.489734843 +0000 UTC m=+0.197830522 container create 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T08:33:16.132 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 podman[76301]: 2024-04-24 08:33:15.793439968 +0000 UTC m=+0.501535657 container init 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T08:33:16.132 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 podman[76301]: 2024-04-24 08:33:15.804820297 +0000 UTC m=+0.512916030 container start 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T08:33:16.132 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 bash[76301]: 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 2024-04-24T08:33:16.132 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:15 smithi149 systemd[1]: Started Ceph osd.3 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:33:16.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-osd[76320]: -- 172.21.15.149:0/3510774223 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561c2d09680 con 0x5561c2cd2c00 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: pgmap v140: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: pgmap v140: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:17.277 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:16 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:33:16.844+0000 7f5fe492d7c0 -1 Falling back to public interface 2024-04-24T08:33:18.478 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:18 smithi149 ceph-mon[66490]: pgmap v141: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:18 smithi090 ceph-mon[85282]: pgmap v141: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:20 smithi149 ceph-mon[66490]: pgmap v142: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:20 smithi090 ceph-mon[85282]: pgmap v142: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.907+0000 7f7fcb200700 1 -- 172.21.15.90:0/969894255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc4069140 msgr2=0x7f7fc4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.907+0000 7f7fcb200700 1 --2- 172.21.15.90:0/969894255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc4069140 0x7f7fc4104eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009910 tx=0x7f7fac009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 -- 172.21.15.90:0/969894255 shutdown_connections 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 --2- 172.21.15.90:0/969894255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc4069140 0x7f7fc4104eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 --2- 172.21.15.90:0/969894255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc40687a0 0x7f7fc4068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 -- 172.21.15.90:0/969894255 >> 172.21.15.90:0/969894255 conn(0x7f7fc4075130 msgr2=0x7f7fc4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 -- 172.21.15.90:0/969894255 shutdown_connections 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.908+0000 7f7fcb200700 1 -- 172.21.15.90:0/969894255 wait complete. 2024-04-24T08:33:21.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 Processor -- start 2024-04-24T08:33:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 -- start start 2024-04-24T08:33:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc4069140 0x7f7fc4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4198c70 con 0x7f7fc40687a0 2024-04-24T08:33:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.909+0000 7f7fcb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc419ca00 con 0x7f7fc4069140 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41194/0 (socket says 172.21.15.90:41194) 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 -- 172.21.15.90:0/1621671134 learned_addr learned my addr 172.21.15.90:0/1621671134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fbb400700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc4069140 0x7f7fc4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 -- 172.21.15.90:0/1621671134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc4069140 msgr2=0x7f7fc4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc4069140 0x7f7fc4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:21.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 -- 172.21.15.90:0/1621671134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0095f0 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.910+0000 7f7fc3e00700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40060b0 tx=0x7f7fb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.911+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb400dda0 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.911+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc419cce0 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.911+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fb400f040 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.911+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb4014790 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.911+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc419d230 con 0x7f7fc40687a0 2024-04-24T08:33:21.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.912+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc404f2a0 con 0x7f7fc40687a0 2024-04-24T08:33:21.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.914+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7fb40148f0 con 0x7f7fc40687a0 2024-04-24T08:33:21.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.915+0000 7f7fc1600700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 0x7f7fa8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:21.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.915+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7fb408e870 con 0x7f7fc40687a0 2024-04-24T08:33:21.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.915+0000 7f7fbb400700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 0x7f7fa8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:21.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.916+0000 7f7fbb400700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 0x7f7fa8071170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4199670 tx=0x7f7fac01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:21.921+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7fb4057710 con 0x7f7fc40687a0 2024-04-24T08:33:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.196+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc419d6b0 con 0x7f7fa806ecc0 2024-04-24T08:33:22.197 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:22.197 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:22.197 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:21 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:22.197 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:21 smithi090 ceph-mon[85282]: pgmap v143: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:22.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.199+0000 7f7fc1600700 1 -- 172.21.15.90:0/1621671134 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7fc419d6b0 con 0x7f7fa806ecc0 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 msgr2=0x7f7fa8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 0x7f7fa8071170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4199670 tx=0x7f7fac01a040 comp rx=0 tx=0).stop 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 msgr2=0x7f7fc4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40060b0 tx=0x7f7fb400d3b0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 shutdown_connections 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7fa806ecc0 0x7f7fa8071170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc40687a0 0x7f7fc4198050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 --2- 172.21.15.90:0/1621671134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fc4069140 0x7f7fc4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 >> 172.21.15.90:0/1621671134 conn(0x7f7fc4075130 msgr2=0x7f7fc4101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 shutdown_connections 2024-04-24T08:33:22.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.202+0000 7f7fcb200700 1 -- 172.21.15.90:0/1621671134 wait complete. 2024-04-24T08:33:22.219 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:33:22.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:22.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:22.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:21 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:22.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:21 smithi149 ceph-mon[66490]: pgmap v143: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-24T08:33:22.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 -- 172.21.15.90:0/1681505283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 msgr2=0x7fa5bc106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/1681505283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc106ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009910 tx=0x7fa5a8009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:22.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 -- 172.21.15.90:0/1681505283 shutdown_connections 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/1681505283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc106ff0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/1681505283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 0x7fa5bc10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 -- 172.21.15.90:0/1681505283 >> 172.21.15.90:0/1681505283 conn(0x7fa5bc075e90 msgr2=0x7fa5bc0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.363+0000 7fa5c1e00700 1 -- 172.21.15.90:0/1681505283 shutdown_connections 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.364+0000 7fa5c1e00700 1 -- 172.21.15.90:0/1681505283 wait complete. 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 Processor -- start 2024-04-24T08:33:22.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 -- start start 2024-04-24T08:33:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 0x7fa5bc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc19ce90 con 0x7fa5bc106b80 2024-04-24T08:33:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.365+0000 7fa5c1e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1a0c20 con 0x7fa5bc10c660 2024-04-24T08:33:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41204/0 (socket says 172.21.15.90:41204) 2024-04-24T08:33:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 -- 172.21.15.90:0/3961309057 learned_addr learned my addr 172.21.15.90:0/3961309057 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:22.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5bbe00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 0x7fa5bc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 -- 172.21.15.90:0/3961309057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 msgr2=0x7fa5bc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 0x7fa5bc19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.366+0000 7fa5c0a00700 1 -- 172.21.15.90:0/3961309057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80095f0 con 0x7fa5bc106b80 2024-04-24T08:33:22.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5c0a00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000d8d0 tx=0x7fa5b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b0009940 con 0x7fa5bc106b80 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b0010460 con 0x7fa5bc106b80 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b000f5f0 con 0x7fa5bc106b80 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc1a0f00 con 0x7fa5bc106b80 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.367+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc1a1450 con 0x7fa5bc106b80 2024-04-24T08:33:22.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.369+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc10eb40 con 0x7fa5bc106b80 2024-04-24T08:33:22.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.370+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa5b00105d0 con 0x7fa5bc106b80 2024-04-24T08:33:22.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.371+0000 7fa5b9600700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 0x7fa5a4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.371+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa5b008ee20 con 0x7fa5bc106b80 2024-04-24T08:33:22.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.371+0000 7fa5bbe00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 0x7fa5a4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.372+0000 7fa5bbe00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 0x7fa5a4071170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000c00 tx=0x7fa5a8005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:22.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.376+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5b0057c30 con 0x7fa5bc106b80 2024-04-24T08:33:22.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.627+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5bc1a1670 con 0x7fa5a406ecc0 2024-04-24T08:33:22.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.630+0000 7fa5b9600700 1 -- 172.21.15.90:0/3961309057 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa5bc1a1670 con 0x7fa5a406ecc0 2024-04-24T08:33:22.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 msgr2=0x7fa5a4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 0x7fa5a4071170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000c00 tx=0x7fa5a8005fd0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 msgr2=0x7fa5bc19c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000d8d0 tx=0x7fa5b000dc90 comp rx=0 tx=0).stop 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 shutdown_connections 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5a406ecc0 0x7fa5a4071170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5bc106b80 0x7fa5bc19c270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 --2- 172.21.15.90:0/3961309057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc10c660 0x7fa5bc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 >> 172.21.15.90:0/3961309057 conn(0x7fa5bc075e90 msgr2=0x7fa5bc0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 shutdown_connections 2024-04-24T08:33:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.633+0000 7fa5c1e00700 1 -- 172.21.15.90:0/3961309057 wait complete. 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 -- 172.21.15.90:0/2388390781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 msgr2=0x7f75a01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 --2- 172.21.15.90:0/2388390781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a01008b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7590009910 tx=0x7f7590009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 -- 172.21.15.90:0/2388390781 shutdown_connections 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 --2- 172.21.15.90:0/2388390781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 0x7f75a0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 --2- 172.21.15.90:0/2388390781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a01008b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 -- 172.21.15.90:0/2388390781 >> 172.21.15.90:0/2388390781 conn(0x7f75a00fbda0 msgr2=0x7f75a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 -- 172.21.15.90:0/2388390781 shutdown_connections 2024-04-24T08:33:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.793+0000 7f75a5000700 1 -- 172.21.15.90:0/2388390781 wait complete. 2024-04-24T08:33:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.794+0000 7f75a5000700 1 Processor -- start 2024-04-24T08:33:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f75a5000700 1 -- start start 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f75a5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f75a5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 0x7f75a0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f75a5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0073ad0 con 0x7f75a01004e0 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f75a5000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0073c10 con 0x7f75a0100e80 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f759de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f759de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41226/0 (socket says 172.21.15.90:41226) 2024-04-24T08:33:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.795+0000 7f759de00700 1 -- 172.21.15.90:0/2349221269 learned_addr learned my addr 172.21.15.90:0/2349221269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.796+0000 7f759d400700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 0x7f75a0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.796+0000 7f759de00700 1 -- 172.21.15.90:0/2349221269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 msgr2=0x7f75a0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.796+0000 7f759de00700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 0x7f75a0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.796+0000 7f759de00700 1 -- 172.21.15.90:0/2349221269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900095f0 con 0x7f75a01004e0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.796+0000 7f759de00700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7590006010 tx=0x7f759000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.797+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759001d070 con 0x7f75a01004e0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.797+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a0073e90 con 0x7f75a01004e0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.797+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7590004fb0 con 0x7f75a01004e0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.797+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7590022ea0 con 0x7f75a01004e0 2024-04-24T08:33:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.797+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01a24b0 con 0x7f75a01004e0 2024-04-24T08:33:22.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.798+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a01087f0 con 0x7f75a01004e0 2024-04-24T08:33:22.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.799+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7590022470 con 0x7f75a01004e0 2024-04-24T08:33:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.800+0000 7f7596a00700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 0x7f7584071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.800+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7590090350 con 0x7f75a01004e0 2024-04-24T08:33:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.800+0000 7f759d400700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 0x7f7584071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:22.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.801+0000 7f759d400700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 0x7f7584071170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:22.805+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7590059210 con 0x7f75a01004e0 2024-04-24T08:33:23.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:22 smithi149 ceph-mon[66490]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:23.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:22 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:23.027 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:22 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:33:22.654+0000 7f5fe492d7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-24T08:33:23.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:22 smithi090 ceph-mon[85282]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:23.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:22 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:23.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.051+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f75a0061190 con 0x7f758406ecc0 2024-04-24T08:33:23.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.064+0000 7f7596a00700 1 -- 172.21.15.90:0/2349221269 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f75a0061190 con 0x7f758406ecc0 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (6m) 31s ago 20m 13.8M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (20m) 31s ago 20m 8942k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (19m) 2s ago 19m 8468k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (2m) 31s ago 20m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (2m) 2s ago 19m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 31s ago 20m 74.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (13m) 31s ago 13m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (13m) 31s ago 13m 18.0M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (13m) 2s ago 13m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (12m) 2s ago 12m 18.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (9m) 31s ago 22m 507M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:33:23.065 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (8m) 2s ago 18m 451M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (4m) 31s ago 22m 49.8M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (3m) 2s ago 18m 39.2M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (7m) 31s ago 20m 7922k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 2s ago 19m 7994k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (117s) 31s ago 17m 88.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (71s) 31s ago 17m 83.9M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (39s) 31s ago 16m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (7s) 2s ago 16m 9.92M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (15m) 2s ago 15m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (14m) 2s ago 14m 203M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:33:23.066 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 31s ago 19m 69.4M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:33:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 msgr2=0x7f7584071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 0x7f7584071170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).stop 2024-04-24T08:33:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 msgr2=0x7f75a0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7590006010 tx=0x7f759000be20 comp rx=0 tx=0).stop 2024-04-24T08:33:23.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 shutdown_connections 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f758406ecc0 0x7f7584071170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75a01004e0 0x7f75a0073050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 --2- 172.21.15.90:0/2349221269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a0100e80 0x7f75a0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 >> 172.21.15.90:0/2349221269 conn(0x7f75a00fbda0 msgr2=0x7f75a00fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.067+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 shutdown_connections 2024-04-24T08:33:23.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.068+0000 7f75a5000700 1 -- 172.21.15.90:0/2349221269 wait complete. 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 -- 172.21.15.90:0/3154286303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 msgr2=0x7f92a81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 --2- 172.21.15.90:0/3154286303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81028e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9294009910 tx=0x7f9294009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 -- 172.21.15.90:0/3154286303 shutdown_connections 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 --2- 172.21.15.90:0/3154286303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81028e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 --2- 172.21.15.90:0/3154286303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 0x7f92a8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 -- 172.21.15.90:0/3154286303 >> 172.21.15.90:0/3154286303 conn(0x7f92a80fdf90 msgr2=0x7f92a81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 -- 172.21.15.90:0/3154286303 shutdown_connections 2024-04-24T08:33:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.226+0000 7f92afe00700 1 -- 172.21.15.90:0/3154286303 wait complete. 2024-04-24T08:33:23.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.227+0000 7f92afe00700 1 Processor -- start 2024-04-24T08:33:23.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92afe00700 1 -- start start 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92afe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92afe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 0x7f92a8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92afe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8198ef0 con 0x7f92a8102470 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92afe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a819cc80 con 0x7f92a8108470 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41250/0 (socket says 172.21.15.90:41250) 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.228+0000 7f92ad400700 1 -- 172.21.15.90:0/395158574 learned_addr learned my addr 172.21.15.90:0/395158574 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:23.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92aca00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 0x7f92a8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92ad400700 1 -- 172.21.15.90:0/395158574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 msgr2=0x7f92a8198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92ad400700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 0x7f92a8198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92ad400700 1 -- 172.21.15.90:0/395158574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92940095f0 con 0x7f92a8102470 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92ad400700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d900 tx=0x7f929c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.229+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f929c004a00 con 0x7f92a8102470 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.230+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f929c0054e0 con 0x7f92a8102470 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.230+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a819cf60 con 0x7f92a8102470 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.230+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f929c009d00 con 0x7f92a8102470 2024-04-24T08:33:23.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.230+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a819d4b0 con 0x7f92a8102470 2024-04-24T08:33:23.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.231+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a810a970 con 0x7f92a8102470 2024-04-24T08:33:23.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.232+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f929c005020 con 0x7f92a8102470 2024-04-24T08:33:23.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.233+0000 7f92a6000700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 0x7f9290071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.233+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f929c08ed30 con 0x7f92a8102470 2024-04-24T08:33:23.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.233+0000 7f92aca00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 0x7f9290071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.234+0000 7f92aca00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 0x7f9290071170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92a81998f0 tx=0x7f929401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:23.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.238+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f929c057b20 con 0x7f92a8102470 2024-04-24T08:33:23.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:33:23.154+0000 7f5fe492d7c0 -1 osd.3 65 log_to_monitors true 2024-04-24T08:33:23.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.561+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f92a819d720 con 0x7f92a8102470 2024-04-24T08:33:23.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.562+0000 7f92a6000700 1 -- 172.21.15.90:0/395158574 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f929c017330 con 0x7f92a8102470 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-04-24T08:33:23.563 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 3 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 7 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:33:23.564 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 msgr2=0x7f9290071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 0x7f9290071170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92a81998f0 tx=0x7f929401a040 comp rx=0 tx=0).stop 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 msgr2=0x7f92a81982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d900 tx=0x7f929c00dc10 comp rx=0 tx=0).stop 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 shutdown_connections 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f929006ecc0 0x7f9290071170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92a8102470 0x7f92a81982d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 --2- 172.21.15.90:0/395158574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a8108470 0x7f92a8198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 >> 172.21.15.90:0/395158574 conn(0x7f92a80fdf90 msgr2=0x7f92a80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 shutdown_connections 2024-04-24T08:33:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.565+0000 7f92afe00700 1 -- 172.21.15.90:0/395158574 wait complete. 2024-04-24T08:33:23.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.720+0000 7f8c45400700 1 -- 172.21.15.90:0/3328237907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 msgr2=0x7f8c401028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.720+0000 7f8c45400700 1 --2- 172.21.15.90:0/3328237907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401028e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009910 tx=0x7f8c2c009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 -- 172.21.15.90:0/3328237907 shutdown_connections 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 --2- 172.21.15.90:0/3328237907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401028e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 --2- 172.21.15.90:0/3328237907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 0x7f8c40108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 -- 172.21.15.90:0/3328237907 >> 172.21.15.90:0/3328237907 conn(0x7f8c400fdf90 msgr2=0x7f8c401003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 -- 172.21.15.90:0/3328237907 shutdown_connections 2024-04-24T08:33:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.721+0000 7f8c45400700 1 -- 172.21.15.90:0/3328237907 wait complete. 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 Processor -- start 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 -- start start 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 0x7f8c40198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40198d80 con 0x7f8c40102470 2024-04-24T08:33:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.722+0000 7f8c45400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40198ec0 con 0x7f8c40108470 2024-04-24T08:33:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41264/0 (socket says 172.21.15.90:41264) 2024-04-24T08:33:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 -- 172.21.15.90:0/2356467554 learned_addr learned my addr 172.21.15.90:0/2356467554 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 -- 172.21.15.90:0/2356467554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 msgr2=0x7f8c40198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3de00700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 0x7f8c40198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 0x7f8c40198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.723+0000 7f8c3e800700 1 -- 172.21.15.90:0/2356467554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0095f0 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c3e800700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c930 tx=0x7f8c3000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c30007ad0 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c30007c30 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c30018610 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4019cd10 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.724+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4019d260 con 0x7f8c40102470 2024-04-24T08:33:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.725+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4010a970 con 0x7f8c40102470 2024-04-24T08:33:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.726+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8c3000f450 con 0x7f8c40102470 2024-04-24T08:33:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.727+0000 7f8c37400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 0x7f8c24071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.727+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8c3008eff0 con 0x7f8c40102470 2024-04-24T08:33:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.728+0000 7f8c3de00700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 0x7f8c24071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:23.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.728+0000 7f8c3de00700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 0x7f8c24071170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c000c00 tx=0x7f8c2c005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:23.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.732+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8c300c3ff0 con 0x7f8c40102470 2024-04-24T08:33:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.998+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c40199620 con 0x7f8c40102470 2024-04-24T08:33:24.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:23.999+0000 7f8c37400700 1 -- 172.21.15.90:0/2356467554 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f8c30057de0 con 0x7f8c40102470 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:33:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:33:24.002 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:24.003 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:33:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:24.004 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:24.004 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.002+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 msgr2=0x7f8c24071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.002+0000 7f8c45400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 0x7f8c24071170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c000c00 tx=0x7f8c2c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.002+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 msgr2=0x7f8c401981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.002+0000 7f8c45400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c930 tx=0x7f8c3000ccf0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 shutdown_connections 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f8c2406ecc0 0x7f8c24071170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c40102470 0x7f8c401981f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 --2- 172.21.15.90:0/2356467554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c40108470 0x7f8c40198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 >> 172.21.15.90:0/2356467554 conn(0x7f8c400fdf90 msgr2=0x7f8c400ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 shutdown_connections 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.003+0000 7f8c45400700 1 -- 172.21.15.90:0/2356467554 wait complete. 2024-04-24T08:33:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:33:24.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:24.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: pgmap v144: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 59/267 objects degraded (22.097%) 2024-04-24T08:33:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:33:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:33:24.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/395158574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:24.028 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:33:23 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:33:23.900+0000 7f5fd3c00700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 -- 172.21.15.90:0/1860591318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 msgr2=0x7f34f0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 --2- 172.21.15.90:0/1860591318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0108840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009910 tx=0x7f34e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 -- 172.21.15.90:0/1860591318 shutdown_connections 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 --2- 172.21.15.90:0/1860591318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 --2- 172.21.15.90:0/1860591318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0108840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 -- 172.21.15.90:0/1860591318 >> 172.21.15.90:0/1860591318 conn(0x7f34f00fdf90 msgr2=0x7f34f01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:24.150 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: pgmap v144: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 59/267 objects degraded (22.097%) 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:33:24.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:23 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/395158574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:24.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 -- 172.21.15.90:0/1860591318 shutdown_connections 2024-04-24T08:33:24.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.149+0000 7f34f7200700 1 -- 172.21.15.90:0/1860591318 wait complete. 2024-04-24T08:33:24.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.150+0000 7f34f7200700 1 Processor -- start 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34f7200700 1 -- start start 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34f7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34f7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34f7200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0198e30 con 0x7f34f0108470 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34f7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f019cbc0 con 0x7f34f0102470 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.151+0000 7f34ef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41274/0 (socket says 172.21.15.90:41274) 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 -- 172.21.15.90:0/46029749 learned_addr learned my addr 172.21.15.90:0/46029749 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34efe00700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 -- 172.21.15.90:0/46029749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 msgr2=0x7f34f0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 -- 172.21.15.90:0/46029749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00095f0 con 0x7f34f0108470 2024-04-24T08:33:24.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34efe00700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.152+0000 7f34ef400700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b4b0 tx=0x7f34dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.153+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34dc011800 con 0x7f34f0108470 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.153+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34dc011e20 con 0x7f34f0108470 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.153+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f019cea0 con 0x7f34f0108470 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.153+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34dc00f2e0 con 0x7f34f0108470 2024-04-24T08:33:24.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.153+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f019d3f0 con 0x7f34f0108470 2024-04-24T08:33:24.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.154+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f010a9e0 con 0x7f34f0108470 2024-04-24T08:33:24.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.155+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f34dc010380 con 0x7f34f0108470 2024-04-24T08:33:24.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.156+0000 7f34ecc00700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 0x7f34d4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.156+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f34dc08e9a0 con 0x7f34f0108470 2024-04-24T08:33:24.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.156+0000 7f34efe00700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 0x7f34d4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.157+0000 7f34efe00700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 0x7f34d4071170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006010 tx=0x7f34e000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:24.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.162+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f34dc0577b0 con 0x7f34f0108470 2024-04-24T08:33:24.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.413+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34f019d670 con 0x7f34d406ecc0 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.415+0000 7f34ecc00700 1 -- 172.21.15.90:0/46029749 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f34f019d670 con 0x7f34d406ecc0 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:33:24.416 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:33:24.417 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:33:24.417 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "10/23 daemons upgraded", 2024-04-24T08:33:24.417 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:33:24.417 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:33:24.417 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 msgr2=0x7f34d4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 0x7f34d4071170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006010 tx=0x7f34e000b540 comp rx=0 tx=0).stop 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 msgr2=0x7f34f0198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b4b0 tx=0x7f34dc00b870 comp rx=0 tx=0).stop 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 shutdown_connections 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f34d406ecc0 0x7f34d4071170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34f0102470 0x7f34f0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 --2- 172.21.15.90:0/46029749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34f0108470 0x7f34f0198750 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 >> 172.21.15.90:0/46029749 conn(0x7f34f00fdf90 msgr2=0x7f34f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 shutdown_connections 2024-04-24T08:33:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.418+0000 7f34f7200700 1 -- 172.21.15.90:0/46029749 wait complete. 2024-04-24T08:33:24.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 -- 172.21.15.90:0/3729576286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac10c8f0 msgr2=0x7f02ac10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 --2- 172.21.15.90:0/3729576286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac10c8f0 0x7f02ac10ccc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009910 tx=0x7f02a0009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 -- 172.21.15.90:0/3729576286 shutdown_connections 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 --2- 172.21.15.90:0/3729576286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac106b80 0x7f02ac106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 --2- 172.21.15.90:0/3729576286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac10c8f0 0x7f02ac10ccc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 -- 172.21.15.90:0/3729576286 >> 172.21.15.90:0/3729576286 conn(0x7f02ac076060 msgr2=0x7f02ac078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 -- 172.21.15.90:0/3729576286 shutdown_connections 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.568+0000 7f02b3a00700 1 -- 172.21.15.90:0/3729576286 wait complete. 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.569+0000 7f02b3a00700 1 Processor -- start 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.569+0000 7f02b3a00700 1 -- start start 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b3a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b3a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac19cba0 0x7f02ac1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b3a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac19d1b0 con 0x7f02ac106b80 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b3a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac19d320 con 0x7f02ac19cba0 2024-04-24T08:33:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41306/0 (socket says 172.21.15.90:41306) 2024-04-24T08:33:24.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 -- 172.21.15.90:0/2553664761 learned_addr learned my addr 172.21.15.90:0/2553664761 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 -- 172.21.15.90:0/2553664761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac19cba0 msgr2=0x7f02ac1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02abe00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac19cba0 0x7f02ac1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac19cba0 0x7f02ac1a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.570+0000 7f02b1000700 1 -- 172.21.15.90:0/2553664761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00095f0 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02b1000700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02a000bd70 tx=0x7f02a000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a001d070 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02a0005dc0 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a0017400 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac1a15b0 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.571+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1a1b00 con 0x7f02ac106b80 2024-04-24T08:33:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.572+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac109b20 con 0x7f02ac106b80 2024-04-24T08:33:24.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.573+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f02a000f460 con 0x7f02ac106b80 2024-04-24T08:33:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.574+0000 7f02a9600700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 0x7f0294071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.574+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f02a0090050 con 0x7f02ac106b80 2024-04-24T08:33:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.574+0000 7f02abe00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 0x7f0294071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.575+0000 7f02abe00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 0x7f0294071170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f029c005950 tx=0x7f029c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:24.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.579+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f02a0058ee0 con 0x7f02ac106b80 2024-04-24T08:33:24.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.893+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02ac1a1e30 con 0x7f02ac106b80 2024-04-24T08:33:24.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.893+0000 7f02a9600700 1 -- 172.21.15.90:0/2553664761 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1372 (secure 0 0 0) 0x7f02a005c5a0 con 0x7f02ac106b80 2024-04-24T08:33:24.894 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded 2024-04-24T08:33:24.894 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-04-24T08:33:24.895 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-04-24T08:33:24.896 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 msgr2=0x7f0294071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 0x7f0294071170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f029c005950 tx=0x7f029c0058e0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 msgr2=0x7f02ac19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02a000bd70 tx=0x7f02a000be50 comp rx=0 tx=0).stop 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 shutdown_connections 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f029406ecc0 0x7f0294071170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02ac106b80 0x7f02ac19c660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 --2- 172.21.15.90:0/2553664761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02ac19cba0 0x7f02ac1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 >> 172.21.15.90:0/2553664761 conn(0x7f02ac076060 msgr2=0x7f02ac077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 shutdown_connections 2024-04-24T08:33:24.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:24.896+0000 7f02b3a00700 1 -- 172.21.15.90:0/2553664761 wait complete. 2024-04-24T08:33:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:24 smithi149 ceph-mon[66490]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:33:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:24 smithi149 ceph-mon[66490]: osdmap e68: 6 total, 5 up, 6 in 2024-04-24T08:33:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:24 smithi149 ceph-mon[66490]: from='osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:33:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:24 smithi149 ceph-mon[66490]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:33:25.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:24 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2356467554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:33:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:24 smithi090 ceph-mon[85282]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:33:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:24 smithi090 ceph-mon[85282]: osdmap e68: 6 total, 5 up, 6 in 2024-04-24T08:33:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:24 smithi090 ceph-mon[85282]: from='osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:33:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:24 smithi090 ceph-mon[85282]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:33:25.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:24 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2356467554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:33:26.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:26.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:33:26.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090] boot 2024-04-24T08:33:26.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T08:33:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:33:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2553664761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:33:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: pgmap v147: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 59/267 objects degraded (22.097%) 2024-04-24T08:33:26.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:25 smithi149 ceph-mon[66490]: osdmap e70: 6 total, 6 up, 6 in 2024-04-24T08:33:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:33:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: osd.3 [v2:172.21.15.149:6800/4083464090,v1:172.21.15.149:6801/4083464090] boot 2024-04-24T08:33:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T08:33:26.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:33:26.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2553664761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:33:26.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: pgmap v147: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 59/267 objects degraded (22.097%) 2024-04-24T08:33:26.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:25 smithi090 ceph-mon[85282]: osdmap e70: 6 total, 6 up, 6 in 2024-04-24T08:33:27.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:27 smithi149 ceph-mon[66490]: pgmap v149: 65 pgs: 3 peering, 14 active+undersized+degraded, 13 active+undersized, 35 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:33:27.480 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:27 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 14 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:27.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:27 smithi090 ceph-mon[85282]: pgmap v149: 65 pgs: 3 peering, 14 active+undersized+degraded, 13 active+undersized, 35 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:33:27.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:27 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 14 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T08:33:29.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:29 smithi149 ceph-mon[66490]: pgmap v150: 65 pgs: 3 peering, 4 active+undersized+degraded, 3 active+undersized, 55 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 14/267 objects degraded (5.243%) 2024-04-24T08:33:29.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T08:33:29.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:29 smithi090 ceph-mon[85282]: pgmap v150: 65 pgs: 3 peering, 4 active+undersized+degraded, 3 active+undersized, 55 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 14/267 objects degraded (5.243%) 2024-04-24T08:33:31.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:31.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:32 smithi149 ceph-mon[66490]: pgmap v151: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-04-24T08:33:32.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:32 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/267 objects degraded (5.243%), 4 pgs degraded) 2024-04-24T08:33:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:32 smithi090 ceph-mon[85282]: pgmap v151: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-04-24T08:33:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:32 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/267 objects degraded (5.243%), 4 pgs degraded) 2024-04-24T08:33:34.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:34 smithi149 ceph-mon[66490]: pgmap v152: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 741 B/s rd, 1 op/s 2024-04-24T08:33:34.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:34 smithi090 ceph-mon[85282]: pgmap v152: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 741 B/s rd, 1 op/s 2024-04-24T08:33:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:36 smithi149 ceph-mon[66490]: pgmap v153: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-24T08:33:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:36 smithi090 ceph-mon[85282]: pgmap v153: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-24T08:33:38.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:38 smithi149 ceph-mon[66490]: pgmap v154: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 998 B/s rd, 1 op/s 2024-04-24T08:33:38.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:38 smithi090 ceph-mon[85282]: pgmap v154: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 998 B/s rd, 1 op/s 2024-04-24T08:33:40.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:40 smithi149 ceph-mon[66490]: pgmap v155: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-24T08:33:40.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:40 smithi090 ceph-mon[85282]: pgmap v155: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-24T08:33:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:41 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:41.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:41 smithi149 ceph-mon[66490]: pgmap v156: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:33:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:41 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:41.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:41 smithi090 ceph-mon[85282]: pgmap v156: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:33:44.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:44 smithi149 ceph-mon[66490]: pgmap v157: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:33:44.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:44 smithi090 ceph-mon[85282]: pgmap v157: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:33:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:45 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:45 smithi149 ceph-mon[66490]: Upgrade: osd.4 is safe to restart 2024-04-24T08:33:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:45 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T08:33:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:45 smithi090 ceph-mon[85282]: Upgrade: osd.4 is safe to restart 2024-04-24T08:33:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:33:46.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:46 smithi149 ceph-mon[66490]: pgmap v158: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:33:46.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:46.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:33:46.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:46 smithi090 ceph-mon[85282]: pgmap v158: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:33:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:33:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:33:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:33:47.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:47 smithi149 ceph-mon[66490]: Upgrade: Updating osd.4 2024-04-24T08:33:47.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:47 smithi149 ceph-mon[66490]: Deploying daemon osd.4 on smithi149 2024-04-24T08:33:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:47 smithi090 ceph-mon[85282]: Upgrade: Updating osd.4 2024-04-24T08:33:47.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:47 smithi090 ceph-mon[85282]: Deploying daemon osd.4 on smithi149 2024-04-24T08:33:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:48 smithi149 ceph-mon[66490]: pgmap v159: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:33:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:48 smithi090 ceph-mon[85282]: pgmap v159: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:33:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:50 smithi149 ceph-mon[66490]: pgmap v160: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:33:50.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:50 smithi090 ceph-mon[85282]: pgmap v160: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:33:51.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:51 smithi149 systemd[1]: Stopping Ceph osd.4 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:33:51.777 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:51 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[42011]: 2024-04-24T08:33:51.504+0000 7f5d54000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:33:51.777 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:51 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[42011]: 2024-04-24T08:33:51.504+0000 7f5d54000700 -1 osd.4 70 *** Got signal Terminated *** 2024-04-24T08:33:51.777 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:51 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[42011]: 2024-04-24T08:33:51.504+0000 7f5d54000700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:33:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:52 smithi149 ceph-mon[66490]: pgmap v161: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:33:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:52 smithi149 ceph-mon[66490]: osd.4 marked itself down and dead 2024-04-24T08:33:52.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:52 smithi090 ceph-mon[85282]: pgmap v161: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:33:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:52 smithi090 ceph-mon[85282]: osd.4 marked itself down and dead 2024-04-24T08:33:53.131 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:52 smithi149 podman[81034]: 2024-04-24 08:33:52.813310148 +0000 UTC m=+1.436213865 container died c68d908da974c9df3402b8591cadecb83f20215bd4e10eac612e23e155f499d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T08:33:53.509 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:53 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:33:53.509 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:53 smithi149 ceph-mon[66490]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T08:33:53.509 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:53 smithi149 podman[81034]: 2024-04-24 08:33:53.307781879 +0000 UTC m=+1.930685627 container cleanup c68d908da974c9df3402b8591cadecb83f20215bd4e10eac612e23e155f499d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD) 2024-04-24T08:33:53.510 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:53 smithi149 bash[81034]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4 2024-04-24T08:33:53.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:53 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:33:53.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:53 smithi090 ceph-mon[85282]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T08:33:53.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:53 smithi149 podman[81053]: 2024-04-24 08:33:53.508509849 +0000 UTC m=+0.676066169 container remove c68d908da974c9df3402b8591cadecb83f20215bd4e10eac612e23e155f499d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:33:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:54 smithi149 ceph-mon[66490]: pgmap v163: 65 pgs: 6 stale+active+clean, 59 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:33:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:54 smithi149 ceph-mon[66490]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T08:33:54.527 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:54 smithi149 podman[81143]: 2024-04-24 08:33:54.264344895 +0000 UTC m=+0.039876077 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:54.527 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:54 smithi149 podman[81143]: 2024-04-24 08:33:54.426192447 +0000 UTC m=+0.201723650 container create 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T08:33:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:54 smithi090 ceph-mon[85282]: pgmap v163: 65 pgs: 6 stale+active+clean, 59 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:33:54.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:54 smithi090 ceph-mon[85282]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T08:33:54.863 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:54 smithi149 podman[81143]: 2024-04-24 08:33:54.739994152 +0000 UTC m=+0.515525385 container init 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T08:33:54.863 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:54 smithi149 podman[81143]: 2024-04-24 08:33:54.753517259 +0000 UTC m=+0.529048433 container start 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T08:33:54.863 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:54 smithi149 podman[81143]: 2024-04-24 08:33:54.862159779 +0000 UTC m=+0.637691025 container attach 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T08:33:55.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.068+0000 7ff73a000700 1 -- 172.21.15.90:0/59028005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 msgr2=0x7ff734106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.068+0000 7ff73a000700 1 --2- 172.21.15.90:0/59028005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff734106ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff720009910 tx=0x7ff720009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:55.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 -- 172.21.15.90:0/59028005 shutdown_connections 2024-04-24T08:33:55.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 --2- 172.21.15.90:0/59028005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff734106ff0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 --2- 172.21.15.90:0/59028005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 0x7ff73410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 -- 172.21.15.90:0/59028005 >> 172.21.15.90:0/59028005 conn(0x7ff734076060 msgr2=0x7ff734078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 -- 172.21.15.90:0/59028005 shutdown_connections 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.069+0000 7ff73a000700 1 -- 172.21.15.90:0/59028005 wait complete. 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.070+0000 7ff73a000700 1 Processor -- start 2024-04-24T08:33:55.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.070+0000 7ff73a000700 1 -- start start 2024-04-24T08:33:55.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff73a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff73a000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 0x7ff73419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff73a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73419d060 con 0x7ff734106b80 2024-04-24T08:33:55.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff73a000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341a0df0 con 0x7ff73410c8f0 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff733400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff733400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40890/0 (socket says 172.21.15.90:40890) 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff733400700 1 -- 172.21.15.90:0/2106573475 learned_addr learned my addr 172.21.15.90:0/2106573475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.071+0000 7ff732a00700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 0x7ff73419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff733400700 1 -- 172.21.15.90:0/2106573475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 msgr2=0x7ff73419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff733400700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 0x7ff73419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff733400700 1 -- 172.21.15.90:0/2106573475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200095f0 con 0x7ff734106b80 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff733400700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff72400c8a0 tx=0x7ff72400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72400cea0 con 0x7ff734106b80 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff724004830 con 0x7ff734106b80 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7240055b0 con 0x7ff734106b80 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7341a10d0 con 0x7ff734106b80 2024-04-24T08:33:55.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.072+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7341a15d0 con 0x7ff734106b80 2024-04-24T08:33:55.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.073+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73410ecc0 con 0x7ff734106b80 2024-04-24T08:33:55.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.075+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff7240049a0 con 0x7ff734106b80 2024-04-24T08:33:55.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.076+0000 7ff72be00700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 0x7ff718071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.076+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff72408eb90 con 0x7ff734106b80 2024-04-24T08:33:55.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.076+0000 7ff732a00700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 0x7ff718071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.077+0000 7ff732a00700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 0x7ff718071170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff73419da60 tx=0x7ff72001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.081+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff724057900 con 0x7ff734106b80 2024-04-24T08:33:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.328+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7341a18a0 con 0x7ff71806ecc0 2024-04-24T08:33:55.329 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:55 smithi090 ceph-mon[85282]: pgmap v165: 65 pgs: 9 active+undersized+degraded, 8 peering, 2 stale+active+clean, 10 active+undersized, 36 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 29/267 objects degraded (10.861%) 2024-04-24T08:33:55.329 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:55 smithi090 ceph-mon[85282]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T08:33:55.330 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:55 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:55.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.331+0000 7ff72be00700 1 -- 172.21.15.90:0/2106573475 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff7341a18a0 con 0x7ff71806ecc0 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 msgr2=0x7ff718071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 0x7ff718071170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff73419da60 tx=0x7ff72001a040 comp rx=0 tx=0).stop 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 msgr2=0x7ff73419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff72400c8a0 tx=0x7ff72400cc60 comp rx=0 tx=0).stop 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 shutdown_connections 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff71806ecc0 0x7ff718071170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff734106b80 0x7ff73419c440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 --2- 172.21.15.90:0/2106573475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73410c8f0 0x7ff73419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.334+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 >> 172.21.15.90:0/2106573475 conn(0x7ff734076060 msgr2=0x7ff734077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:55.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.335+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 shutdown_connections 2024-04-24T08:33:55.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.335+0000 7ff73a000700 1 -- 172.21.15.90:0/2106573475 wait complete. 2024-04-24T08:33:55.351 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 -- 172.21.15.90:0/2955418489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 msgr2=0x7f3bb8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 --2- 172.21.15.90:0/2955418489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb8106e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009910 tx=0x7f3ba8009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 -- 172.21.15.90:0/2955418489 shutdown_connections 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 --2- 172.21.15.90:0/2955418489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb8106e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 --2- 172.21.15.90:0/2955418489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 0x7f3bb81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 -- 172.21.15.90:0/2955418489 >> 172.21.15.90:0/2955418489 conn(0x7f3bb80fdd50 msgr2=0x7f3bb8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:55.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 -- 172.21.15.90:0/2955418489 shutdown_connections 2024-04-24T08:33:55.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.480+0000 7f3bbf200700 1 -- 172.21.15.90:0/2955418489 wait complete. 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.481+0000 7f3bbf200700 1 Processor -- start 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.481+0000 7f3bbf200700 1 -- start start 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbf200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 0x7f3bb8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbf200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8198b90 con 0x7f3bb8102e70 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbf200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819c920 con 0x7f3bb81024d0 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40910/0 (socket says 172.21.15.90:40910) 2024-04-24T08:33:55.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.482+0000 7f3bbd400700 1 -- 172.21.15.90:0/2246160621 learned_addr learned my addr 172.21.15.90:0/2246160621 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:55.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbde00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 0x7f3bb8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbd400700 1 -- 172.21.15.90:0/2246160621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 msgr2=0x7f3bb8197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbd400700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 0x7f3bb8197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbd400700 1 -- 172.21.15.90:0/2246160621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80095f0 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbd400700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80092e0 tx=0x7f3ba800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba801d070 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba800f460 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbf200700 1 -- 172.21.15.90:0/2246160621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb819cba0 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba80175f0 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.483+0000 7f3bbf200700 1 -- 172.21.15.90:0/2246160621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb819d090 con 0x7f3bb8102e70 2024-04-24T08:33:55.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.485+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b940052f0 con 0x7f3bb8102e70 2024-04-24T08:33:55.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.486+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ba800f5d0 con 0x7f3bb8102e70 2024-04-24T08:33:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.486+0000 7f3bb2a00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 0x7f3ba0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.487+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3ba80902a0 con 0x7f3bb8102e70 2024-04-24T08:33:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.487+0000 7f3bbde00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 0x7f3ba0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.487+0000 7f3bbde00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 0x7f3ba0071170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006fd0 tx=0x7f3bac008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.491+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ba8059130 con 0x7f3bb8102e70 2024-04-24T08:33:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:55 smithi149 ceph-mon[66490]: pgmap v165: 65 pgs: 9 active+undersized+degraded, 8 peering, 2 stale+active+clean, 10 active+undersized, 36 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 29/267 objects degraded (10.861%) 2024-04-24T08:33:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:55 smithi149 ceph-mon[66490]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T08:33:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:55 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T08:33:55.527 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:55 smithi149 podman[81143]: 2024-04-24 08:33:55.2032439 +0000 UTC m=+0.978775153 container died 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T08:33:55.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.738+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b94000bc0 con 0x7f3ba006ecc0 2024-04-24T08:33:55.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.740+0000 7f3bb2a00700 1 -- 172.21.15.90:0/2246160621 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3b94000bc0 con 0x7f3ba006ecc0 2024-04-24T08:33:55.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 msgr2=0x7f3ba0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 0x7f3ba0071170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006fd0 tx=0x7f3bac008040 comp rx=0 tx=0).stop 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 msgr2=0x7f3bb81984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80092e0 tx=0x7f3ba800be20 comp rx=0 tx=0).stop 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 shutdown_connections 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ba006ecc0 0x7f3ba0071170 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bb81024d0 0x7f3bb8197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 --2- 172.21.15.90:0/2246160621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bb8102e70 0x7f3bb81984b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 >> 172.21.15.90:0/2246160621 conn(0x7f3bb80fdd50 msgr2=0x7f3bb80ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 shutdown_connections 2024-04-24T08:33:55.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.743+0000 7f3ba7e00700 1 -- 172.21.15.90:0/2246160621 wait complete. 2024-04-24T08:33:55.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 -- 172.21.15.90:0/3790638443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 msgr2=0x7f640c10c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 --2- 172.21.15.90:0/3790638443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c10c360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009910 tx=0x7f63f8009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:55.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 -- 172.21.15.90:0/3790638443 shutdown_connections 2024-04-24T08:33:55.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 --2- 172.21.15.90:0/3790638443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c10c360 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 --2- 172.21.15.90:0/3790638443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 0x7f640c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 -- 172.21.15.90:0/3790638443 >> 172.21.15.90:0/3790638443 conn(0x7f640c0fbb60 msgr2=0x7f640c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:55.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 -- 172.21.15.90:0/3790638443 shutdown_connections 2024-04-24T08:33:55.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.893+0000 7f6412600700 1 -- 172.21.15.90:0/3790638443 wait complete. 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.894+0000 7f6412600700 1 Processor -- start 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f6412600700 1 -- start start 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f6412600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 0x7f640c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f6412600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f6412600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c198940 con 0x7f640c073a00 2024-04-24T08:33:55.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f6412600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c19c6d0 con 0x7f640c0730f0 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f640be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f640be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40926/0 (socket says 172.21.15.90:40926) 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.895+0000 7f640be00700 1 -- 172.21.15.90:0/1518646262 learned_addr learned my addr 172.21.15.90:0/1518646262 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6411200700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 0x7f640c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f640be00700 1 -- 172.21.15.90:0/1518646262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 msgr2=0x7f640c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f640be00700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 0x7f640c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f640be00700 1 -- 172.21.15.90:0/1518646262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80095f0 con 0x7f640c073a00 2024-04-24T08:33:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f640be00700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b5c0 tx=0x7f63f800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f801d070 con 0x7f640c073a00 2024-04-24T08:33:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63f800f460 con 0x7f640c073a00 2024-04-24T08:33:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c19c9b0 con 0x7f640c073a00 2024-04-24T08:33:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f8017630 con 0x7f640c073a00 2024-04-24T08:33:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.896+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c19cf00 con 0x7f640c073a00 2024-04-24T08:33:55.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.898+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c109a60 con 0x7f640c073a00 2024-04-24T08:33:55.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.899+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f63f800f5d0 con 0x7f640c073a00 2024-04-24T08:33:55.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.900+0000 7f6409600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 0x7f63f4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:55.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.900+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f63f8090360 con 0x7f640c073a00 2024-04-24T08:33:55.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.900+0000 7f6411200700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 0x7f63f4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:55.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.901+0000 7f6411200700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 0x7f63f4071170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f640000ba60 tx=0x7f6400005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:55.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:55.905+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f63f8059150 con 0x7f640c073a00 2024-04-24T08:33:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.128+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f640c19d250 con 0x7f63f406ecc0 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.146+0000 7f6409600700 1 -- 172.21.15.90:0/1518646262 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f640c19d250 con 0x7f63f406ecc0 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (6m) 64s ago 20m 13.8M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (21m) 64s ago 21m 8942k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (19m) 35s ago 19m 8468k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:33:56.147 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (3m) 64s ago 21m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (3m) 35s ago 19m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (6m) 64s ago 20m 74.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (13m) 64s ago 13m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (13m) 64s ago 13m 18.0M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (13m) 35s ago 13m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (13m) 35s ago 13m 18.0M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (9m) 64s ago 22m 507M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (8m) 35s ago 19m 451M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (4m) 64s ago 22m 49.8M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (4m) 35s ago 19m 39.2M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 64s ago 21m 7922k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 35s ago 19m 7994k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:33:56.148 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (2m) 64s ago 18m 88.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (104s) 64s ago 17m 83.9M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (72s) 64s ago 17m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (40s) 35s ago 16m 9.92M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (16m) 35s ago 16m 222M 4096M 18.2.0 dc2bc1663786 c68d908da974 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (15m) 35s ago 15m 203M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:33:56.149 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 64s ago 20m 69.4M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:33:56.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 msgr2=0x7f63f4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 0x7f63f4071170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f640000ba60 tx=0x7f6400005d50 comp rx=0 tx=0).stop 2024-04-24T08:33:56.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 msgr2=0x7f640c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b5c0 tx=0x7f63f800be00 comp rx=0 tx=0).stop 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 shutdown_connections 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63f406ecc0 0x7f63f4071170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.149+0000 7f6412600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f640c0730f0 0x7f640c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.150+0000 7f6412600700 1 --2- 172.21.15.90:0/1518646262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c073a00 0x7f640c198260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.150+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 >> 172.21.15.90:0/1518646262 conn(0x7f640c0fbb60 msgr2=0x7f640c106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.150+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 shutdown_connections 2024-04-24T08:33:56.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.150+0000 7f6412600700 1 -- 172.21.15.90:0/1518646262 wait complete. 2024-04-24T08:33:56.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 podman[81169]: 2024-04-24 08:33:56.023989029 +0000 UTC m=+0.801235910 container remove 3c03c4eae848645322f6cf9ec26e5be10a32e657793cd8a8102a8625e25cf061 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T08:33:56.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-24T08:33:56.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 systemd[1]: Stopped Ceph osd.4 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:33:56.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.4.service: Consumed 1min 37.757s CPU time. 2024-04-24T08:33:56.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 -- 172.21.15.90:0/3500131684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 msgr2=0x7f63cc106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 --2- 172.21.15.90:0/3500131684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc106e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009910 tx=0x7f63b4009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 -- 172.21.15.90:0/3500131684 shutdown_connections 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 --2- 172.21.15.90:0/3500131684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc106e40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 --2- 172.21.15.90:0/3500131684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 0x7f63cc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 -- 172.21.15.90:0/3500131684 >> 172.21.15.90:0/3500131684 conn(0x7f63cc0fdd50 msgr2=0x7f63cc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 -- 172.21.15.90:0/3500131684 shutdown_connections 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.301+0000 7f63d0a00700 1 -- 172.21.15.90:0/3500131684 wait complete. 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 Processor -- start 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 -- start start 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 0x7f63cc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc198a00 con 0x7f63cc102e70 2024-04-24T08:33:56.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.302+0000 7f63d0a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc19c790 con 0x7f63cc1024d0 2024-04-24T08:33:56.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40946/0 (socket says 172.21.15.90:40946) 2024-04-24T08:33:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 -- 172.21.15.90:0/1078478492 learned_addr learned my addr 172.21.15.90:0/1078478492 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63cb400700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 0x7f63cc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 -- 172.21.15.90:0/1078478492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 msgr2=0x7f63cc197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 0x7f63cc197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 -- 172.21.15.90:0/1078478492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40095f0 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.303+0000 7f63c3e00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005f50 tx=0x7f63b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.304+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b401d070 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.304+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b400f460 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.304+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b4017630 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.304+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc19ca10 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.304+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc19cf00 con 0x7f63cc102e70 2024-04-24T08:33:56.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.305+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc10a840 con 0x7f63cc102e70 2024-04-24T08:33:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.306+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f63b400f5d0 con 0x7f63cc102e70 2024-04-24T08:33:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.307+0000 7f63c8c00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 0x7f63b0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.307+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f63b40903c0 con 0x7f63cc102e70 2024-04-24T08:33:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.307+0000 7f63cb400700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 0x7f63b0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.308+0000 7f63cb400700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 0x7f63b0071170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63bc005fd0 tx=0x7f63bc00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.311+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f63b40591b0 con 0x7f63cc102e70 2024-04-24T08:33:56.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:56 smithi090 ceph-mon[85282]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:56 smithi090 ceph-mon[85282]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:56 smithi090 ceph-mon[85282]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.611+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f63cc062380 con 0x7f63cc102e70 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.612+0000 7f63c8c00700 1 -- 172.21.15.90:0/1078478492 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f63b405c870 con 0x7f63cc102e70 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:56.613 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 8 2024-04-24T08:33:56.614 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:33:56.615 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 msgr2=0x7f63b0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 0x7f63b0071170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63bc005fd0 tx=0x7f63bc00a560 comp rx=0 tx=0).stop 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 msgr2=0x7f63cc198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005f50 tx=0x7f63b400be20 comp rx=0 tx=0).stop 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 shutdown_connections 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f63b006ecc0 0x7f63b0071170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63cc1024d0 0x7f63cc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 --2- 172.21.15.90:0/1078478492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63cc102e70 0x7f63cc198320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 >> 172.21.15.90:0/1078478492 conn(0x7f63cc0fdd50 msgr2=0x7f63cc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:56.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 shutdown_connections 2024-04-24T08:33:56.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.616+0000 7f63d0a00700 1 -- 172.21.15.90:0/1078478492 wait complete. 2024-04-24T08:33:56.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:56 smithi149 ceph-mon[66490]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:56 smithi149 ceph-mon[66490]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:56 smithi149 ceph-mon[66490]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:56.777 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 systemd[1]: Starting Ceph osd.4 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:33:56.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 -- 172.21.15.90:0/865359746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d8100e60 msgr2=0x7f91d8104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 --2- 172.21.15.90:0/865359746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d8100e60 0x7f91d8104e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009910 tx=0x7f91c4009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 -- 172.21.15.90:0/865359746 shutdown_connections 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 --2- 172.21.15.90:0/865359746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d8100e60 0x7f91d8104e30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 --2- 172.21.15.90:0/865359746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d81004c0 0x7f91d8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 -- 172.21.15.90:0/865359746 >> 172.21.15.90:0/865359746 conn(0x7f91d80fbd80 msgr2=0x7f91d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 -- 172.21.15.90:0/865359746 shutdown_connections 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.780+0000 7f91de600700 1 -- 172.21.15.90:0/865359746 wait complete. 2024-04-24T08:33:56.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.781+0000 7f91de600700 1 Processor -- start 2024-04-24T08:33:56.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.781+0000 7f91de600700 1 -- start start 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91de600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91de600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d8100e60 0x7f91d8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91de600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8073ad0 con 0x7f91d81004c0 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91de600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8073c10 con 0x7f91d8100e60 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40956/0 (socket says 172.21.15.90:40956) 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 -- 172.21.15.90:0/385970398 learned_addr learned my addr 172.21.15.90:0/385970398 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d6a00700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d8100e60 0x7f91d8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 -- 172.21.15.90:0/385970398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d8100e60 msgr2=0x7f91d8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d8100e60 0x7f91d8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.782+0000 7f91d7400700 1 -- 172.21.15.90:0/385970398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40095f0 con 0x7f91d81004c0 2024-04-24T08:33:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91d7400700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b4b0 tx=0x7f91c800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91c8011800 con 0x7f91d81004c0 2024-04-24T08:33:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d81a2200 con 0x7f91d81004c0 2024-04-24T08:33:56.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91c8011e20 con 0x7f91d81004c0 2024-04-24T08:33:56.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91c800f2e0 con 0x7f91d81004c0 2024-04-24T08:33:56.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.783+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d81a2610 con 0x7f91d81004c0 2024-04-24T08:33:56.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.784+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8108760 con 0x7f91d81004c0 2024-04-24T08:33:56.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.785+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f91c8010380 con 0x7f91d81004c0 2024-04-24T08:33:56.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.786+0000 7f91cfe00700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 0x7f91bc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:56.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.786+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f91c808e930 con 0x7f91d81004c0 2024-04-24T08:33:56.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.787+0000 7f91d6a00700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 0x7f91bc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:56.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.787+0000 7f91d6a00700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 0x7f91bc071170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91c4000c00 tx=0x7f91c4005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:56.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:56.791+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f91c8057870 con 0x7f91d81004c0 2024-04-24T08:33:57.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.055+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f91d804f2a0 con 0x7f91d81004c0 2024-04-24T08:33:57.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.056+0000 7f91cfe00700 1 -- 172.21.15.90:0/385970398 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f91c805af30 con 0x7f91d81004c0 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:33:57.058 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:33:57.059 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:57.060 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:57.061 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:57.061 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:33:57.061 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:33:57.061 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:57.061 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 msgr2=0x7f91bc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 0x7f91bc071170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91c4000c00 tx=0x7f91c4005fd0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 msgr2=0x7f91d8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b4b0 tx=0x7f91c800b870 comp rx=0 tx=0).stop 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 shutdown_connections 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f91bc06ecc0 0x7f91bc071170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91d81004c0 0x7f91d8073050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 --2- 172.21.15.90:0/385970398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91d8100e60 0x7f91d8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.059+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 >> 172.21.15.90:0/385970398 conn(0x7f91d80fbd80 msgr2=0x7f91d80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:57.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.060+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 shutdown_connections 2024-04-24T08:33:57.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.060+0000 7f91de600700 1 -- 172.21.15.90:0/385970398 wait complete. 2024-04-24T08:33:57.063 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:33:57.088 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 podman[81269]: 2024-04-24 08:33:56.726900291 +0000 UTC m=+0.032738362 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:57.088 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:56 smithi149 podman[81269]: 2024-04-24 08:33:56.833397275 +0000 UTC m=+0.139235373 container create 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T08:33:57.088 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:57 smithi149 podman[81269]: 2024-04-24 08:33:57.087528514 +0000 UTC m=+0.393366622 container init 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 -- 172.21.15.90:0/3126976799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 msgr2=0x7efe9810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 --2- 172.21.15.90:0/3126976799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9810ccc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009910 tx=0x7efe8c009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 -- 172.21.15.90:0/3126976799 shutdown_connections 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 --2- 172.21.15.90:0/3126976799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 0x7efe98106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 --2- 172.21.15.90:0/3126976799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9810ccc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 -- 172.21.15.90:0/3126976799 >> 172.21.15.90:0/3126976799 conn(0x7efe98076060 msgr2=0x7efe98078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 -- 172.21.15.90:0/3126976799 shutdown_connections 2024-04-24T08:33:57.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.213+0000 7efea0800700 1 -- 172.21.15.90:0/3126976799 wait complete. 2024-04-24T08:33:57.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.214+0000 7efea0800700 1 Processor -- start 2024-04-24T08:33:57.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.214+0000 7efea0800700 1 -- start start 2024-04-24T08:33:57.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.214+0000 7efea0800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 0x7efe9819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efea0800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efea0800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9819d120 con 0x7efe9810c8f0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efea0800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe981a0eb0 con 0x7efe98106b80 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40966/0 (socket says 172.21.15.90:40966) 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 -- 172.21.15.90:0/2563189273 learned_addr learned my addr 172.21.15.90:0/2563189273 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9de00700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 0x7efe9819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 -- 172.21.15.90:0/2563189273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 msgr2=0x7efe9819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 0x7efe9819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.215+0000 7efe9d400700 1 -- 172.21.15.90:0/2563189273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0095f0 con 0x7efe9810c8f0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efe9d400700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c8a0 tx=0x7efe8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8800cea0 con 0x7efe9810c8f0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe88004830 con 0x7efe9810c8f0 2024-04-24T08:33:57.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe981a1190 con 0x7efe9810c8f0 2024-04-24T08:33:57.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe880055b0 con 0x7efe9810c8f0 2024-04-24T08:33:57.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.216+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe981a16e0 con 0x7efe9810c8f0 2024-04-24T08:33:57.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.217+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe98109b20 con 0x7efe9810c8f0 2024-04-24T08:33:57.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.218+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efe880049a0 con 0x7efe9810c8f0 2024-04-24T08:33:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.219+0000 7efe92a00700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 0x7efe80071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.219+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efe8808eb90 con 0x7efe9810c8f0 2024-04-24T08:33:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.219+0000 7efe9de00700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 0x7efe80071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.220+0000 7efe9de00700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 0x7efe80071170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:57.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.224+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efe88057900 con 0x7efe9810c8f0 2024-04-24T08:33:57.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:57 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1078478492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:57.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:57 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/385970398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:33:57.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:57 smithi149 ceph-mon[66490]: pgmap v166: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 36/267 objects degraded (13.483%) 2024-04-24T08:33:57.364 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:57 smithi149 podman[81269]: 2024-04-24 08:33:57.099684261 +0000 UTC m=+0.405522365 container start 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:33:57.364 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:57 smithi149 podman[81269]: 2024-04-24 08:33:57.177649285 +0000 UTC m=+0.483487436 container attach 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:33:57.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.462+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe981a19d0 con 0x7efe8006ecc0 2024-04-24T08:33:57.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.464+0000 7efe92a00700 1 -- 172.21.15.90:0/2563189273 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7efe981a19d0 con 0x7efe8006ecc0 2024-04-24T08:33:57.465 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:33:57.465 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "10/23 daemons upgraded", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:33:57.466 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:33:57.467 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:33:57.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 msgr2=0x7efe80071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 0x7efe80071170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00b540 comp rx=0 tx=0).stop 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 msgr2=0x7efe9819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c8a0 tx=0x7efe8800cc60 comp rx=0 tx=0).stop 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 shutdown_connections 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7efe8006ecc0 0x7efe80071170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe98106b80 0x7efe9819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 --2- 172.21.15.90:0/2563189273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810c8f0 0x7efe9819ca40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.467+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 >> 172.21.15.90:0/2563189273 conn(0x7efe98076060 msgr2=0x7efe98077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.468+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 shutdown_connections 2024-04-24T08:33:57.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.468+0000 7efea0800700 1 -- 172.21.15.90:0/2563189273 wait complete. 2024-04-24T08:33:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:57 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1078478492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:33:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:57 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/385970398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:33:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:57 smithi090 ceph-mon[85282]: pgmap v166: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 36/267 objects degraded (13.483%) 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 -- 172.21.15.90:0/3362403101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d810c660 msgr2=0x7f08d810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3362403101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d810c660 0x7f08d810ca30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009910 tx=0x7f08c8009c20 comp rx=0 tx=0).stop 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 -- 172.21.15.90:0/3362403101 shutdown_connections 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3362403101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d8106b80 0x7f08d8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3362403101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d810c660 0x7f08d810ca30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 -- 172.21.15.90:0/3362403101 >> 172.21.15.90:0/3362403101 conn(0x7f08d8075e90 msgr2=0x7f08d80782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 -- 172.21.15.90:0/3362403101 shutdown_connections 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.614+0000 7f08dcc00700 1 -- 172.21.15.90:0/3362403101 wait complete. 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.615+0000 7f08dcc00700 1 Processor -- start 2024-04-24T08:33:57.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.615+0000 7f08dcc00700 1 -- start start 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08dcc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08dcc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d810c660 0x7f08d819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08dcc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d819cea0 con 0x7f08d8106b80 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08dcc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d81a0c30 con 0x7f08d810c660 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40996/0 (socket says 172.21.15.90:40996) 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 -- 172.21.15.90:0/3131595006 learned_addr learned my addr 172.21.15.90:0/3131595006 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:33:57.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d6a00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d810c660 0x7f08d819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 -- 172.21.15.90:0/3131595006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d810c660 msgr2=0x7f08d819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d810c660 0x7f08d819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.616+0000 7f08d7400700 1 -- 172.21.15.90:0/3131595006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80095f0 con 0x7f08d8106b80 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08d7400700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b5c0 tx=0x7f08c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c801d070 con 0x7f08d8106b80 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c8005e00 con 0x7f08d8106b80 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c8017400 con 0x7f08d8106b80 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d81a0eb0 con 0x7f08d8106b80 2024-04-24T08:33:57.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.617+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d81a13a0 con 0x7f08d8106b80 2024-04-24T08:33:57.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.618+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d810eb60 con 0x7f08d8106b80 2024-04-24T08:33:57.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.619+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f08c800f460 con 0x7f08d8106b80 2024-04-24T08:33:57.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.620+0000 7f08cfe00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 0x7f08bc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:33:57.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.620+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f08c8090310 con 0x7f08d8106b80 2024-04-24T08:33:57.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.620+0000 7f08d6a00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 0x7f08bc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:33:57.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.621+0000 7f08d6a00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 0x7f08bc071170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005950 tx=0x7f08c40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:33:57.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.625+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f08c8059100 con 0x7f08d8106b80 2024-04-24T08:33:57.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.925+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f08d810e540 con 0x7f08d8106b80 2024-04-24T08:33:57.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.926+0000 7f08cfe00700 1 -- 172.21.15.90:0/3131595006 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1333 (secure 0 0 0) 0x7f08c805c7c0 con 0x7f08d8106b80 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering; Degraded data redundancy: 36/267 objects degraded (13.483%), 11 pgs degraded 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout: osd.4 (root=default,host=smithi149) is down 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-04-24T08:33:57.927 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.f is stuck peering for 6m, current state peering, last acting [0,5] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 3.3 is stuck peering for 6m, current state peering, last acting [0,3] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/267 objects degraded (13.483%), 11 pgs degraded 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-04-24T08:33:57.928 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 msgr2=0x7f08bc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 0x7f08bc071170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08c4005950 tx=0x7f08c40058e0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 msgr2=0x7f08d819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b5c0 tx=0x7f08c800be20 comp rx=0 tx=0).stop 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 shutdown_connections 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08bc06ecc0 0x7f08bc071170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08d8106b80 0x7f08d819c280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 --2- 172.21.15.90:0/3131595006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d810c660 0x7f08d819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:33:57.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 >> 172.21.15.90:0/3131595006 conn(0x7f08d8075e90 msgr2=0x7f08d8077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:33:57.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 shutdown_connections 2024-04-24T08:33:57.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:33:57.929+0000 7f08dcc00700 1 -- 172.21.15.90:0/3131595006 wait complete. 2024-04-24T08:33:58.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-mon[66490]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:58.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3131595006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T08:33:58.528 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 bash[81269]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate[81285]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T08:33:58.529 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:58 smithi149 podman[81269]: 2024-04-24 08:33:58.119246524 +0000 UTC m=+1.425084628 container died 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:33:58.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:58 smithi090 ceph-mon[85282]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:33:58.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:58 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3131595006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:33:59.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 podman[81410]: 2024-04-24 08:33:59.006402352 +0000 UTC m=+0.868601824 container remove 9ef3b760a30f17ec435debdd717b5feff5ac56191e25d605be7914a96f0c9995 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:33:59.559 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:33:59 smithi149 ceph-mon[66490]: pgmap v167: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:33:59.559 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 podman[81469]: 2024-04-24 08:33:59.279434941 +0000 UTC m=+0.034402519 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:33:59.559 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 podman[81469]: 2024-04-24 08:33:59.390801435 +0000 UTC m=+0.145769050 container create 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T08:33:59.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:33:59 smithi090 ceph-mon[85282]: pgmap v167: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:00.027 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 podman[81469]: 2024-04-24 08:33:59.719820112 +0000 UTC m=+0.474787737 container init 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T08:34:00.027 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 podman[81469]: 2024-04-24 08:33:59.731588719 +0000 UTC m=+0.486556367 container start 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T08:34:00.027 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 ceph-osd[81488]: -- 172.21.15.149:0/2172693439 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c58005680 con 0x559c57fcec00 2024-04-24T08:34:00.027 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 bash[81469]: 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac 2024-04-24T08:34:00.028 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:33:59 smithi149 systemd[1]: Started Ceph osd.4 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:34:00.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:34:00.308+0000 7f2b1b2eb7c0 -1 Falling back to public interface 2024-04-24T08:34:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T08:34:00.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:00.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T08:34:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:00.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:01 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:01 smithi149 ceph-mon[66490]: pgmap v168: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 805 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:01 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:01 smithi090 ceph-mon[85282]: pgmap v168: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 805 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:04.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:04 smithi149 ceph-mon[66490]: pgmap v169: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:04.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:04 smithi090 ceph-mon[85282]: pgmap v169: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:05.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:34:05 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:34:05.466+0000 7f2b1b2eb7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-24T08:34:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:05 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:05.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:05 smithi149 ceph-mon[66490]: pgmap v170: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:05.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:05 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:05.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:05 smithi090 ceph-mon[85282]: pgmap v170: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:06.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:34:05 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:34:05.885+0000 7f2b1b2eb7c0 -1 osd.4 70 log_to_monitors true 2024-04-24T08:34:06.689 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:06 smithi149 ceph-mon[66490]: from='osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:34:06.689 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:06 smithi149 ceph-mon[66490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:34:06.690 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:06.690 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:06 smithi090 ceph-mon[85282]: from='osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:34:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:06 smithi090 ceph-mon[85282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:34:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:06.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:07.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-mon[66490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:34:07.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-mon[66490]: osdmap e73: 6 total, 5 up, 6 in 2024-04-24T08:34:07.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-mon[66490]: from='osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:07.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-mon[66490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:07.778 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-mon[66490]: pgmap v172: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:07.779 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:34:07 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:34:07.673+0000 7f2b05600700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:34:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:07 smithi090 ceph-mon[85282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:34:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:07 smithi090 ceph-mon[85282]: osdmap e73: 6 total, 5 up, 6 in 2024-04-24T08:34:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:07 smithi090 ceph-mon[85282]: from='osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:07 smithi090 ceph-mon[85282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:07 smithi090 ceph-mon[85282]: pgmap v172: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 46/267 objects degraded (17.228%) 2024-04-24T08:34:08.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:08 smithi149 ceph-mon[66490]: from='osd.4 ' entity='osd.4' 2024-04-24T08:34:09.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:08 smithi090 ceph-mon[85282]: from='osd.4 ' entity='osd.4' 2024-04-24T08:34:10.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:34:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354] boot 2024-04-24T08:34:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T08:34:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:34:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: pgmap v174: 65 pgs: 10 peering, 10 active+undersized+degraded, 14 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 24/267 objects degraded (8.989%) 2024-04-24T08:34:10.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:09 smithi149 ceph-mon[66490]: osdmap e75: 6 total, 6 up, 6 in 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: osd.4 [v2:172.21.15.149:6808/3688072354,v1:172.21.15.149:6809/3688072354] boot 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: pgmap v174: 65 pgs: 10 peering, 10 active+undersized+degraded, 14 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 24/267 objects degraded (8.989%) 2024-04-24T08:34:10.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:09 smithi090 ceph-mon[85282]: osdmap e75: 6 total, 6 up, 6 in 2024-04-24T08:34:11.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:10 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 24/267 objects degraded (8.989%), 10 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:11.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:10 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 24/267 objects degraded (8.989%), 10 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:12.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:11 smithi149 ceph-mon[66490]: pgmap v176: 65 pgs: 17 peering, 5 active+undersized+degraded, 7 active+undersized, 36 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 12/267 objects degraded (4.494%) 2024-04-24T08:34:12.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:11 smithi090 ceph-mon[85282]: pgmap v176: 65 pgs: 17 peering, 5 active+undersized+degraded, 7 active+undersized, 36 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 12/267 objects degraded (4.494%) 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:13 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:13.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:13 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:14.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:14 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:14 smithi149 ceph-mon[66490]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-04-24T08:34:14.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:14 smithi149 ceph-mon[66490]: pgmap v177: 65 pgs: 17 peering, 3 active+undersized+degraded, 3 active+undersized, 42 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 460 B/s rd, 1 op/s; 7/267 objects degraded (2.622%) 2024-04-24T08:34:14.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:14 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:14.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:14 smithi090 ceph-mon[85282]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-04-24T08:34:14.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:14 smithi090 ceph-mon[85282]: pgmap v177: 65 pgs: 17 peering, 3 active+undersized+degraded, 3 active+undersized, 42 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 460 B/s rd, 1 op/s; 7/267 objects degraded (2.622%) 2024-04-24T08:34:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:16 smithi149 ceph-mon[66490]: pgmap v178: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:16.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:16 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:16 smithi090 ceph-mon[85282]: pgmap v178: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:16 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:17.509 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:17 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-24T08:34:17.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:17 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-24T08:34:18.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:18 smithi149 ceph-mon[66490]: pgmap v179: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 967 B/s rd, 1 op/s 2024-04-24T08:34:18.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:18 smithi090 ceph-mon[85282]: pgmap v179: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 967 B/s rd, 1 op/s 2024-04-24T08:34:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:20 smithi149 ceph-mon[66490]: pgmap v180: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:20.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:20 smithi090 ceph-mon[85282]: pgmap v180: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:21.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:21.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:21 smithi149 ceph-mon[66490]: pgmap v181: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 892 B/s rd, 1 op/s 2024-04-24T08:34:21.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:21.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:21 smithi090 ceph-mon[85282]: pgmap v181: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 892 B/s rd, 1 op/s 2024-04-24T08:34:24.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:24 smithi149 ceph-mon[66490]: pgmap v182: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:34:24.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:24 smithi090 ceph-mon[85282]: pgmap v182: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:34:26.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:26 smithi149 ceph-mon[66490]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:26.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:26 smithi090 ceph-mon[85282]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 -- 172.21.15.90:0/658645918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3941012f0 msgr2=0x7fa3941016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 --2- 172.21.15.90:0/658645918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3941012f0 0x7fa3941016c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b0a0 tx=0x7fa38800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 -- 172.21.15.90:0/658645918 shutdown_connections 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 --2- 172.21.15.90:0/658645918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394101c00 0x7fa39410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 --2- 172.21.15.90:0/658645918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3941012f0 0x7fa3941016c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 -- 172.21.15.90:0/658645918 >> 172.21.15.90:0/658645918 conn(0x7fa3940fac80 msgr2=0x7fa3940fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 -- 172.21.15.90:0/658645918 shutdown_connections 2024-04-24T08:34:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.102+0000 7fa39c000700 1 -- 172.21.15.90:0/658645918 wait complete. 2024-04-24T08:34:28.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.103+0000 7fa39c000700 1 Processor -- start 2024-04-24T08:34:28.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.103+0000 7fa39c000700 1 -- start start 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa39c000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3941012f0 0x7fa394198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa39c000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa39c000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394198ce0 con 0x7fa394101c00 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa39c000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394198e20 con 0x7fa3941012f0 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa398c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa398c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58650/0 (socket says 172.21.15.90:58650) 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa398c00700 1 -- 172.21.15.90:0/2081037761 learned_addr learned my addr 172.21.15.90:0/2081037761 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.104+0000 7fa399600700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3941012f0 0x7fa394198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa398c00700 1 -- 172.21.15.90:0/2081037761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3941012f0 msgr2=0x7fa394198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa398c00700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3941012f0 0x7fa394198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa398c00700 1 -- 172.21.15.90:0/2081037761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388009d00 con 0x7fa394101c00 2024-04-24T08:34:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa398c00700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa38000c930 tx=0x7fa38000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa380007ad0 con 0x7fa394101c00 2024-04-24T08:34:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa38000ce80 con 0x7fa394101c00 2024-04-24T08:34:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39419cc70 con 0x7fa394101c00 2024-04-24T08:34:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.105+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa380018630 con 0x7fa394101c00 2024-04-24T08:34:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.106+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39419d1c0 con 0x7fa394101c00 2024-04-24T08:34:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.107+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa394109d70 con 0x7fa394101c00 2024-04-24T08:34:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.108+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa380007c30 con 0x7fa394101c00 2024-04-24T08:34:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.109+0000 7fa392000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 0x7fa37c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.109+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa38008eee0 con 0x7fa394101c00 2024-04-24T08:34:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.110+0000 7fa399600700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 0x7fa37c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.110+0000 7fa399600700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 0x7fa37c071170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa388000f80 tx=0x7fa3880097e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.119+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa380057c50 con 0x7fa394101c00 2024-04-24T08:34:28.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:28 smithi090 ceph-mon[85282]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:34:28.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:28.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.373+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39402cca0 con 0x7fa37c06ecc0 2024-04-24T08:34:28.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.375+0000 7fa392000700 1 -- 172.21.15.90:0/2081037761 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa39402cca0 con 0x7fa37c06ecc0 2024-04-24T08:34:28.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.378+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 msgr2=0x7fa37c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.378+0000 7fa39c000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 0x7fa37c071170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa388000f80 tx=0x7fa3880097e0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.378+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 msgr2=0x7fa394198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.378+0000 7fa39c000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa38000c930 tx=0x7fa38000cc40 comp rx=0 tx=0).stop 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 shutdown_connections 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa37c06ecc0 0x7fa37c071170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3941012f0 0x7fa394198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 --2- 172.21.15.90:0/2081037761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa394101c00 0x7fa394198690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 >> 172.21.15.90:0/2081037761 conn(0x7fa3940fac80 msgr2=0x7fa394106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 shutdown_connections 2024-04-24T08:34:28.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.379+0000 7fa39c000700 1 -- 172.21.15.90:0/2081037761 wait complete. 2024-04-24T08:34:28.395 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:34:28.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:28 smithi149 ceph-mon[66490]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:34:28.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:28.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 -- 172.21.15.90:0/2819769902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075b90 msgr2=0x7f6d48110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2819769902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075b90 0x7f6d48110c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009910 tx=0x7f6d30009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 -- 172.21.15.90:0/2819769902 shutdown_connections 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2819769902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075b90 0x7f6d48110c20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2819769902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48075280 0x7f6d48075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 -- 172.21.15.90:0/2819769902 >> 172.21.15.90:0/2819769902 conn(0x7f6d480fdfa0 msgr2=0x7f6d481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 -- 172.21.15.90:0/2819769902 shutdown_connections 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.540+0000 7f6d4e200700 1 -- 172.21.15.90:0/2819769902 wait complete. 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 Processor -- start 2024-04-24T08:34:28.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 -- start start 2024-04-24T08:34:28.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48075b90 0x7f6d4819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819d150 con 0x7f6d48075280 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.541+0000 7f6d4e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481a0ee0 con 0x7f6d48075b90 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58670/0 (socket says 172.21.15.90:58670) 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 -- 172.21.15.90:0/2634709276 learned_addr learned my addr 172.21.15.90:0/2634709276 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 -- 172.21.15.90:0/2634709276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48075b90 msgr2=0x7f6d4819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48075b90 0x7f6d4819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 -- 172.21.15.90:0/2634709276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300095f0 con 0x7f6d48075280 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.542+0000 7f6d47400700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b4b0 tx=0x7f6d3800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.543+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d38011800 con 0x7f6d48075280 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.543+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d38011e20 con 0x7f6d48075280 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.543+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d3800f2e0 con 0x7f6d48075280 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.543+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d481a11c0 con 0x7f6d48075280 2024-04-24T08:34:28.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.543+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481a1710 con 0x7f6d48075280 2024-04-24T08:34:28.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.544+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4810e390 con 0x7f6d48075280 2024-04-24T08:34:28.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.545+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6d38010380 con 0x7f6d48075280 2024-04-24T08:34:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.546+0000 7f6d44c00700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 0x7f6d2c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.547+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6d3808e9d0 con 0x7f6d48075280 2024-04-24T08:34:28.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.547+0000 7f6d3ea00700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 0x7f6d2c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.548+0000 7f6d3ea00700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 0x7f6d2c071170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b850 tx=0x7f6d3000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.552+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6d38057870 con 0x7f6d48075280 2024-04-24T08:34:28.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.803+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d481a19e0 con 0x7f6d2c06ecc0 2024-04-24T08:34:28.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.805+0000 7f6d44c00700 1 -- 172.21.15.90:0/2634709276 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6d481a19e0 con 0x7f6d2c06ecc0 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 msgr2=0x7f6d2c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 0x7f6d2c071170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b850 tx=0x7f6d3000b540 comp rx=0 tx=0).stop 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 msgr2=0x7f6d4819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b4b0 tx=0x7f6d3800b870 comp rx=0 tx=0).stop 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 shutdown_connections 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6d2c06ecc0 0x7f6d2c071170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d48075280 0x7f6d4819c530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 --2- 172.21.15.90:0/2634709276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48075b90 0x7f6d4819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 >> 172.21.15.90:0/2634709276 conn(0x7f6d480fdfa0 msgr2=0x7f6d480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 shutdown_connections 2024-04-24T08:34:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.808+0000 7f6d4e200700 1 -- 172.21.15.90:0/2634709276 wait complete. 2024-04-24T08:34:28.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.957+0000 7f25b3e00700 1 -- 172.21.15.90:0/1100779465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b4106b80 msgr2=0x7f25b4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.957+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1100779465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b4106b80 0x7f25b4106ff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009910 tx=0x7f25a0009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 -- 172.21.15.90:0/1100779465 shutdown_connections 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1100779465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b4106b80 0x7f25b4106ff0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1100779465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b410c660 0x7f25b410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 -- 172.21.15.90:0/1100779465 >> 172.21.15.90:0/1100779465 conn(0x7f25b4075e90 msgr2=0x7f25b40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 -- 172.21.15.90:0/1100779465 shutdown_connections 2024-04-24T08:34:28.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.958+0000 7f25b3e00700 1 -- 172.21.15.90:0/1100779465 wait complete. 2024-04-24T08:34:28.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.959+0000 7f25b3e00700 1 Processor -- start 2024-04-24T08:34:28.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.959+0000 7f25b3e00700 1 -- start start 2024-04-24T08:34:28.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b4106b80 0x7f25b419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b419cee0 con 0x7f25b410c660 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b41a0c70 con 0x7f25b4106b80 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58694/0 (socket says 172.21.15.90:58694) 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b2000700 1 -- 172.21.15.90:0/1585974220 learned_addr learned my addr 172.21.15.90:0/1585974220 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:28.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.960+0000 7f25b2a00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b4106b80 0x7f25b419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b2000700 1 -- 172.21.15.90:0/1585974220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b4106b80 msgr2=0x7f25b419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b2000700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b4106b80 0x7f25b419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b2000700 1 -- 172.21.15.90:0/1585974220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00095f0 con 0x7f25b410c660 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b2000700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a00092e0 tx=0x7f25a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a001d070 con 0x7f25b410c660 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25a0005e00 con 0x7f25b410c660 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a0017400 con 0x7f25b410c660 2024-04-24T08:34:28.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b41a0ef0 con 0x7f25b410c660 2024-04-24T08:34:28.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.961+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b41a13e0 con 0x7f25b410c660 2024-04-24T08:34:28.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.963+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b410eb40 con 0x7f25b410c660 2024-04-24T08:34:28.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.963+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f25a000f6f0 con 0x7f25b410c660 2024-04-24T08:34:28.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.965+0000 7f25ab400700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 0x7f2598071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:28.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.965+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f25a00901d0 con 0x7f25b410c660 2024-04-24T08:34:28.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.965+0000 7f25b2a00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 0x7f2598071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:28.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.966+0000 7f25b2a00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 0x7f2598071170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25a4004630 tx=0x7f25a400b3c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:28.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:28.970+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f25a0059060 con 0x7f25b410c660 2024-04-24T08:34:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.193+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b402cca0 con 0x7f259806ecc0 2024-04-24T08:34:29.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.205+0000 7f25ab400700 1 -- 172.21.15.90:0/1585974220 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f25b402cca0 con 0x7f259806ecc0 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 97s ago 21m 13.8M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (21m) 97s ago 21m 8942k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (20m) 24s ago 20m 8715k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (3m) 97s ago 21m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:34:29.206 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (3m) 24s ago 20m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (6m) 97s ago 21m 74.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (14m) 97s ago 14m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (14m) 97s ago 14m 18.0M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (14m) 24s ago 14m 152M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (14m) 24s ago 14m 18.3M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (10m) 97s ago 23m 507M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (9m) 24s ago 20m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (5m) 97s ago 23m 49.8M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (4m) 24s ago 19m 39.5M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 97s ago 21m 7922k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (8m) 24s ago 20m 8098k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (3m) 97s ago 19m 88.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (2m) 97s ago 18m 83.9M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:34:29.207 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (105s) 97s ago 17m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:34:29.208 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (73s) 24s ago 17m 90.1M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:34:29.208 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (29s) 24s ago 16m 11.4M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:34:29.208 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (16m) 24s ago 16m 203M 4096M 18.2.0 dc2bc1663786 78ba75820370 2024-04-24T08:34:29.208 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 97s ago 20m 69.4M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 msgr2=0x7f2598071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 0x7f2598071170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25a4004630 tx=0x7f25a400b3c0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 msgr2=0x7f25b419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a00092e0 tx=0x7f25a000bd70 comp rx=0 tx=0).stop 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 shutdown_connections 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f259806ecc0 0x7f2598071170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b4106b80 0x7f25b419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 --2- 172.21.15.90:0/1585974220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25b410c660 0x7f25b419c800 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 >> 172.21.15.90:0/1585974220 conn(0x7f25b4075e90 msgr2=0x7f25b40779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 shutdown_connections 2024-04-24T08:34:29.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.208+0000 7f25b3e00700 1 -- 172.21.15.90:0/1585974220 wait complete. 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 -- 172.21.15.90:0/1110030836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 msgr2=0x7f83fc075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 --2- 172.21.15.90:0/1110030836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc075650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009910 tx=0x7f83ec009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 -- 172.21.15.90:0/1110030836 shutdown_connections 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 --2- 172.21.15.90:0/1110030836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 0x7f83fc110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 --2- 172.21.15.90:0/1110030836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc075650 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 -- 172.21.15.90:0/1110030836 >> 172.21.15.90:0/1110030836 conn(0x7f83fc0fdfa0 msgr2=0x7f83fc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:29.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 -- 172.21.15.90:0/1110030836 shutdown_connections 2024-04-24T08:34:29.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.352+0000 7f8401c00700 1 -- 172.21.15.90:0/1110030836 wait complete. 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 Processor -- start 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 -- start start 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 0x7f83fc19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc19cff0 con 0x7f83fc075280 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f8401c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc1a0d80 con 0x7f83fc075b90 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f83faa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f83faa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58714/0 (socket says 172.21.15.90:58714) 2024-04-24T08:34:29.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.354+0000 7f83faa00700 1 -- 172.21.15.90:0/1940740201 learned_addr learned my addr 172.21.15.90:0/1940740201 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:29.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83fa000700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 0x7f83fc19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83faa00700 1 -- 172.21.15.90:0/1940740201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 msgr2=0x7f83fc19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83faa00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 0x7f83fc19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83faa00700 1 -- 172.21.15.90:0/1940740201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0095f0 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83faa00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0092e0 tx=0x7f83ec0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ec01d070 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83ec00f460 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ec021640 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.355+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc1a1000 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.356+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc1a1550 con 0x7f83fc075280 2024-04-24T08:34:29.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.357+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc10e390 con 0x7f83fc075280 2024-04-24T08:34:29.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.358+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f83ec00f5d0 con 0x7f83fc075280 2024-04-24T08:34:29.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.359+0000 7f83f3400700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 0x7f83e0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.359+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f83ec090450 con 0x7f83fc075280 2024-04-24T08:34:29.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.359+0000 7f83fa000700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 0x7f83e0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.360+0000 7f83fa000700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 0x7f83e0071170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005950 tx=0x7f83e80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:29.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.363+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f83ec0592c0 con 0x7f83fc075280 2024-04-24T08:34:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:29 smithi149 ceph-mon[66490]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:29.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:29 smithi149 ceph-mon[66490]: Upgrade: osd.5 is safe to restart 2024-04-24T08:34:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:29 smithi090 ceph-mon[85282]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T08:34:29.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:29 smithi090 ceph-mon[85282]: Upgrade: osd.5 is safe to restart 2024-04-24T08:34:29.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.684+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83fc1a17c0 con 0x7f83fc075280 2024-04-24T08:34:29.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.685+0000 7f83f3400700 1 -- 172.21.15.90:0/1940740201 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f83ec05c980 con 0x7f83fc075280 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 5 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:34:29.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 9 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:34:29.687 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:34:29.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.687+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 msgr2=0x7f83e0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.687+0000 7f8401c00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 0x7f83e0071170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005950 tx=0x7f83e80058e0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.687+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 msgr2=0x7f83fc19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.687+0000 7f8401c00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0092e0 tx=0x7f83ec0056a0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 shutdown_connections 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83e006ecc0 0x7f83e0071170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc075280 0x7f83fc19c3d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 --2- 172.21.15.90:0/1940740201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83fc075b90 0x7f83fc19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 >> 172.21.15.90:0/1940740201 conn(0x7f83fc0fdfa0 msgr2=0x7f83fc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 shutdown_connections 2024-04-24T08:34:29.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.688+0000 7f8401c00700 1 -- 172.21.15.90:0/1940740201 wait complete. 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 -- 172.21.15.90:0/159109089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 msgr2=0x7f25dc108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 --2- 172.21.15.90:0/159109089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc108840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 -- 172.21.15.90:0/159109089 shutdown_connections 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 --2- 172.21.15.90:0/159109089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 0x7f25dc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 --2- 172.21.15.90:0/159109089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc108840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 -- 172.21.15.90:0/159109089 >> 172.21.15.90:0/159109089 conn(0x7f25dc0fdf90 msgr2=0x7f25dc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:29.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 -- 172.21.15.90:0/159109089 shutdown_connections 2024-04-24T08:34:29.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.836+0000 7f25e1400700 1 -- 172.21.15.90:0/159109089 wait complete. 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.837+0000 7f25e1400700 1 Processor -- start 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25e1400700 1 -- start start 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25e1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 0x7f25dc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25e1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25e1400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc198d80 con 0x7f25dc108470 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25e1400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc198ec0 con 0x7f25dc102470 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56266/0 (socket says 172.21.15.90:56266) 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25d3e00700 1 -- 172.21.15.90:0/4036257168 learned_addr learned my addr 172.21.15.90:0/4036257168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.838+0000 7f25da800700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 0x7f25dc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3e00700 1 -- 172.21.15.90:0/4036257168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 msgr2=0x7f25dc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3e00700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 0x7f25dc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3e00700 1 -- 172.21.15.90:0/4036257168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25dc108470 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3e00700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c930 tx=0x7f25c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c4007ad0 con 0x7f25dc108470 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25c400ce80 con 0x7f25dc108470 2024-04-24T08:34:29.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc19cd10 con 0x7f25dc108470 2024-04-24T08:34:29.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c4018630 con 0x7f25dc108470 2024-04-24T08:34:29.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.839+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc19d260 con 0x7f25dc108470 2024-04-24T08:34:29.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.840+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc10a9e0 con 0x7f25dc108470 2024-04-24T08:34:29.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.841+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f25c4007c30 con 0x7f25dc108470 2024-04-24T08:34:29.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.842+0000 7f25d3400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 0x7f25c0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:29.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.842+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f25c408ee40 con 0x7f25dc108470 2024-04-24T08:34:29.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.843+0000 7f25da800700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 0x7f25c0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:29.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.843+0000 7f25da800700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 0x7f25c0071170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:29.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:29.847+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f25c4057c50 con 0x7f25dc108470 2024-04-24T08:34:30.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.133+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25dc199620 con 0x7f25dc108470 2024-04-24T08:34:30.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.134+0000 7f25d3400700 1 -- 172.21.15.90:0/4036257168 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f25c405b310 con 0x7f25dc108470 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:e17 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 17 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:34:30.135 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:20:31.598807+0000 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 47 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:34:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:34:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:14482} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.90:6828/752337863,v1:172.21.15.90:6829/752337863] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:34:30.138 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 msgr2=0x7f25c0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 0x7f25c0071170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc01a040 comp rx=0 tx=0).stop 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 msgr2=0x7f25dc198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c930 tx=0x7f25c400cc40 comp rx=0 tx=0).stop 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 shutdown_connections 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f25c006ecc0 0x7f25c0071170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc102470 0x7f25dc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 --2- 172.21.15.90:0/4036257168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25dc108470 0x7f25dc198730 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.137+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 >> 172.21.15.90:0/4036257168 conn(0x7f25dc0fdf90 msgr2=0x7f25dc1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.138+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 shutdown_connections 2024-04-24T08:34:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.138+0000 7f25e1400700 1 -- 172.21.15.90:0/4036257168 wait complete. 2024-04-24T08:34:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-24T08:34:30.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 -- 172.21.15.90:0/2639648842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e04100e80 msgr2=0x7f7e04104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/2639648842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e04100e80 0x7f7e04104e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009910 tx=0x7f7df0009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:30.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 -- 172.21.15.90:0/2639648842 shutdown_connections 2024-04-24T08:34:30.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/2639648842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e04100e80 0x7f7e04104e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/2639648842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e041004e0 0x7f7e041008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 -- 172.21.15.90:0/2639648842 >> 172.21.15.90:0/2639648842 conn(0x7f7e040fbda0 msgr2=0x7f7e040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:30.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 -- 172.21.15.90:0/2639648842 shutdown_connections 2024-04-24T08:34:30.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.298+0000 7f7e0bc00700 1 -- 172.21.15.90:0/2639648842 wait complete. 2024-04-24T08:34:30.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.299+0000 7f7e0bc00700 1 Processor -- start 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.299+0000 7f7e0bc00700 1 -- start start 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e0bc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e0bc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 0x7f7e04198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e0bc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04198d00 con 0x7f7e041004e0 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e0bc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0419ca90 con 0x7f7e04100e80 2024-04-24T08:34:30.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56282/0 (socket says 172.21.15.90:56282) 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 -- 172.21.15.90:0/3793091271 learned_addr learned my addr 172.21.15.90:0/3793091271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e03e00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 0x7f7e04198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 -- 172.21.15.90:0/3793091271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 msgr2=0x7f7e04198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 0x7f7e04198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e09200700 1 -- 172.21.15.90:0/3793091271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00095f0 con 0x7f7e041004e0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.300+0000 7f7e03e00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 0x7f7e04198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e09200700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d8d0 tx=0x7f7df800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df8009940 con 0x7f7e041004e0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7df8010460 con 0x7f7e041004e0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df800f5f0 con 0x7f7e041004e0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0419cd70 con 0x7f7e041004e0 2024-04-24T08:34:30.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.301+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0419d2c0 con 0x7f7e041004e0 2024-04-24T08:34:30.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.302+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e041045d0 con 0x7f7e041004e0 2024-04-24T08:34:30.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.303+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7df80105d0 con 0x7f7e041004e0 2024-04-24T08:34:30.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.305+0000 7f7e01600700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 0x7f7dec071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.305+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7df808ed10 con 0x7f7e041004e0 2024-04-24T08:34:30.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.305+0000 7f7e03e00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 0x7f7dec071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.305+0000 7f7e03e00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 0x7f7dec071170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e04199700 tx=0x7f7df001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:30.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.309+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7df8057ba0 con 0x7f7e041004e0 2024-04-24T08:34:30.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1940740201' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/4036257168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:34:30.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:30.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.539+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e0419d530 con 0x7f7dec06ecc0 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1940740201' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:30.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:34:30.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:30.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:30.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/4036257168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:34:30.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:30.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.542+0000 7f7e01600700 1 -- 172.21.15.90:0/3793091271 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7e0419d530 con 0x7f7dec06ecc0 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "11/23 daemons upgraded", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T08:34:30.543 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:34:30.544 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 msgr2=0x7f7dec071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 0x7f7dec071170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e04199700 tx=0x7f7df001a040 comp rx=0 tx=0).stop 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 msgr2=0x7f7e041980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d8d0 tx=0x7f7df800dc90 comp rx=0 tx=0).stop 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 shutdown_connections 2024-04-24T08:34:30.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7dec06ecc0 0x7f7dec071170 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e041004e0 0x7f7e041980e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 --2- 172.21.15.90:0/3793091271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e04100e80 0x7f7e04198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.545+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 >> 172.21.15.90:0/3793091271 conn(0x7f7e040fbda0 msgr2=0x7f7e040fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:30.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.546+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 shutdown_connections 2024-04-24T08:34:30.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.546+0000 7f7e0bc00700 1 -- 172.21.15.90:0/3793091271 wait complete. 2024-04-24T08:34:30.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.696+0000 7f83f6a00700 1 -- 172.21.15.90:0/733648378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0102460 msgr2=0x7f83f01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.696+0000 7f83f6a00700 1 --2- 172.21.15.90:0/733648378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0102460 0x7f83f01028d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009910 tx=0x7f83dc009c20 comp rx=0 tx=0).stop 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 -- 172.21.15.90:0/733648378 shutdown_connections 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 --2- 172.21.15.90:0/733648378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0102460 0x7f83f01028d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 --2- 172.21.15.90:0/733648378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0108460 0x7f83f0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 -- 172.21.15.90:0/733648378 >> 172.21.15.90:0/733648378 conn(0x7f83f00fdf40 msgr2=0x7f83f0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 -- 172.21.15.90:0/733648378 shutdown_connections 2024-04-24T08:34:30.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.697+0000 7f83f6a00700 1 -- 172.21.15.90:0/733648378 wait complete. 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 Processor -- start 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 -- start start 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 0x7f83f0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f0198db0 con 0x7f83f0108460 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.698+0000 7f83f6a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f019cb40 con 0x7f83f0102460 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56304/0 (socket says 172.21.15.90:56304) 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 -- 172.21.15.90:0/3227499553 learned_addr learned my addr 172.21.15.90:0/3227499553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83efe00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 0x7f83f0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 -- 172.21.15.90:0/3227499553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 msgr2=0x7f83f0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 0x7f83f0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 -- 172.21.15.90:0/3227499553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0095f0 con 0x7f83f0108460 2024-04-24T08:34:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83efe00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 0x7f83f0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:34:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.699+0000 7f83ef400700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00b5c0 tx=0x7f83dc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.700+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83dc01d070 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.700+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83dc00f460 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.700+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f019cdc0 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.700+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83dc017610 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.700+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f019d2b0 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.701+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f010a9f0 con 0x7f83f0108460 2024-04-24T08:34:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.702+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f83dc00f5d0 con 0x7f83f0108460 2024-04-24T08:34:30.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.703+0000 7f83ecc00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 0x7f83d4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:34:30.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.703+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f83dc090350 con 0x7f83f0108460 2024-04-24T08:34:30.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.703+0000 7f83efe00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 0x7f83d4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:34:30.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.704+0000 7f83efe00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 0x7f83d4071170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83e00099d0 tx=0x7f83e0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:34:30.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:30.708+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f83dc0591e0 con 0x7f83f0108460 2024-04-24T08:34:31.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.007+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f83f004f2a0 con 0x7f83f0108460 2024-04-24T08:34:31.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.008+0000 7f83ecc00700 1 -- 172.21.15.90:0/3227499553 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f83dc05c8a0 con 0x7f83f0108460 2024-04-24T08:34:31.009 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:34:31.009 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:34:31.009 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:34:31.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.010+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 msgr2=0x7f83d4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:31.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.010+0000 7f83f6a00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 0x7f83d4071170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83e00099d0 tx=0x7f83e0008040 comp rx=0 tx=0).stop 2024-04-24T08:34:31.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 msgr2=0x7f83f01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:34:31.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00b5c0 tx=0x7f83dc00be00 comp rx=0 tx=0).stop 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 shutdown_connections 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f83d406ecc0 0x7f83d4071170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83f0102460 0x7f83f0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 --2- 172.21.15.90:0/3227499553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83f0108460 0x7f83f01986d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 >> 172.21.15.90:0/3227499553 conn(0x7f83f00fdf40 msgr2=0x7f83f00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 shutdown_connections 2024-04-24T08:34:31.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:34:31.011+0000 7f83f6a00700 1 -- 172.21.15.90:0/3227499553 wait complete. 2024-04-24T08:34:31.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:31 smithi090 ceph-mon[85282]: Upgrade: Updating osd.5 2024-04-24T08:34:31.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:31 smithi090 ceph-mon[85282]: Deploying daemon osd.5 on smithi149 2024-04-24T08:34:31.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:31 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3227499553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:34:31.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:31 smithi149 ceph-mon[66490]: Upgrade: Updating osd.5 2024-04-24T08:34:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:31 smithi149 ceph-mon[66490]: Deploying daemon osd.5 on smithi149 2024-04-24T08:34:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:31 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3227499553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:34:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:32 smithi149 ceph-mon[66490]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:32.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:32 smithi149 ceph-mon[66490]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:34:32.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:32 smithi090 ceph-mon[85282]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:34:32.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:32 smithi090 ceph-mon[85282]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:34:33.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:33 smithi149 ceph-mon[66490]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:34:33.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:33 smithi090 ceph-mon[85282]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:34:35.027 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:34 smithi149 systemd[1]: Stopping Ceph osd.5 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:34:35.527 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[46832]: 2024-04-24T08:34:35.249+0000 7f5004200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:34:35.527 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[46832]: 2024-04-24T08:34:35.250+0000 7f5004200700 -1 osd.5 75 *** Got signal Terminated *** 2024-04-24T08:34:35.527 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[46832]: 2024-04-24T08:34:35.250+0000 7f5004200700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:34:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:36 smithi149 ceph-mon[66490]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:36.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:36 smithi149 ceph-mon[66490]: osd.5 marked itself down and dead 2024-04-24T08:34:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:36 smithi090 ceph-mon[85282]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:34:36.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:36 smithi090 ceph-mon[85282]: osd.5 marked itself down and dead 2024-04-24T08:34:37.155 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:36 smithi149 podman[85905]: 2024-04-24 08:34:36.804038452 +0000 UTC m=+1.757172664 container died 78ba7582037047ce547419f3b8759d12c42115fa1230919d91ae7fbffc331fc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, RELEASE=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-24T08:34:37.417 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:37 smithi149 podman[85905]: 2024-04-24 08:34:37.391633434 +0000 UTC m=+2.344767683 container cleanup 78ba7582037047ce547419f3b8759d12c42115fa1230919d91ae7fbffc331fc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2) 2024-04-24T08:34:37.418 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:37 smithi149 bash[85905]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5 2024-04-24T08:34:37.418 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:37 smithi149 ceph-mon[66490]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:34:37.418 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:37 smithi149 ceph-mon[66490]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T08:34:37.418 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:37 smithi149 ceph-mon[66490]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T08:34:37.418 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:37 smithi149 ceph-mon[66490]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T08:34:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:37 smithi090 ceph-mon[85282]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T08:34:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:37 smithi090 ceph-mon[85282]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T08:34:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:37 smithi090 ceph-mon[85282]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T08:34:37.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:37 smithi090 ceph-mon[85282]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T08:34:38.026 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:37 smithi149 podman[85924]: 2024-04-24 08:34:37.668229477 +0000 UTC m=+0.848997186 container remove 78ba7582037047ce547419f3b8759d12c42115fa1230919d91ae7fbffc331fc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, RELEASE=HEAD) 2024-04-24T08:34:38.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:38 smithi149 podman[86007]: 2024-04-24 08:34:38.385595819 +0000 UTC m=+0.033477824 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:34:38.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:38 smithi149 podman[86007]: 2024-04-24 08:34:38.502322543 +0000 UTC m=+0.150204596 container create b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:34:38.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:38 smithi149 ceph-mon[66490]: pgmap v190: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:38.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:38 smithi149 ceph-mon[66490]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T08:34:38.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:38 smithi090 ceph-mon[85282]: pgmap v190: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-24T08:34:38.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:38 smithi090 ceph-mon[85282]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T08:34:39.027 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:38 smithi149 podman[86007]: 2024-04-24 08:34:38.814150176 +0000 UTC m=+0.462032225 container init b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T08:34:39.027 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:38 smithi149 podman[86007]: 2024-04-24 08:34:38.825435088 +0000 UTC m=+0.473317138 container start b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T08:34:39.027 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:38 smithi149 podman[86007]: 2024-04-24 08:34:38.943596958 +0000 UTC m=+0.591479064 container attach b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T08:34:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:39 smithi149 ceph-mon[66490]: pgmap v192: 65 pgs: 6 active+undersized+degraded, 7 peering, 4 stale+active+clean, 7 active+undersized, 41 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 24/267 objects degraded (8.989%) 2024-04-24T08:34:39.527 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:39 smithi149 podman[86007]: 2024-04-24 08:34:39.281137534 +0000 UTC m=+0.929019548 container died b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:34:39.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:39 smithi090 ceph-mon[85282]: pgmap v192: 65 pgs: 6 active+undersized+degraded, 7 peering, 4 stale+active+clean, 7 active+undersized, 41 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 24/267 objects degraded (8.989%) 2024-04-24T08:34:40.499 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:40 smithi149 ceph-mon[66490]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:40.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 podman[86033]: 2024-04-24 08:34:40.088593011 +0000 UTC m=+0.790126712 container remove b58007b474b8bc84d6ad7f61031de83c469c9bd6c9ea13be6bf0d1a31d045af0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T08:34:40.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-24T08:34:40.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 systemd[1]: Stopped Ceph osd.5 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:34:40.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 systemd[1]: ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 39.856s CPU time. 2024-04-24T08:34:40.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 systemd[1]: Starting Ceph osd.5 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:34:40.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:40 smithi090 ceph-mon[85282]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:41.113 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 podman[86135]: 2024-04-24 08:34:40.761874041 +0000 UTC m=+0.033377596 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:34:41.113 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:40 smithi149 podman[86135]: 2024-04-24 08:34:40.938023005 +0000 UTC m=+0.209526594 container create 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T08:34:41.400 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:41 smithi149 ceph-mon[66490]: pgmap v193: 65 pgs: 10 active+undersized+degraded, 7 peering, 11 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/267 objects degraded (11.610%) 2024-04-24T08:34:41.400 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:41 smithi149 podman[86135]: 2024-04-24 08:34:41.277030292 +0000 UTC m=+0.548533964 container init 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T08:34:41.400 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:41 smithi149 podman[86135]: 2024-04-24 08:34:41.291271308 +0000 UTC m=+0.562774891 container start 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:34:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:41 smithi090 ceph-mon[85282]: pgmap v193: 65 pgs: 10 active+undersized+degraded, 7 peering, 11 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/267 objects degraded (11.610%) 2024-04-24T08:34:41.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:41 smithi149 podman[86135]: 2024-04-24 08:34:41.39937982 +0000 UTC m=+0.670883469 container attach 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T08:34:42.528 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 bash[86135]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate[86149]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T08:34:42.529 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:42 smithi149 podman[86135]: 2024-04-24 08:34:42.304555468 +0000 UTC m=+1.576059107 container died 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:34:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:43 smithi149 ceph-mon[66490]: pgmap v194: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:43 smithi149 ceph-mon[66490]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T08:34:43.527 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 podman[86290]: 2024-04-24 08:34:43.177938283 +0000 UTC m=+0.854666525 container remove 2973a28aea1cdc529d75c39b04a268a4257be3a78b390bc2cf552a8df03c5fb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:34:43.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:43 smithi090 ceph-mon[85282]: pgmap v194: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:43.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:43 smithi090 ceph-mon[85282]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T08:34:43.892 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 podman[86355]: 2024-04-24 08:34:43.460805587 +0000 UTC m=+0.033364755 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:34:43.892 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 podman[86355]: 2024-04-24 08:34:43.578975447 +0000 UTC m=+0.151534655 container create ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T08:34:43.892 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 podman[86355]: 2024-04-24 08:34:43.8905791 +0000 UTC m=+0.463138311 container init ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:34:44.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 podman[86355]: 2024-04-24 08:34:43.901150475 +0000 UTC m=+0.473709732 container start ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T08:34:44.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:43 smithi149 ceph-osd[86374]: -- 172.21.15.149:0/1869480690 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8ff93d680 con 0x55c8ff906c00 2024-04-24T08:34:44.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:44 smithi149 bash[86355]: ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d 2024-04-24T08:34:44.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:44 smithi149 systemd[1]: Started Ceph osd.5 for 10d437de-0212-11ef-bc93-c7b262605968. 2024-04-24T08:34:44.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:44 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:34:44.481+0000 7fc3fa9d47c0 -1 Falling back to public interface 2024-04-24T08:34:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:34:46.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:46 smithi149 ceph-mon[66490]: pgmap v195: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 687 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:46.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:46 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:46 smithi090 ceph-mon[85282]: pgmap v195: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 687 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:46.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:46 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:48 smithi149 ceph-mon[66490]: pgmap v196: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:48 smithi090 ceph-mon[85282]: pgmap v196: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:49.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:49 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:34:49.120+0000 7fc3fa9d47c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-24T08:34:49.500 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:49 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:34:49.498+0000 7fc3fa9d47c0 -1 osd.5 75 log_to_monitors true 2024-04-24T08:34:50.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:34:50.222+0000 7fc3e9e00700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:34:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-mon[66490]: pgmap v197: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 2 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-mon[66490]: from='osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:34:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-mon[66490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:34:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:50.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:50 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:50.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:50 smithi090 ceph-mon[85282]: pgmap v197: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 2 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:50.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:50 smithi090 ceph-mon[85282]: from='osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:34:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:50 smithi090 ceph-mon[85282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:34:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:50 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:50.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:50 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:51.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:51 smithi149 ceph-mon[66490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:34:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:51 smithi149 ceph-mon[66490]: osdmap e78: 6 total, 5 up, 6 in 2024-04-24T08:34:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:51 smithi149 ceph-mon[66490]: from='osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:51.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:51 smithi149 ceph-mon[66490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:51 smithi090 ceph-mon[85282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:34:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:51 smithi090 ceph-mon[85282]: osdmap e78: 6 total, 5 up, 6 in 2024-04-24T08:34:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:51 smithi090 ceph-mon[85282]: from='osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:51.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:51 smithi090 ceph-mon[85282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: pgmap v199: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429] boot 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:34:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:52.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:52 smithi149 ceph-mon[66490]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: pgmap v199: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 41/267 objects degraded (15.356%) 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: osd.5 [v2:172.21.15.149:6816/1799649429,v1:172.21.15.149:6817/1799649429] boot 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:52 smithi090 ceph-mon[85282]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T08:34:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:53 smithi149 ceph-mon[66490]: pgmap v202: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/267 objects degraded (13.483%) 2024-04-24T08:34:53.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:53 smithi149 ceph-mon[66490]: Health check update: Degraded data redundancy: 36/267 objects degraded (13.483%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:53.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:53 smithi090 ceph-mon[85282]: pgmap v202: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/267 objects degraded (13.483%) 2024-04-24T08:34:53.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:53 smithi090 ceph-mon[85282]: Health check update: Degraded data redundancy: 36/267 objects degraded (13.483%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T08:34:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:56 smithi149 ceph-mon[66490]: pgmap v203: 65 pgs: 3 peering, 5 active+undersized+degraded, 10 active+undersized, 47 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 15/267 objects degraded (5.618%) 2024-04-24T08:34:56.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:56 smithi090 ceph-mon[85282]: pgmap v203: 65 pgs: 3 peering, 5 active+undersized+degraded, 10 active+undersized, 47 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 15/267 objects degraded (5.618%) 2024-04-24T08:34:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:57 smithi149 ceph-mon[66490]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 440 B/s rd, 1 op/s 2024-04-24T08:34:57.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:57 smithi149 ceph-mon[66490]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-04-24T08:34:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:57 smithi090 ceph-mon[85282]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 440 B/s rd, 1 op/s 2024-04-24T08:34:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:57 smithi090 ceph-mon[85282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all osd 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T08:34:59.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:58 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T08:34:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:34:59.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all osd 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T08:34:59.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T08:34:59.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:58 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-24T08:35:00.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:34:59 smithi149 ceph-mon[66490]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-24T08:35:00.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:34:59 smithi090 ceph-mon[85282]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:01.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:01.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:01.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T08:35:01.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 -- 172.21.15.90:0/3931326242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 msgr2=0x7f46e8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 --2- 172.21.15.90:0/3931326242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8068b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009910 tx=0x7f46d8009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 -- 172.21.15.90:0/3931326242 shutdown_connections 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 --2- 172.21.15.90:0/3931326242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 --2- 172.21.15.90:0/3931326242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8068b70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 -- 172.21.15.90:0/3931326242 >> 172.21.15.90:0/3931326242 conn(0x7f46e8074fa0 msgr2=0x7f46e80753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 -- 172.21.15.90:0/3931326242 shutdown_connections 2024-04-24T08:35:01.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.178+0000 7f46eda00700 1 -- 172.21.15.90:0/3931326242 wait complete. 2024-04-24T08:35:01.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.179+0000 7f46eda00700 1 Processor -- start 2024-04-24T08:35:01.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46eda00700 1 -- start start 2024-04-24T08:35:01.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46eda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46eda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46eda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e81989e0 con 0x7f46e80687a0 2024-04-24T08:35:01.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46eda00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819c770 con 0x7f46e8069140 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33484/0 (socket says 172.21.15.90:33484) 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.180+0000 7f46e7e00700 1 -- 172.21.15.90:0/3592601207 learned_addr learned my addr 172.21.15.90:0/3592601207 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7400700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7e00700 1 -- 172.21.15.90:0/3592601207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 msgr2=0x7f46e8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7e00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7e00700 1 -- 172.21.15.90:0/3592601207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80095f0 con 0x7f46e80687a0 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7400700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.181+0000 7f46e7e00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46d80092e0 tx=0x7f46d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.182+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d801d070 con 0x7f46e80687a0 2024-04-24T08:35:01.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.182+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46d8005e00 con 0x7f46e80687a0 2024-04-24T08:35:01.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.182+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e819c9f0 con 0x7f46e80687a0 2024-04-24T08:35:01.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.182+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d8017400 con 0x7f46e80687a0 2024-04-24T08:35:01.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.182+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e819cf60 con 0x7f46e80687a0 2024-04-24T08:35:01.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.183+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e8062380 con 0x7f46e80687a0 2024-04-24T08:35:01.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.184+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f46d8003a60 con 0x7f46e80687a0 2024-04-24T08:35:01.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.185+0000 7f46e4c00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 0x7f46cc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.185+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f46d80902c0 con 0x7f46e80687a0 2024-04-24T08:35:01.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.185+0000 7f46e7400700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 0x7f46cc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.186+0000 7f46e7400700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 0x7f46cc071170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f46d4005fd0 tx=0x7f46d4005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.190+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f46d80590b0 con 0x7f46e80687a0 2024-04-24T08:35:01.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.477+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46e819d430 con 0x7f46cc06ecc0 2024-04-24T08:35:01.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.479+0000 7f46e4c00700 1 -- 172.21.15.90:0/3592601207 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f46e819d430 con 0x7f46cc06ecc0 2024-04-24T08:35:01.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 msgr2=0x7f46cc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 0x7f46cc071170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f46d4005fd0 tx=0x7f46d4005ee0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 msgr2=0x7f46e8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46d80092e0 tx=0x7f46d800be20 comp rx=0 tx=0).stop 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 shutdown_connections 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f46cc06ecc0 0x7f46cc071170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e80687a0 0x7f46e8197dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 --2- 172.21.15.90:0/3592601207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8069140 0x7f46e8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.482+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 >> 172.21.15.90:0/3592601207 conn(0x7f46e8074fa0 msgr2=0x7f46e80ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.483+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 shutdown_connections 2024-04-24T08:35:01.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.483+0000 7f46eda00700 1 -- 172.21.15.90:0/3592601207 wait complete. 2024-04-24T08:35:01.500 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:35:01.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 -- 172.21.15.90:0/1248476772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38068540 msgr2=0x7f0d380689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 --2- 172.21.15.90:0/1248476772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38068540 0x7f0d380689b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009910 tx=0x7f0d24009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:01.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 -- 172.21.15.90:0/1248476772 shutdown_connections 2024-04-24T08:35:01.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 --2- 172.21.15.90:0/1248476772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38068540 0x7f0d380689b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 --2- 172.21.15.90:0/1248476772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38101070 0x7f0d38101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.637+0000 7f0d3d000700 1 -- 172.21.15.90:0/1248476772 >> 172.21.15.90:0/1248476772 conn(0x7f0d38075130 msgr2=0x7f0d38075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:01.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.638+0000 7f0d3d000700 1 -- 172.21.15.90:0/1248476772 shutdown_connections 2024-04-24T08:35:01.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.638+0000 7f0d3d000700 1 -- 172.21.15.90:0/1248476772 wait complete. 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 Processor -- start 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 -- start start 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38068540 0x7f0d381982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38198ed0 con 0x7f0d38101070 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.639+0000 7f0d3d000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3819cc60 con 0x7f0d38068540 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33504/0 (socket says 172.21.15.90:33504) 2024-04-24T08:35:01.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 -- 172.21.15.90:0/2107971090 learned_addr learned my addr 172.21.15.90:0/2107971090 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:01.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35e00700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38068540 0x7f0d381982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 -- 172.21.15.90:0/2107971090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38068540 msgr2=0x7f0d381982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38068540 0x7f0d381982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 -- 172.21.15.90:0/2107971090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d240095f0 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.640+0000 7f0d35400700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d24006010 tx=0x7f0d2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.641+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d2401d070 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.641+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d24004fb0 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.641+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d24022ea0 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.641+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3819cee0 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.641+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3819d3d0 con 0x7f0d38101070 2024-04-24T08:35:01.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.642+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3810a9b0 con 0x7f0d38101070 2024-04-24T08:35:01.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.643+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0d24022470 con 0x7f0d38101070 2024-04-24T08:35:01.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.644+0000 7f0d2ea00700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 0x7f0d1c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:01.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.644+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0d24090360 con 0x7f0d38101070 2024-04-24T08:35:01.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.644+0000 7f0d35e00700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 0x7f0d1c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:01.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.645+0000 7f0d35e00700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 0x7f0d1c071170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006fd0 tx=0x7f0d28008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.651+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0d24059200 con 0x7f0d38101070 2024-04-24T08:35:01.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.901+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d3819d6a0 con 0x7f0d1c06ecc0 2024-04-24T08:35:01.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.903+0000 7f0d2ea00700 1 -- 172.21.15.90:0/2107971090 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0d3819d6a0 con 0x7f0d1c06ecc0 2024-04-24T08:35:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 msgr2=0x7f0d1c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 0x7f0d1c071170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006fd0 tx=0x7f0d28008040 comp rx=0 tx=0).stop 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 msgr2=0x7f0d381987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d24006010 tx=0x7f0d2400be20 comp rx=0 tx=0).stop 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 shutdown_connections 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0d1c06ecc0 0x7f0d1c071170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d38068540 0x7f0d381982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 --2- 172.21.15.90:0/2107971090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d38101070 0x7f0d381987f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 >> 172.21.15.90:0/2107971090 conn(0x7f0d38075130 msgr2=0x7f0d380fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 shutdown_connections 2024-04-24T08:35:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:01.906+0000 7f0d3d000700 1 -- 172.21.15.90:0/2107971090 wait complete. 2024-04-24T08:35:02.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 -- 172.21.15.90:0/1981588619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5390100e80 msgr2=0x7f5390104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 --2- 172.21.15.90:0/1981588619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5390100e80 0x7f5390104e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f537c009910 tx=0x7f537c009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 -- 172.21.15.90:0/1981588619 shutdown_connections 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 --2- 172.21.15.90:0/1981588619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5390100e80 0x7f5390104e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 --2- 172.21.15.90:0/1981588619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53901004e0 0x7f53901008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.060+0000 7f5397200700 1 -- 172.21.15.90:0/1981588619 >> 172.21.15.90:0/1981588619 conn(0x7f53900fbda0 msgr2=0x7f53900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.061+0000 7f5397200700 1 -- 172.21.15.90:0/1981588619 shutdown_connections 2024-04-24T08:35:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.061+0000 7f5397200700 1 -- 172.21.15.90:0/1981588619 wait complete. 2024-04-24T08:35:02.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 Processor -- start 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 -- start start 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5390100e80 0x7f53901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5390198ca0 con 0x7f53901004e0 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f5397200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539019ca30 con 0x7f5390100e80 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.062+0000 7f538fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33512/0 (socket says 172.21.15.90:33512) 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 -- 172.21.15.90:0/4089567288 learned_addr learned my addr 172.21.15.90:0/4089567288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538f400700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5390100e80 0x7f53901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 -- 172.21.15.90:0/4089567288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5390100e80 msgr2=0x7f53901985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5390100e80 0x7f53901985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 -- 172.21.15.90:0/4089567288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0095f0 con 0x7f53901004e0 2024-04-24T08:35:02.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.063+0000 7f538fe00700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f538000d8d0 tx=0x7f538000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.064+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5380009940 con 0x7f53901004e0 2024-04-24T08:35:02.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.064+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539019cd10 con 0x7f53901004e0 2024-04-24T08:35:02.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.064+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5380010460 con 0x7f53901004e0 2024-04-24T08:35:02.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.064+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f538000f5f0 con 0x7f53901004e0 2024-04-24T08:35:02.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.064+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539019d260 con 0x7f53901004e0 2024-04-24T08:35:02.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.065+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53901045d0 con 0x7f53901004e0 2024-04-24T08:35:02.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.066+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f53800105d0 con 0x7f53901004e0 2024-04-24T08:35:02.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.067+0000 7f538cc00700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 0x7f5374071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.067+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f538008edb0 con 0x7f53901004e0 2024-04-24T08:35:02.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.067+0000 7f538f400700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 0x7f5374071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.068+0000 7f538f400700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 0x7f5374071170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f537c006010 tx=0x7f537c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.072+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5380057ba0 con 0x7f53901004e0 2024-04-24T08:35:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.313+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f539019d540 con 0x7f537406ecc0 2024-04-24T08:35:02.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:02 smithi090 ceph-mon[85282]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-04-24T08:35:02.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:02 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6828/4061836021,v1:172.21.15.90:6829/4061836021] up:boot 2024-04-24T08:35:02.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:02 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:02.315 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:02 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.332+0000 7f538cc00700 1 -- 172.21.15.90:0/4089567288 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f539019d540 con 0x7f537406ecc0 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 2m ago 22m 13.8M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (22m) 2m ago 22m 8942k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (20m) 12s ago 20m 8988k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (4m) 2m ago 22m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (4m) 12s ago 20m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 2m ago 21m 74.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (14m) 2m ago 14m 86.9M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (14m) 2m ago 14m 18.0M - 18.2.0 dc2bc1663786 631c997943eb 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (14m) 12s ago 14m 151M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (14m) 12s ago 14m 18.6M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (10m) 2m ago 23m 507M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (9m) 12s ago 20m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (5m) 2m ago 24m 49.8M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (5m) 12s ago 20m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 2m ago 22m 7922k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (9m) 12s ago 20m 7730k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:35:02.335 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (3m) 2m ago 19m 88.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (2m) 2m ago 18m 83.9M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (2m) 2m ago 18m 11.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (106s) 12s ago 17m 89.2M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (63s) 12s ago 17m 89.9M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (18s) 12s ago 16m 13.3M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (8m) 2m ago 21m 69.4M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:35:02.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 msgr2=0x7f5374071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 0x7f5374071170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f537c006010 tx=0x7f537c00b540 comp rx=0 tx=0).stop 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 msgr2=0x7f5390198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f538000d8d0 tx=0x7f538000dc90 comp rx=0 tx=0).stop 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 shutdown_connections 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f537406ecc0 0x7f5374071170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53901004e0 0x7f5390198080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 --2- 172.21.15.90:0/4089567288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5390100e80 0x7f53901985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.335+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 >> 172.21.15.90:0/4089567288 conn(0x7f53900fbda0 msgr2=0x7f53900fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:02.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.336+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 shutdown_connections 2024-04-24T08:35:02.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.336+0000 7f5397200700 1 -- 172.21.15.90:0/4089567288 wait complete. 2024-04-24T08:35:02.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 -- 172.21.15.90:0/317357497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 msgr2=0x7fce1010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 --2- 172.21.15.90:0/317357497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1010ccc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce00009910 tx=0x7fce00009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:02.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 -- 172.21.15.90:0/317357497 shutdown_connections 2024-04-24T08:35:02.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 --2- 172.21.15.90:0/317357497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10106b80 0x7fce10106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 --2- 172.21.15.90:0/317357497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1010ccc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.482+0000 7fce17000700 1 -- 172.21.15.90:0/317357497 >> 172.21.15.90:0/317357497 conn(0x7fce10076060 msgr2=0x7fce10078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:02.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.483+0000 7fce17000700 1 -- 172.21.15.90:0/317357497 shutdown_connections 2024-04-24T08:35:02.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.483+0000 7fce17000700 1 -- 172.21.15.90:0/317357497 wait complete. 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 Processor -- start 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 -- start start 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10106b80 0x7fce1019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019d100 con 0x7fce1010c8f0 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce17000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce101a0e90 con 0x7fce10106b80 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce07e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce07e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33528/0 (socket says 172.21.15.90:33528) 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.484+0000 7fce07e00700 1 -- 172.21.15.90:0/1052244357 learned_addr learned my addr 172.21.15.90:0/1052244357 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce07e00700 1 -- 172.21.15.90:0/1052244357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10106b80 msgr2=0x7fce1019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce07e00700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10106b80 0x7fce1019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce07e00700 1 -- 172.21.15.90:0/1052244357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce000095f0 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce07e00700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b4b0 tx=0x7fcdf800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf8011800 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf8011e20 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf800f2e0 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce101a1170 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.485+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce101a16c0 con 0x7fce1010c8f0 2024-04-24T08:35:02.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.486+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce10109b20 con 0x7fce1010c8f0 2024-04-24T08:35:02.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.488+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcdf8010380 con 0x7fce1010c8f0 2024-04-24T08:35:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.489+0000 7fce0d600700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 0x7fcdf4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.489+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcdf808e9a0 con 0x7fce1010c8f0 2024-04-24T08:35:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.489+0000 7fce0fe00700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 0x7fcdf4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.490+0000 7fce0fe00700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 0x7fcdf4071170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b5c0 tx=0x7fce00005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.493+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcdf80577b0 con 0x7fce1010c8f0 2024-04-24T08:35:02.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:02 smithi149 ceph-mon[66490]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-04-24T08:35:02.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:02 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6828/4061836021,v1:172.21.15.90:6829/4061836021] up:boot 2024-04-24T08:35:02.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:02 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:02.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:02 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:35:02.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.819+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce1004f2a0 con 0x7fce1010c8f0 2024-04-24T08:35:02.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.820+0000 7fce0d600700 1 -- 172.21.15.90:0/1052244357 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7fcdf805ae70 con 0x7fce1010c8f0 2024-04-24T08:35:02.821 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:35:02.821 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-04-24T08:35:02.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 10 2024-04-24T08:35:02.823 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:35:02.823 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:35:02.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 msgr2=0x7fcdf4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 0x7fcdf4071170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b5c0 tx=0x7fce00005fd0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 msgr2=0x7fce1019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b4b0 tx=0x7fcdf800b870 comp rx=0 tx=0).stop 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 shutdown_connections 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcdf406ecc0 0x7fcdf4071170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10106b80 0x7fce1019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 --2- 172.21.15.90:0/1052244357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce1010c8f0 0x7fce1019ca20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 >> 172.21.15.90:0/1052244357 conn(0x7fce10076060 msgr2=0x7fce10077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 shutdown_connections 2024-04-24T08:35:02.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.823+0000 7fce17000700 1 -- 172.21.15.90:0/1052244357 wait complete. 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 -- 172.21.15.90:0/32831315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d4100e80 msgr2=0x7f06d4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 --2- 172.21.15.90:0/32831315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d4100e80 0x7f06d4104e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009910 tx=0x7f06c0009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 -- 172.21.15.90:0/32831315 shutdown_connections 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 --2- 172.21.15.90:0/32831315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d4100e80 0x7f06d4104e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 --2- 172.21.15.90:0/32831315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d41004e0 0x7f06d41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 -- 172.21.15.90:0/32831315 >> 172.21.15.90:0/32831315 conn(0x7f06d40fbda0 msgr2=0x7f06d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 -- 172.21.15.90:0/32831315 shutdown_connections 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.980+0000 7f06dbe00700 1 -- 172.21.15.90:0/32831315 wait complete. 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.981+0000 7f06dbe00700 1 Processor -- start 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.981+0000 7f06dbe00700 1 -- start start 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06dbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 0x7f06d4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06dbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4198cc0 con 0x7f06d41004e0 2024-04-24T08:35:02.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06dbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4198e00 con 0x7f06d4100e80 2024-04-24T08:35:02.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06d9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06d9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33554/0 (socket says 172.21.15.90:33554) 2024-04-24T08:35:02.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.982+0000 7f06d9400700 1 -- 172.21.15.90:0/1273578732 learned_addr learned my addr 172.21.15.90:0/1273578732 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:02.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d8a00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 0x7f06d4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d9400700 1 -- 172.21.15.90:0/1273578732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 msgr2=0x7f06d4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d9400700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 0x7f06d4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d9400700 1 -- 172.21.15.90:0/1273578732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00095f0 con 0x7f06d41004e0 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d8a00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 0x7f06d4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d9400700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c930 tx=0x7f06c800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c8007ad0 con 0x7f06d41004e0 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d419cc50 con 0x7f06d41004e0 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.983+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06c8007c30 con 0x7f06d41004e0 2024-04-24T08:35:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.984+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c8018610 con 0x7f06d41004e0 2024-04-24T08:35:02.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.984+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d419d1a0 con 0x7f06d41004e0 2024-04-24T08:35:02.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.985+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d41045d0 con 0x7f06d41004e0 2024-04-24T08:35:02.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.986+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f06c800f450 con 0x7f06d41004e0 2024-04-24T08:35:02.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.987+0000 7f06d2000700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 0x7f06bc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:02.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.987+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f06c808eff0 con 0x7f06d41004e0 2024-04-24T08:35:02.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.988+0000 7f06d8a00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 0x7f06bc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:02.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.988+0000 7f06d8a00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 0x7f06bc071170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006010 tx=0x7f06c000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:02.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:02.992+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f06c80c3ff0 con 0x7f06d41004e0 2024-04-24T08:35:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.268+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f06d4103fb0 con 0x7f06d41004e0 2024-04-24T08:35:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.269+0000 7f06d2000700 1 -- 172.21.15.90:0/1273578732 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f06c8057de0 con 0x7f06d41004e0 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:e19 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 18 2024-04-24T08:35:03.270 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:35:00.123022+0000 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:35:03.271 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 24223 members: 24223 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:03.272 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/13627278,v1:172.21.15.90:6827/13627278] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/4061836021,v1:172.21.15.90:6829/4061836021] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 msgr2=0x7f06bc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 0x7f06bc071170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006010 tx=0x7f06c000b540 comp rx=0 tx=0).stop 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 msgr2=0x7f06d4198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c930 tx=0x7f06c800ccf0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 shutdown_connections 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f06bc06ecc0 0x7f06bc071170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d41004e0 0x7f06d4198130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 --2- 172.21.15.90:0/1273578732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06d4100e80 0x7f06d4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 >> 172.21.15.90:0/1273578732 conn(0x7f06d40fbda0 msgr2=0x7f06d40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.272+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 shutdown_connections 2024-04-24T08:35:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.273+0000 7f06dbe00700 1 -- 172.21.15.90:0/1273578732 wait complete. 2024-04-24T08:35:03.275 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1052244357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:03.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:03.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 -- 172.21.15.90:0/1286284045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 msgr2=0x7f54e8110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 --2- 172.21.15.90:0/1286284045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e8110a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009940 tx=0x7f54d4009c50 comp rx=0 tx=0).stop 2024-04-24T08:35:03.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 -- 172.21.15.90:0/1286284045 shutdown_connections 2024-04-24T08:35:03.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 --2- 172.21.15.90:0/1286284045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e8110a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 --2- 172.21.15.90:0/1286284045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e8068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 -- 172.21.15.90:0/1286284045 >> 172.21.15.90:0/1286284045 conn(0x7f54e8075130 msgr2=0x7f54e8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:03.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 -- 172.21.15.90:0/1286284045 shutdown_connections 2024-04-24T08:35:03.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.436+0000 7f54ee200700 1 -- 172.21.15.90:0/1286284045 wait complete. 2024-04-24T08:35:03.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.437+0000 7f54ee200700 1 Processor -- start 2024-04-24T08:35:03.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.437+0000 7f54ee200700 1 -- start start 2024-04-24T08:35:03.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54ee200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54ee200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54ee200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e819d170 con 0x7f54e8068ee0 2024-04-24T08:35:03.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54ee200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81a0f00 con 0x7f54e8068540 2024-04-24T08:35:03.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33574/0 (socket says 172.21.15.90:33574) 2024-04-24T08:35:03.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54e6a00700 1 -- 172.21.15.90:0/1600616153 learned_addr learned my addr 172.21.15.90:0/1600616153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.438+0000 7f54e7400700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54e6a00700 1 -- 172.21.15.90:0/1600616153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 msgr2=0x7f54e819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54e6a00700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54e6a00700 1 -- 172.21.15.90:0/1600616153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d40095f0 con 0x7f54e8068ee0 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54e7400700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54e6a00700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b810 tx=0x7f54d40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d401d070 con 0x7f54e8068ee0 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54d4005e00 con 0x7f54e8068ee0 2024-04-24T08:35:03.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e81a1180 con 0x7f54e8068ee0 2024-04-24T08:35:03.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.439+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d4021410 con 0x7f54e8068ee0 2024-04-24T08:35:03.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.440+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e81a16d0 con 0x7f54e8068ee0 2024-04-24T08:35:03.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.441+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e8101410 con 0x7f54e8068ee0 2024-04-24T08:35:03.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.442+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f54d400f7f0 con 0x7f54e8068ee0 2024-04-24T08:35:03.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.443+0000 7f54dfe00700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 0x7f54cc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.443+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f54d40901a0 con 0x7f54e8068ee0 2024-04-24T08:35:03.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.444+0000 7f54e7400700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 0x7f54cc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.444+0000 7f54e7400700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 0x7f54cc071170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54d8007900 tx=0x7f54d8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.447+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f54d4058fb0 con 0x7f54e8068ee0 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1052244357' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.qbyvpv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:03.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.721+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e81a18f0 con 0x7f54cc06ecc0 2024-04-24T08:35:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.722+0000 7f54dfe00700 1 -- 172.21.15.90:0/1600616153 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f54e81a18f0 con 0x7f54cc06ecc0 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "12/23 daemons upgraded", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:35:03.723 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:35:03.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 msgr2=0x7f54cc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 0x7f54cc071170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54d8007900 tx=0x7f54d8008040 comp rx=0 tx=0).stop 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 msgr2=0x7f54e819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b810 tx=0x7f54d40056a0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 shutdown_connections 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54cc06ecc0 0x7f54cc071170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54e8068540 0x7f54e819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 --2- 172.21.15.90:0/1600616153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54e8068ee0 0x7f54e819ca90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 >> 172.21.15.90:0/1600616153 conn(0x7f54e8075130 msgr2=0x7f54e810f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 shutdown_connections 2024-04-24T08:35:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.725+0000 7f54ee200700 1 -- 172.21.15.90:0/1600616153 wait complete. 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 -- 172.21.15.90:0/350104254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf8100e80 msgr2=0x7feaf8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 --2- 172.21.15.90:0/350104254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf8100e80 0x7feaf8104e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feae8009910 tx=0x7feae8009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 -- 172.21.15.90:0/350104254 shutdown_connections 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 --2- 172.21.15.90:0/350104254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf8100e80 0x7feaf8104e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 --2- 172.21.15.90:0/350104254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf81004e0 0x7feaf81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 -- 172.21.15.90:0/350104254 >> 172.21.15.90:0/350104254 conn(0x7feaf80fbda0 msgr2=0x7feaf80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:03.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 -- 172.21.15.90:0/350104254 shutdown_connections 2024-04-24T08:35:03.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.875+0000 7feb00800700 1 -- 172.21.15.90:0/350104254 wait complete. 2024-04-24T08:35:03.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.877+0000 7feb00800700 1 Processor -- start 2024-04-24T08:35:03.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.877+0000 7feb00800700 1 -- start start 2024-04-24T08:35:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feb00800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feb00800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf8100e80 0x7feaf8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feb00800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8073ad0 con 0x7feaf81004e0 2024-04-24T08:35:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feb00800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8073c10 con 0x7feaf8100e80 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33590/0 (socket says 172.21.15.90:33590) 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 -- 172.21.15.90:0/2291466505 learned_addr learned my addr 172.21.15.90:0/2291466505 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafd400700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf8100e80 0x7feaf8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 -- 172.21.15.90:0/2291466505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf8100e80 msgr2=0x7feaf8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf8100e80 0x7feaf8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.878+0000 7feafde00700 1 -- 172.21.15.90:0/2291466505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80095f0 con 0x7feaf81004e0 2024-04-24T08:35:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feafde00700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c8a0 tx=0x7feaec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaec00cea0 con 0x7feaf81004e0 2024-04-24T08:35:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf81a2100 con 0x7feaf81004e0 2024-04-24T08:35:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaec004830 con 0x7feaf81004e0 2024-04-24T08:35:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaec0055b0 con 0x7feaf81004e0 2024-04-24T08:35:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.879+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf81a2510 con 0x7feaf81004e0 2024-04-24T08:35:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.880+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8108850 con 0x7feaf81004e0 2024-04-24T08:35:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.881+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feaec007ad0 con 0x7feaf81004e0 2024-04-24T08:35:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.882+0000 7feaf2a00700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 0x7feae0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.882+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feaec08ead0 con 0x7feaf81004e0 2024-04-24T08:35:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.883+0000 7feafd400700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 0x7feae0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.884+0000 7feafd400700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 0x7feae0071170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae8000c00 tx=0x7feae8005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:03.887+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feaec0578e0 con 0x7feaf81004e0 2024-04-24T08:35:04.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.201+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feaf8108230 con 0x7feaf81004e0 2024-04-24T08:35:04.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.202+0000 7feaf2a00700 1 -- 172.21.15.90:0/2291466505 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7feaec01d0e0 con 0x7feaf81004e0 2024-04-24T08:35:04.203 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:35:04.203 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:35:04.203 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 msgr2=0x7feae0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 0x7feae0071170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae8000c00 tx=0x7feae8005fd0 comp rx=0 tx=0).stop 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 msgr2=0x7feaf8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c8a0 tx=0x7feaec00cc60 comp rx=0 tx=0).stop 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 shutdown_connections 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7feae006ecc0 0x7feae0071170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.204+0000 7feb00800700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feaf81004e0 0x7feaf8073050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.205+0000 7feb00800700 1 --2- 172.21.15.90:0/2291466505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf8100e80 0x7feaf8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:04.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.205+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 >> 172.21.15.90:0/2291466505 conn(0x7feaf80fbda0 msgr2=0x7feaf80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.205+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 shutdown_connections 2024-04-24T08:35:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:04.205+0000 7feb00800700 1 -- 172.21.15.90:0/2291466505 wait complete. 2024-04-24T08:35:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:04 smithi090 ceph-mon[85282]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:04 smithi090 ceph-mon[85282]: Upgrade: Updating mds.cephfs.smithi090.qbyvpv 2024-04-24T08:35:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:04 smithi090 ceph-mon[85282]: Deploying daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:35:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:04 smithi090 ceph-mon[85282]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-24T08:35:04.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:04 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1273578732' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:35:04.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:04 smithi149 ceph-mon[66490]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:04.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:04 smithi149 ceph-mon[66490]: Upgrade: Updating mds.cephfs.smithi090.qbyvpv 2024-04-24T08:35:04.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:04 smithi149 ceph-mon[66490]: Deploying daemon mds.cephfs.smithi090.qbyvpv on smithi090 2024-04-24T08:35:04.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:04 smithi149 ceph-mon[66490]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-24T08:35:04.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:04 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1273578732' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:35:05.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:05 smithi149 ceph-mon[66490]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:05.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:05 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2291466505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:35:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:05 smithi090 ceph-mon[85282]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:05.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:05 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2291466505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:35:06.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:06 smithi149 ceph-mon[66490]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:35:06.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:06.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:06 smithi149 ceph-mon[66490]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T08:35:06.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:06 smithi090 ceph-mon[85282]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:35:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:06 smithi090 ceph-mon[85282]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T08:35:07.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:07 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:07.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:07 smithi149 ceph-mon[66490]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-04-24T08:35:07.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:07 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:07.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:07 smithi090 ceph-mon[85282]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-04-24T08:35:10.456 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:10 smithi090 ceph-mon[85282]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 107 B/s rd, 0 op/s 2024-04-24T08:35:10.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:10 smithi149 ceph-mon[66490]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 107 B/s rd, 0 op/s 2024-04-24T08:35:12.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:boot 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:12.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:12 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:boot 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:12 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:14.411 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:14 smithi090 ceph-mon[85282]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-24T08:35:14.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:14 smithi149 ceph-mon[66490]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-24T08:35:15.524 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:16 smithi149 ceph-mon[66490]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:16 smithi090 ceph-mon[85282]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:17.562 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:17 smithi090 ceph-mon[85282]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:17 smithi149 ceph-mon[66490]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:19 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:19.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:19 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:19.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:19 smithi149 ceph-mon[66490]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:19 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:19 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:19 smithi090 ceph-mon[85282]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:21 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:21 smithi149 ceph-mon[66490]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:21 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:21 smithi090 ceph-mon[85282]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:24.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:24 smithi149 ceph-mon[66490]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:24.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:24 smithi090 ceph-mon[85282]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:26.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:26.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:26.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:26.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:26.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:26.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:26.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:27.498 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:27.499 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.499 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.499 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.499 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:27 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:27.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:27.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:27 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:28.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:28 smithi090 ceph-mon[85282]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:28.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:28 smithi149 ceph-mon[66490]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: Upgrade: Updating mds.cephfs.smithi090.lumztb 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: Deploying daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:35:29.759 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:29 smithi090 ceph-mon[85282]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:30.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: Upgrade: Updating mds.cephfs.smithi090.lumztb 2024-04-24T08:35:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.lumztb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: Deploying daemon mds.cephfs.smithi090.lumztb on smithi090 2024-04-24T08:35:30.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:29 smithi149 ceph-mon[66490]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:31.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:32.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:31 smithi149 ceph-mon[66490]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:32.035 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:31 smithi090 ceph-mon[85282]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:34.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:33 smithi149 ceph-mon[66490]: osdmap e84: 6 total, 6 up, 6 in 2024-04-24T08:35:34.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:33 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:34.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:33 smithi149 ceph-mon[66490]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:34.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:33 smithi090 ceph-mon[85282]: osdmap e84: 6 total, 6 up, 6 in 2024-04-24T08:35:34.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:33 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 2 up:standby 2024-04-24T08:35:34.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:33 smithi090 ceph-mon[85282]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.379+0000 7f7e7ea00700 1 -- 172.21.15.90:0/464743921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 msgr2=0x7f7e7810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.379+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/464743921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e7810c5e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009940 tx=0x7f7e60009c50 comp rx=0 tx=0).stop 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 -- 172.21.15.90:0/464743921 shutdown_connections 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/464743921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e7810c5e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/464743921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 0x7f7e78073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 -- 172.21.15.90:0/464743921 >> 172.21.15.90:0/464743921 conn(0x7f7e780fbda0 msgr2=0x7f7e780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:34.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 -- 172.21.15.90:0/464743921 shutdown_connections 2024-04-24T08:35:34.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.380+0000 7f7e7ea00700 1 -- 172.21.15.90:0/464743921 wait complete. 2024-04-24T08:35:34.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 Processor -- start 2024-04-24T08:35:34.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 -- start start 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 0x7f7e78198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78198e40 con 0x7f7e780739a0 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.381+0000 7f7e7ea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7819cbd0 con 0x7f7e78073090 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51072/0 (socket says 172.21.15.90:51072) 2024-04-24T08:35:34.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 -- 172.21.15.90:0/1308536633 learned_addr learned my addr 172.21.15.90:0/1308536633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e77e00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 0x7f7e78198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 -- 172.21.15.90:0/1308536633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 msgr2=0x7f7e78198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 0x7f7e78198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.382+0000 7f7e6fe00700 1 -- 172.21.15.90:0/1308536633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600095f0 con 0x7f7e780739a0 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e6fe00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e600055f0 tx=0x7f7e600056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e6001d070 con 0x7f7e780739a0 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7819ce50 con 0x7f7e780739a0 2024-04-24T08:35:34.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e6000f460 con 0x7f7e780739a0 2024-04-24T08:35:34.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7819d340 con 0x7f7e780739a0 2024-04-24T08:35:34.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.383+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e6002b610 con 0x7f7e780739a0 2024-04-24T08:35:34.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.384+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78109ce0 con 0x7f7e780739a0 2024-04-24T08:35:34.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.385+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7e6000f5d0 con 0x7f7e780739a0 2024-04-24T08:35:34.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.386+0000 7f7e75600700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 0x7f7e5c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.386+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7e60090030 con 0x7f7e780739a0 2024-04-24T08:35:34.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.387+0000 7f7e77e00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 0x7f7e5c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.387+0000 7f7e77e00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 0x7f7e5c071170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005fd0 tx=0x7f7e6800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:34.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.392+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7e60058ef0 con 0x7f7e780739a0 2024-04-24T08:35:34.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.645+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e7819d670 con 0x7f7e5c06ecc0 2024-04-24T08:35:34.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.647+0000 7f7e75600700 1 -- 172.21.15.90:0/1308536633 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7e7819d670 con 0x7f7e5c06ecc0 2024-04-24T08:35:34.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 msgr2=0x7f7e5c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 0x7f7e5c071170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005fd0 tx=0x7f7e6800a560 comp rx=0 tx=0).stop 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 msgr2=0x7f7e78198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e600055f0 tx=0x7f7e600056d0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 shutdown_connections 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f7e5c06ecc0 0x7f7e5c071170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78073090 0x7f7e78198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 --2- 172.21.15.90:0/1308536633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e780739a0 0x7f7e78198760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 >> 172.21.15.90:0/1308536633 conn(0x7f7e780fbda0 msgr2=0x7f7e78106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 shutdown_connections 2024-04-24T08:35:34.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.650+0000 7f7e7ea00700 1 -- 172.21.15.90:0/1308536633 wait complete. 2024-04-24T08:35:34.667 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:35:34.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.810+0000 7fbb9a600700 1 -- 172.21.15.90:0/1182253994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101420 msgr2=0x7fbb941017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.810+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1182253994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101420 0x7fbb941017f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009910 tx=0x7fbb84009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 -- 172.21.15.90:0/1182253994 shutdown_connections 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1182253994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101d30 0x7fbb9410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1182253994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101420 0x7fbb941017f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 -- 172.21.15.90:0/1182253994 >> 172.21.15.90:0/1182253994 conn(0x7fbb940fac80 msgr2=0x7fbb940fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 -- 172.21.15.90:0/1182253994 shutdown_connections 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.811+0000 7fbb9a600700 1 -- 172.21.15.90:0/1182253994 wait complete. 2024-04-24T08:35:34.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 Processor -- start 2024-04-24T08:35:34.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 -- start start 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101420 0x7fbb94198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94198da0 con 0x7fbb94101d30 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.812+0000 7fbb9a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9419cb30 con 0x7fbb94101420 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51090/0 (socket says 172.21.15.90:51090) 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 -- 172.21.15.90:0/1591449323 learned_addr learned my addr 172.21.15.90:0/1591449323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb93400700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101420 0x7fbb94198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 -- 172.21.15.90:0/1591449323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101420 msgr2=0x7fbb94198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101420 0x7fbb94198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.813+0000 7fbb92a00700 1 -- 172.21.15.90:0/1591449323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840095f0 con 0x7fbb94101d30 2024-04-24T08:35:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb92a00700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b4b0 tx=0x7fbb8000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb80011800 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb80011e20 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8000f2e0 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9419ce10 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.814+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9419d360 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.815+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94109d40 con 0x7fbb94101d30 2024-04-24T08:35:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.816+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbb80010380 con 0x7fbb94101d30 2024-04-24T08:35:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.817+0000 7fbb8be00700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 0x7fbb78071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:34.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.817+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb8008e9d0 con 0x7fbb94101d30 2024-04-24T08:35:34.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.817+0000 7fbb93400700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 0x7fbb78071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:34.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.818+0000 7fbb93400700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 0x7fbb78071170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:34.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:34.822+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbb80057870 con 0x7fbb94101d30 2024-04-24T08:35:35.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.078+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb9419d630 con 0x7fbb7806ecc0 2024-04-24T08:35:35.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.079+0000 7fbb8be00700 1 -- 172.21.15.90:0/1591449323 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbb9419d630 con 0x7fbb7806ecc0 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.082+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 msgr2=0x7fbb78071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.082+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 0x7fbb78071170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8401a040 comp rx=0 tx=0).stop 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.082+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 msgr2=0x7fbb941986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.082+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b4b0 tx=0x7fbb8000b870 comp rx=0 tx=0).stop 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 shutdown_connections 2024-04-24T08:35:35.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb7806ecc0 0x7fbb78071170 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb94101420 0x7fbb94198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 --2- 172.21.15.90:0/1591449323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb94101d30 0x7fbb941986c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 >> 172.21.15.90:0/1591449323 conn(0x7fbb940fac80 msgr2=0x7fbb940ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:35.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 shutdown_connections 2024-04-24T08:35:35.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.083+0000 7fbb9a600700 1 -- 172.21.15.90:0/1591449323 wait complete. 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 -- 172.21.15.90:0/4160536181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 msgr2=0x7f32d410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 --2- 172.21.15.90:0/4160536181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d410ccc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009910 tx=0x7f32c4009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 -- 172.21.15.90:0/4160536181 shutdown_connections 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 --2- 172.21.15.90:0/4160536181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 0x7f32d4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 --2- 172.21.15.90:0/4160536181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d410ccc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 -- 172.21.15.90:0/4160536181 >> 172.21.15.90:0/4160536181 conn(0x7f32d4076060 msgr2=0x7f32d4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 -- 172.21.15.90:0/4160536181 shutdown_connections 2024-04-24T08:35:35.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.239+0000 7f32da200700 1 -- 172.21.15.90:0/4160536181 wait complete. 2024-04-24T08:35:35.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.240+0000 7f32da200700 1 Processor -- start 2024-04-24T08:35:35.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32da200700 1 -- start start 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32da200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 0x7f32d419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32da200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32da200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d419d090 con 0x7f32d410c8f0 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32da200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d419d1d0 con 0x7f32d4106b80 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51120/0 (socket says 172.21.15.90:51120) 2024-04-24T08:35:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.241+0000 7f32d2a00700 1 -- 172.21.15.90:0/1957517264 learned_addr learned my addr 172.21.15.90:0/1957517264 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32d2a00700 1 -- 172.21.15.90:0/1957517264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 msgr2=0x7f32d419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32d3400700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 0x7f32d419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32d2a00700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 0x7f32d419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32d2a00700 1 -- 172.21.15.90:0/1957517264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40095f0 con 0x7f32d410c8f0 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32d2a00700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c930 tx=0x7f32c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32c0007ad0 con 0x7f32d410c8f0 2024-04-24T08:35:35.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32c0007c30 con 0x7f32d410c8f0 2024-04-24T08:35:35.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d41a1020 con 0x7f32d410c8f0 2024-04-24T08:35:35.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32c0018610 con 0x7f32d410c8f0 2024-04-24T08:35:35.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.242+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d41a1570 con 0x7f32d410c8f0 2024-04-24T08:35:35.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.243+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4109b20 con 0x7f32d410c8f0 2024-04-24T08:35:35.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.244+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f32c000f450 con 0x7f32d410c8f0 2024-04-24T08:35:35.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.245+0000 7f32cbe00700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 0x7f32b8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.245+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f32c008f0c0 con 0x7f32d410c8f0 2024-04-24T08:35:35.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.246+0000 7f32d3400700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 0x7f32b8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.246+0000 7f32d3400700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 0x7f32b8071170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32c400b5c0 tx=0x7f32c4005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:35.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.250+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f32c0057e30 con 0x7f32d410c8f0 2024-04-24T08:35:35.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.484+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f32d41a17f0 con 0x7f32b806ecc0 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.499+0000 7f32cbe00700 1 -- 172.21.15.90:0/1957517264 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f32d41a17f0 con 0x7f32b806ecc0 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 17s ago 22m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (22m) 17s ago 22m 9433k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (21m) 45s ago 21m 8988k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:35:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (4m) 17s ago 22m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (4m) 45s ago 21m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 17s ago 22m 72.1M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (15m) 17s ago 15m 11.5M - 18.2.0 dc2bc1663786 9b3a74fe34e6 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (24s) 17s ago 15m 11.0M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (15m) 45s ago 15m 151M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (15m) 45s ago 15m 18.6M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (11m) 17s ago 24m 510M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (10m) 45s ago 21m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (6m) 17s ago 24m 55.2M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (5m) 45s ago 21m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 17s ago 22m 7972k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (9m) 45s ago 21m 7730k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:35:35.501 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (4m) 17s ago 20m 93.3M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (3m) 17s ago 19m 87.3M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (2m) 17s ago 18m 76.1M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (2m) 45s ago 18m 89.2M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (96s) 45s ago 17m 89.9M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (52s) 45s ago 17m 13.3M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:35:35.502 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (9m) 17s ago 21m 75.6M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.502+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 msgr2=0x7f32b8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.502+0000 7f32da200700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 0x7f32b8071170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32c400b5c0 tx=0x7f32c4005960 comp rx=0 tx=0).stop 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.502+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 msgr2=0x7f32d419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.502+0000 7f32da200700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c930 tx=0x7f32c000ccf0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 shutdown_connections 2024-04-24T08:35:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f32b806ecc0 0x7f32b8071170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32d4106b80 0x7f32d419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 --2- 172.21.15.90:0/1957517264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32d410c8f0 0x7f32d419ca40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 >> 172.21.15.90:0/1957517264 conn(0x7f32d4076060 msgr2=0x7f32d4077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 shutdown_connections 2024-04-24T08:35:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.503+0000 7f32da200700 1 -- 172.21.15.90:0/1957517264 wait complete. 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 -- 172.21.15.90:0/140582780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27900687a0 msgr2=0x7f2790068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 --2- 172.21.15.90:0/140582780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27900687a0 0x7f2790068b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2784009910 tx=0x7f2784009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 -- 172.21.15.90:0/140582780 shutdown_connections 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 --2- 172.21.15.90:0/140582780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2790069140 0x7f2790104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 --2- 172.21.15.90:0/140582780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27900687a0 0x7f2790068b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.665+0000 7f2797600700 1 -- 172.21.15.90:0/140582780 >> 172.21.15.90:0/140582780 conn(0x7f2790075130 msgr2=0x7f2790075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:35.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.666+0000 7f2797600700 1 -- 172.21.15.90:0/140582780 shutdown_connections 2024-04-24T08:35:35.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.666+0000 7f2797600700 1 -- 172.21.15.90:0/140582780 wait complete. 2024-04-24T08:35:35.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 Processor -- start 2024-04-24T08:35:35.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 -- start start 2024-04-24T08:35:35.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27900687a0 0x7f27900ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2790100720 con 0x7f2790069140 2024-04-24T08:35:35.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.667+0000 7f2797600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2790100860 con 0x7f27900687a0 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51140/0 (socket says 172.21.15.90:51140) 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 -- 172.21.15.90:0/474422642 learned_addr learned my addr 172.21.15.90:0/474422642 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f2794c00700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27900687a0 0x7f27900ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 -- 172.21.15.90:0/474422642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27900687a0 msgr2=0x7f27900ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27900687a0 0x7f27900ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 -- 172.21.15.90:0/474422642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840095f0 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.668+0000 7f278fe00700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d8d0 tx=0x7f277c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.669+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277c0041f0 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.669+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277c007620 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.669+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277c003c50 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.669+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27901a20b0 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.669+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27901a24c0 con 0x7f2790069140 2024-04-24T08:35:35.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.670+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2790108770 con 0x7f2790069140 2024-04-24T08:35:35.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.671+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f277c004350 con 0x7f2790069140 2024-04-24T08:35:35.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.672+0000 7f278d600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 0x7f2778071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:35.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.672+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f277c08e980 con 0x7f2790069140 2024-04-24T08:35:35.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.673+0000 7f2794c00700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 0x7f2778071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:35.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.673+0000 7f2794c00700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 0x7f2778071170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2784006010 tx=0x7f278401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:35.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.677+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f277c057770 con 0x7f2790069140 2024-04-24T08:35:35.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.992+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f279004f2a0 con 0x7f2790069140 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.994+0000 7f278d600700 1 -- 172.21.15.90:0/474422642 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f277c05ae30 con 0x7f2790069140 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:35.994 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 1 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 11 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:35:35.995 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 msgr2=0x7f2778071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 0x7f2778071170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2784006010 tx=0x7f278401a040 comp rx=0 tx=0).stop 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 msgr2=0x7f27901001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d8d0 tx=0x7f277c00dc90 comp rx=0 tx=0).stop 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 shutdown_connections 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f277806ecc0 0x7f2778071170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27900687a0 0x7f27900ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 --2- 172.21.15.90:0/474422642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2790069140 0x7f27901001e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:35.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.996+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 >> 172.21.15.90:0/474422642 conn(0x7f2790075130 msgr2=0x7f27900fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:35.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.997+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 shutdown_connections 2024-04-24T08:35:35.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:35.997+0000 7f2797600700 1 -- 172.21.15.90:0/474422642 wait complete. 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 -- 172.21.15.90:0/3549217264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504073a00 msgr2=0x7f4504110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 --2- 172.21.15.90:0/3549217264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504073a00 0x7f4504110d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009910 tx=0x7f44f0009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 -- 172.21.15.90:0/3549217264 shutdown_connections 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 --2- 172.21.15.90:0/3549217264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504073a00 0x7f4504110d50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 --2- 172.21.15.90:0/3549217264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45040730f0 0x7f45040734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 -- 172.21.15.90:0/3549217264 >> 172.21.15.90:0/3549217264 conn(0x7f45040fbda0 msgr2=0x7f45040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 -- 172.21.15.90:0/3549217264 shutdown_connections 2024-04-24T08:35:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.154+0000 7f4509a00700 1 -- 172.21.15.90:0/3549217264 wait complete. 2024-04-24T08:35:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.155+0000 7f4509a00700 1 Processor -- start 2024-04-24T08:35:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.155+0000 7f4509a00700 1 -- start start 2024-04-24T08:35:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4509a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4509a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504073a00 0x7f450419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4509a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450419d140 con 0x7f45040730f0 2024-04-24T08:35:36.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4509a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45041a0ed0 con 0x7f4504073a00 2024-04-24T08:35:36.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51158/0 (socket says 172.21.15.90:51158) 2024-04-24T08:35:36.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502a00700 1 -- 172.21.15.90:0/2229256263 learned_addr learned my addr 172.21.15.90:0/2229256263 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502a00700 1 -- 172.21.15.90:0/2229256263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504073a00 msgr2=0x7f450419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502000700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504073a00 0x7f450419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.156+0000 7f4502a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504073a00 0x7f450419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f4502a00700 1 -- 172.21.15.90:0/2229256263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f00095f0 con 0x7f45040730f0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f4502a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b4b0 tx=0x7f44f400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f4011800 con 0x7f45040730f0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f4011e20 con 0x7f45040730f0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45041a11b0 con 0x7f45040730f0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45041a1700 con 0x7f45040730f0 2024-04-24T08:35:36.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.157+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f400f2e0 con 0x7f45040730f0 2024-04-24T08:35:36.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.158+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450410e4d0 con 0x7f45040730f0 2024-04-24T08:35:36.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.159+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f44f4011960 con 0x7f45040730f0 2024-04-24T08:35:36.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.160+0000 7f44fb400700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 0x7f44e8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.160+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f44f408e9a0 con 0x7f45040730f0 2024-04-24T08:35:36.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.161+0000 7f4502000700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 0x7f44e8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.161+0000 7f4502000700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 0x7f44e8071170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44f0006010 tx=0x7f44f000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:36.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.165+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f44f40577b0 con 0x7f45040730f0 2024-04-24T08:35:36.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:36 smithi090 ceph-mon[85282]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:36.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:36 smithi090 ceph-mon[85282]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:36.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:36 smithi090 ceph-mon[85282]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:36.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:36 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/474422642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.447+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f45041a19d0 con 0x7f45040730f0 2024-04-24T08:35:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.448+0000 7f44fb400700 1 -- 172.21.15.90:0/2229256263 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1714 (secure 0 0 0) 0x7f44f405ae70 con 0x7f45040730f0 2024-04-24T08:35:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:e22 2024-04-24T08:35:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:35:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:35:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 18 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:35:00.123022+0000 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:35:36.450 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24223} 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:35:36.451 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 24223 members: 24223 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{0:24223} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.149:6824/1555648344,v1:172.21.15.149:6825/1555648344] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:24237} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:35:36.452 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 msgr2=0x7f44e8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 0x7f44e8071170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44f0006010 tx=0x7f44f000b540 comp rx=0 tx=0).stop 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 msgr2=0x7f450419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b4b0 tx=0x7f44f400b870 comp rx=0 tx=0).stop 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 shutdown_connections 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f44e806ecc0 0x7f44e8071170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45040730f0 0x7f450419c520 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 --2- 172.21.15.90:0/2229256263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504073a00 0x7f450419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 >> 172.21.15.90:0/2229256263 conn(0x7f45040fbda0 msgr2=0x7f4504102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:36.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 shutdown_connections 2024-04-24T08:35:36.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.451+0000 7f4509a00700 1 -- 172.21.15.90:0/2229256263 wait complete. 2024-04-24T08:35:36.454 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 22 2024-04-24T08:35:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:36 smithi149 ceph-mon[66490]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:36 smithi149 ceph-mon[66490]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:36 smithi149 ceph-mon[66490]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:36.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:36 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/474422642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:36.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 -- 172.21.15.90:0/652382592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 msgr2=0x7f6e2810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 --2- 172.21.15.90:0/652382592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2810ccc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009910 tx=0x7f6e18009c20 comp rx=0 tx=0).stop 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 -- 172.21.15.90:0/652382592 shutdown_connections 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 --2- 172.21.15.90:0/652382592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 0x7f6e28106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 --2- 172.21.15.90:0/652382592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2810ccc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 -- 172.21.15.90:0/652382592 >> 172.21.15.90:0/652382592 conn(0x7f6e28076060 msgr2=0x7f6e28078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 -- 172.21.15.90:0/652382592 shutdown_connections 2024-04-24T08:35:36.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.614+0000 7f6e2f200700 1 -- 172.21.15.90:0/652382592 wait complete. 2024-04-24T08:35:36.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 Processor -- start 2024-04-24T08:35:36.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 -- start start 2024-04-24T08:35:36.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 0x7f6e2819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819d100 con 0x7f6e2810c8f0 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.615+0000 7f6e2f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e281a0e90 con 0x7f6e28106b80 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51174/0 (socket says 172.21.15.90:51174) 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 -- 172.21.15.90:0/239187292 learned_addr learned my addr 172.21.15.90:0/239187292 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e27e00700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 0x7f6e2819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 -- 172.21.15.90:0/239187292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 msgr2=0x7f6e2819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 0x7f6e2819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 -- 172.21.15.90:0/239187292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e180095f0 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.616+0000 7f6e1fe00700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b4b0 tx=0x7f6e1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.617+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e10011800 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.617+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e281a1170 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.617+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e10011e20 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.617+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1000f2e0 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.617+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281a16c0 con 0x7f6e2810c8f0 2024-04-24T08:35:36.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.618+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28109b20 con 0x7f6e2810c8f0 2024-04-24T08:35:36.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.619+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6e10010380 con 0x7f6e2810c8f0 2024-04-24T08:35:36.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.620+0000 7f6e25600700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 0x7f6e0c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:36.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.620+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6e1008ea40 con 0x7f6e2810c8f0 2024-04-24T08:35:36.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.620+0000 7f6e27e00700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 0x7f6e0c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:36.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.621+0000 7f6e27e00700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 0x7f6e0c071170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e18006010 tx=0x7f6e1801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.624+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6e100577b0 con 0x7f6e2810c8f0 2024-04-24T08:35:36.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.887+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e281a1940 con 0x7f6e0c06ecc0 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.889+0000 7f6e25600700 1 -- 172.21.15.90:0/239187292 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6e281a1940 con 0x7f6e0c06ecc0 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:35:36.890 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "13/23 daemons upgraded", 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:35:36.891 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 msgr2=0x7f6e0c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 0x7f6e0c071170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e18006010 tx=0x7f6e1801a040 comp rx=0 tx=0).stop 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 msgr2=0x7f6e2819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b4b0 tx=0x7f6e1000b870 comp rx=0 tx=0).stop 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 shutdown_connections 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6e0c06ecc0 0x7f6e0c071170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e28106b80 0x7f6e2819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 --2- 172.21.15.90:0/239187292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e2810c8f0 0x7f6e2819ca20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:36.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.892+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 >> 172.21.15.90:0/239187292 conn(0x7f6e28076060 msgr2=0x7f6e28077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:36.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.894+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 shutdown_connections 2024-04-24T08:35:36.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:36.894+0000 7f6e2f200700 1 -- 172.21.15.90:0/239187292 wait complete. 2024-04-24T08:35:37.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.062+0000 7f0e6da00700 1 -- 172.21.15.90:0/658591654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ff890 msgr2=0x7f0e680ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:37.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.062+0000 7f0e6da00700 1 --2- 172.21.15.90:0/658591654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ff890 0x7f0e680ffd00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009940 tx=0x7f0e54009c50 comp rx=0 tx=0).stop 2024-04-24T08:35:37.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 -- 172.21.15.90:0/658591654 shutdown_connections 2024-04-24T08:35:37.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 --2- 172.21.15.90:0/658591654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ff890 0x7f0e680ffd00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 --2- 172.21.15.90:0/658591654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e68075e60 0x7f0e680ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 -- 172.21.15.90:0/658591654 >> 172.21.15.90:0/658591654 conn(0x7f0e680748b0 msgr2=0x7f0e68074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:37.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 -- 172.21.15.90:0/658591654 shutdown_connections 2024-04-24T08:35:37.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.063+0000 7f0e6da00700 1 -- 172.21.15.90:0/658591654 wait complete. 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 Processor -- start 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 -- start start 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e680ff890 0x7f0e68198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68198be0 con 0x7f0e68075e60 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.064+0000 7f0e6da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68198d20 con 0x7f0e680ff890 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:37.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51192/0 (socket says 172.21.15.90:51192) 2024-04-24T08:35:37.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 -- 172.21.15.90:0/2967817927 learned_addr learned my addr 172.21.15.90:0/2967817927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:35:37.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66000700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e680ff890 0x7f0e68198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 -- 172.21.15.90:0/2967817927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e680ff890 msgr2=0x7f0e68198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e680ff890 0x7f0e68198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 -- 172.21.15.90:0/2967817927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540095f0 con 0x7f0e68075e60 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.065+0000 7f0e66a00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cfd0 tx=0x7f0e5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.066+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5800ad70 con 0x7f0e68075e60 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.066+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e58004020 con 0x7f0e68075e60 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.066+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e58012650 con 0x7f0e68075e60 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.066+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6819cb70 con 0x7f0e68075e60 2024-04-24T08:35:37.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.066+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6819d090 con 0x7f0e68075e60 2024-04-24T08:35:37.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.068+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6804f2a0 con 0x7f0e68075e60 2024-04-24T08:35:37.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.068+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0e58004190 con 0x7f0e68075e60 2024-04-24T08:35:37.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.069+0000 7f0e5f400700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 0x7f0e4c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:35:37.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.069+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0e5808efe0 con 0x7f0e68075e60 2024-04-24T08:35:37.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.070+0000 7f0e66000700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 0x7f0e4c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:35:37.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.071+0000 7f0e66000700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 0x7f0e4c071170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e54005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:35:37.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.074+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0e58057df0 con 0x7f0e68075e60 2024-04-24T08:35:37.426 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:37 smithi090 ceph-mon[85282]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:37 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2229256263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:35:37.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.425+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0e6819d400 con 0x7f0e68075e60 2024-04-24T08:35:37.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.427+0000 7f0e5f400700 1 -- 172.21.15.90:0/2967817927 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0e5805b4b0 con 0x7f0e68075e60 2024-04-24T08:35:37.428 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:35:37.428 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:35:37.428 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.430+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 msgr2=0x7f0e4c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.430+0000 7f0e6da00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 0x7f0e4c071170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e54005ad0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.430+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 msgr2=0x7f0e68198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.430+0000 7f0e6da00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cfd0 tx=0x7f0e5800a340 comp rx=0 tx=0).stop 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 shutdown_connections 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f0e4c06ecc0 0x7f0e4c071170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68075e60 0x7f0e68198050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 --2- 172.21.15.90:0/2967817927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e680ff890 0x7f0e68198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:35:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 >> 172.21.15.90:0/2967817927 conn(0x7f0e680748b0 msgr2=0x7f0e680fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:35:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 shutdown_connections 2024-04-24T08:35:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:35:37.431+0000 7f0e6da00700 1 -- 172.21.15.90:0/2967817927 wait complete. 2024-04-24T08:35:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:37 smithi149 ceph-mon[66490]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:37 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2229256263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:35:38.379 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:38 smithi090 ceph-mon[85282]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:38.379 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:38 smithi090 ceph-mon[85282]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:38.380 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:38 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2967817927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:35:38.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:38 smithi149 ceph-mon[66490]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:35:38.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:38 smithi149 ceph-mon[66490]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:38.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:38 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2967817927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:35:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] up:boot 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:39.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:39 smithi149 ceph-mon[66490]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] up:boot 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.eazcmc=up:active} 3 up:standby 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:39.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:39 smithi090 ceph-mon[85282]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:42.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:42 smithi149 ceph-mon[66490]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:42.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:42 smithi090 ceph-mon[85282]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:44 smithi149 ceph-mon[66490]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:44.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:44 smithi090 ceph-mon[85282]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:45.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:35:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:46 smithi149 ceph-mon[66490]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:46.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:46 smithi090 ceph-mon[85282]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:47.585 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:47 smithi149 ceph-mon[66490]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:47.586 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:47 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:47.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:47 smithi090 ceph-mon[85282]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:47.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:47 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:48.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:48 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:48.586 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:48 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:49.588 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:49 smithi090 ceph-mon[85282]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:49 smithi149 ceph-mon[66490]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:52.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:52 smithi149 ceph-mon[66490]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:52.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:52 smithi090 ceph-mon[85282]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:54.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:54.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:35:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:56 smithi149 ceph-mon[66490]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:56 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:56.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:56 smithi090 ceph-mon[85282]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:35:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.eazcmc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:35:56.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:56 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:35:57.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:57 smithi149 ceph-mon[66490]: Upgrade: Updating mds.cephfs.smithi149.eazcmc 2024-04-24T08:35:57.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:57 smithi149 ceph-mon[66490]: Deploying daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:35:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:57 smithi090 ceph-mon[85282]: Upgrade: Updating mds.cephfs.smithi149.eazcmc 2024-04-24T08:35:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:57 smithi090 ceph-mon[85282]: Deploying daemon mds.cephfs.smithi149.eazcmc on smithi149 2024-04-24T08:35:58.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:58 smithi149 ceph-mon[66490]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:58.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:58 smithi090 ceph-mon[85282]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:35:59.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[85278]: 2024-04-24T08:35:59.005+0000 7f9cc0800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:35:59.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:35:59.259 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: osdmap e85: 6 total, 6 up, 6 in 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: Standby daemon mds.cephfs.smithi149.zwjtmw assigned to filesystem cephfs as rank 0 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T08:35:59.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:35:59 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:replay} 2 up:standby 2024-04-24T08:35:59.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: osdmap e85: 6 total, 6 up, 6 in 2024-04-24T08:35:59.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: Standby daemon mds.cephfs.smithi149.zwjtmw assigned to filesystem cephfs as rank 0 2024-04-24T08:35:59.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:35:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T08:35:59.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:35:59 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:replay} 2 up:standby 2024-04-24T08:36:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:00 smithi149 ceph-mon[66490]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:36:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:00 smithi090 ceph-mon[85282]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:36:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:01 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:01 smithi149 ceph-mon[66490]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-04-24T08:36:01.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:01 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:01.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:01 smithi090 ceph-mon[85282]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-04-24T08:36:04.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:04 smithi149 ceph-mon[66490]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-24T08:36:04.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:04 smithi090 ceph-mon[85282]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-24T08:36:06.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:06 smithi090 ceph-mon[85282]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-24T08:36:06.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:06 smithi149 ceph-mon[66490]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 -- 172.21.15.90:0/78099797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a80687a0 msgr2=0x7f61a8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 --2- 172.21.15.90:0/78099797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a80687a0 0x7f61a8068b70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f619c009910 tx=0x7f619c009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 -- 172.21.15.90:0/78099797 shutdown_connections 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 --2- 172.21.15.90:0/78099797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a8069140 0x7f61a8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 --2- 172.21.15.90:0/78099797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a80687a0 0x7f61a8068b70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 -- 172.21.15.90:0/78099797 >> 172.21.15.90:0/78099797 conn(0x7f61a8075130 msgr2=0x7f61a8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 -- 172.21.15.90:0/78099797 shutdown_connections 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.601+0000 7f61b0400700 1 -- 172.21.15.90:0/78099797 wait complete. 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.602+0000 7f61b0400700 1 Processor -- start 2024-04-24T08:36:07.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61b0400700 1 -- start start 2024-04-24T08:36:07.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61b0400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80687a0 0x7f61a8195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:07.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61b0400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61b0400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8196ab0 con 0x7f61a8069140 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61b0400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a819a840 con 0x7f61a80687a0 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61ad000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61ad000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57822/0 (socket says 172.21.15.90:57822) 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61ad000700 1 -- 172.21.15.90:0/3644445505 learned_addr learned my addr 172.21.15.90:0/3644445505 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.603+0000 7f61ada00700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80687a0 0x7f61a8195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.604+0000 7f61ad000700 1 -- 172.21.15.90:0/3644445505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80687a0 msgr2=0x7f61a8195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.604+0000 7f61ad000700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80687a0 0x7f61a8195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.604+0000 7f61ad000700 1 -- 172.21.15.90:0/3644445505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c0095f0 con 0x7f61a8069140 2024-04-24T08:36:07.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.604+0000 7f61ad000700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f619400d8d0 tx=0x7f619400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.605+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6194004a00 con 0x7f61a8069140 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.605+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a819ab20 con 0x7f61a8069140 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.605+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61940054e0 con 0x7f61a8069140 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.605+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f619400fc20 con 0x7f61a8069140 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.605+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a819b070 con 0x7f61a8069140 2024-04-24T08:36:07.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.606+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a8108770 con 0x7f61a8069140 2024-04-24T08:36:07.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.607+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f619400fd80 con 0x7f61a8069140 2024-04-24T08:36:07.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.608+0000 7f61a6000700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 0x7f6190071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:07.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.608+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f619408e6d0 con 0x7f61a8069140 2024-04-24T08:36:07.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.608+0000 7f61ada00700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 0x7f6190071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:07.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.609+0000 7f61ada00700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 0x7f6190071170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:07.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.615+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6194057460 con 0x7f61a8069140 2024-04-24T08:36:07.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.866+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61a819b350 con 0x7f619006ecc0 2024-04-24T08:36:07.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.869+0000 7f61a6000700 1 -- 172.21.15.90:0/3644445505 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f61a819b350 con 0x7f619006ecc0 2024-04-24T08:36:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 msgr2=0x7f6190071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 0x7f6190071170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c01a040 comp rx=0 tx=0).stop 2024-04-24T08:36:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 msgr2=0x7f61a81963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f619400d8d0 tx=0x7f619400dbe0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 shutdown_connections 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f619006ecc0 0x7f6190071170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80687a0 0x7f61a8195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 --2- 172.21.15.90:0/3644445505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61a8069140 0x7f61a81963d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 >> 172.21.15.90:0/3644445505 conn(0x7f61a8075130 msgr2=0x7f61a80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 shutdown_connections 2024-04-24T08:36:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:07.873+0000 7f61b0400700 1 -- 172.21.15.90:0/3644445505 wait complete. 2024-04-24T08:36:07.891 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:36:08.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.033+0000 7fa275e00700 1 -- 172.21.15.90:0/3604287209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa270100e80 msgr2=0x7fa270104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.033+0000 7fa275e00700 1 --2- 172.21.15.90:0/3604287209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa270100e80 0x7fa270104e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa258009910 tx=0x7fa258009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.033+0000 7fa275e00700 1 -- 172.21.15.90:0/3604287209 shutdown_connections 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.034+0000 7fa275e00700 1 --2- 172.21.15.90:0/3604287209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa270100e80 0x7fa270104e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.034+0000 7fa275e00700 1 --2- 172.21.15.90:0/3604287209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2701004e0 0x7fa2701008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.034+0000 7fa275e00700 1 -- 172.21.15.90:0/3604287209 >> 172.21.15.90:0/3604287209 conn(0x7fa2700fbda0 msgr2=0x7fa2700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.034+0000 7fa275e00700 1 -- 172.21.15.90:0/3604287209 shutdown_connections 2024-04-24T08:36:08.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.034+0000 7fa275e00700 1 -- 172.21.15.90:0/3604287209 wait complete. 2024-04-24T08:36:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 Processor -- start 2024-04-24T08:36:08.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 -- start start 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa270100e80 0x7fa270196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270196b50 con 0x7fa2701004e0 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.035+0000 7fa275e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019a8e0 con 0x7fa270100e80 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57836/0 (socket says 172.21.15.90:57836) 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 -- 172.21.15.90:0/3975583682 learned_addr learned my addr 172.21.15.90:0/3975583682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa266000700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa270100e80 0x7fa270196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 -- 172.21.15.90:0/3975583682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa270100e80 msgr2=0x7fa270196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa270100e80 0x7fa270196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 -- 172.21.15.90:0/3975583682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580095f0 con 0x7fa2701004e0 2024-04-24T08:36:08.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.036+0000 7fa26ea00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2600060b0 tx=0x7fa26000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.037+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26000dda0 con 0x7fa2701004e0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.037+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa26000f040 con 0x7fa2701004e0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.037+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa260014790 con 0x7fa2701004e0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.037+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27019abc0 con 0x7fa2701004e0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.037+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27019b110 con 0x7fa2701004e0 2024-04-24T08:36:08.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.038+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa270108850 con 0x7fa2701004e0 2024-04-24T08:36:08.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.039+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa2600148f0 con 0x7fa2701004e0 2024-04-24T08:36:08.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.040+0000 7fa274a00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 0x7fa254071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.040+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa26008e8d0 con 0x7fa2701004e0 2024-04-24T08:36:08.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.040+0000 7fa266000700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 0x7fa254071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.041+0000 7fa266000700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 0x7fa254071170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa258006010 tx=0x7fa258005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.045+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa260057710 con 0x7fa2701004e0 2024-04-24T08:36:08.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.295+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27019b450 con 0x7fa25406ecc0 2024-04-24T08:36:08.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.298+0000 7fa274a00700 1 -- 172.21.15.90:0/3975583682 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa27019b450 con 0x7fa25406ecc0 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 msgr2=0x7fa254071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 0x7fa254071170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa258006010 tx=0x7fa258005960 comp rx=0 tx=0).stop 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 msgr2=0x7fa270195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2600060b0 tx=0x7fa26000d3b0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 shutdown_connections 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa25406ecc0 0x7fa254071170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2701004e0 0x7fa270195f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 --2- 172.21.15.90:0/3975583682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa270100e80 0x7fa270196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.301+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 >> 172.21.15.90:0/3975583682 conn(0x7fa2700fbda0 msgr2=0x7fa2700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:08.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.302+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 shutdown_connections 2024-04-24T08:36:08.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.302+0000 7fa275e00700 1 -- 172.21.15.90:0/3975583682 wait complete. 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 -- 172.21.15.90:0/2042799988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 msgr2=0x7fbbec1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/2042799988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1028e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009910 tx=0x7fbbd8009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 -- 172.21.15.90:0/2042799988 shutdown_connections 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/2042799988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1028e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/2042799988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 0x7fbbec108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.461+0000 7fbbf0a00700 1 -- 172.21.15.90:0/2042799988 >> 172.21.15.90:0/2042799988 conn(0x7fbbec0fdf90 msgr2=0x7fbbec1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.462+0000 7fbbf0a00700 1 -- 172.21.15.90:0/2042799988 shutdown_connections 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.462+0000 7fbbf0a00700 1 -- 172.21.15.90:0/2042799988 wait complete. 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 Processor -- start 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 -- start start 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 0x7fbbec198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec198e10 con 0x7fbbec102470 2024-04-24T08:36:08.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbf0a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec19cba0 con 0x7fbbec108470 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.463+0000 7fbbe9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57854/0 (socket says 172.21.15.90:57854) 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 -- 172.21.15.90:0/3294831553 learned_addr learned my addr 172.21.15.90:0/3294831553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9400700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 0x7fbbec198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 -- 172.21.15.90:0/3294831553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 msgr2=0x7fbbec198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 0x7fbbec198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 -- 172.21.15.90:0/3294831553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80095f0 con 0x7fbbec102470 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.464+0000 7fbbe9e00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00d8d0 tx=0x7fbbdc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.465+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc0098a0 con 0x7fbbec102470 2024-04-24T08:36:08.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.465+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbdc010460 con 0x7fbbec102470 2024-04-24T08:36:08.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.465+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec19ce80 con 0x7fbbec102470 2024-04-24T08:36:08.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.465+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc00f5f0 con 0x7fbbec102470 2024-04-24T08:36:08.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.465+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec19d3d0 con 0x7fbbec102470 2024-04-24T08:36:08.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.466+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec10a970 con 0x7fbbec102470 2024-04-24T08:36:08.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.467+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbbdc009a00 con 0x7fbbec102470 2024-04-24T08:36:08.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.468+0000 7fbbe2a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 0x7fbbd0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.468+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbbdc08edd0 con 0x7fbbec102470 2024-04-24T08:36:08.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.469+0000 7fbbe9400700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 0x7fbbd0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.469+0000 7fbbe9400700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 0x7fbbd0071170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbec199810 tx=0x7fbbd801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.473+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbbdc057ae0 con 0x7fbbec102470 2024-04-24T08:36:08.507 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:08 smithi149 ceph-mon[66490]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T08:36:08.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:08 smithi090 ceph-mon[85282]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T08:36:08.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.721+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbec199690 con 0x7fbbd006ecc0 2024-04-24T08:36:08.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.753+0000 7fbbe2a00700 1 -- 172.21.15.90:0/3294831553 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbbd801e040 con 0x7fbbd006ecc0 2024-04-24T08:36:08.754 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 23s ago 23m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (23m) 23s ago 23m 9466k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (22m) 79s ago 22m 8988k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (5m) 23s ago 23m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (5m) 79s ago 21m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (8m) 23s ago 22m 73.8M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (30s) 23s ago 15m 10.9M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 247c152849b1 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (58s) 23s ago 15m 12.0M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:36:08.755 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (15m) 79s ago 15m 151M - 18.2.0 dc2bc1663786 a7a7d755e323 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (15m) 79s ago 15m 18.6M - 18.2.0 dc2bc1663786 c10e504b633f 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (12m) 23s ago 24m 511M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (10m) 79s ago 21m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (7m) 23s ago 25m 56.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (6m) 79s ago 21m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (10m) 23s ago 23m 8040k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (10m) 79s ago 21m 7730k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (4m) 23s ago 20m 93.4M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (3m) 23s ago 20m 87.5M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (3m) 23s ago 19m 76.2M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (2m) 79s ago 18m 89.2M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (2m) 79s ago 18m 89.9M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:36:08.756 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (85s) 79s ago 17m 13.3M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:36:08.757 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (9m) 23s ago 22m 76.0M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 msgr2=0x7fbbd0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 0x7fbbd0071170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbec199810 tx=0x7fbbd801a040 comp rx=0 tx=0).stop 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 msgr2=0x7fbbec1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00d8d0 tx=0x7fbbdc00dbe0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 shutdown_connections 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbbd006ecc0 0x7fbbd0071170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbec102470 0x7fbbec1981f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 --2- 172.21.15.90:0/3294831553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbec108470 0x7fbbec198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 >> 172.21.15.90:0/3294831553 conn(0x7fbbec0fdf90 msgr2=0x7fbbec0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 shutdown_connections 2024-04-24T08:36:08.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.756+0000 7fbbf0a00700 1 -- 172.21.15.90:0/3294831553 wait complete. 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 -- 172.21.15.90:0/2873290837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 msgr2=0x7f3aa4110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2873290837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa4110b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009910 tx=0x7f3a8c009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 -- 172.21.15.90:0/2873290837 shutdown_connections 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2873290837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa4110b00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2873290837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 0x7f3aa40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 -- 172.21.15.90:0/2873290837 >> 172.21.15.90:0/2873290837 conn(0x7f3aa40fbb10 msgr2=0x7f3aa40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 -- 172.21.15.90:0/2873290837 shutdown_connections 2024-04-24T08:36:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.907+0000 7f3aa9600700 1 -- 172.21.15.90:0/2873290837 wait complete. 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.908+0000 7f3aa9600700 1 Processor -- start 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.908+0000 7f3aa9600700 1 -- start start 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3aa9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 0x7f3aa419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3aa9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3aa9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa419cdc0 con 0x7f3aa4073a00 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3aa9600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41a0b50 con 0x7f3aa40730f0 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3a9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3a9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57880/0 (socket says 172.21.15.90:57880) 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3a9be00700 1 -- 172.21.15.90:0/2877119467 learned_addr learned my addr 172.21.15.90:0/2877119467 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.909+0000 7f3aa3e00700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 0x7f3aa419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3a9be00700 1 -- 172.21.15.90:0/2877119467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 msgr2=0x7f3aa419c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3a9be00700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 0x7f3aa419c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:08.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3a9be00700 1 -- 172.21.15.90:0/2877119467 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0095f0 con 0x7f3aa4073a00 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3a9be00700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c01d070 con 0x7f3aa4073a00 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c00f460 con 0x7f3aa4073a00 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c017610 con 0x7f3aa4073a00 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa41a0e30 con 0x7f3aa4073a00 2024-04-24T08:36:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.910+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa41a1380 con 0x7f3aa4073a00 2024-04-24T08:36:08.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.912+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa410e280 con 0x7f3aa4073a00 2024-04-24T08:36:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.913+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3a8c00f5d0 con 0x7f3aa4073a00 2024-04-24T08:36:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.913+0000 7f3aa1600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 0x7f3a88071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.914+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3a8c090350 con 0x7f3aa4073a00 2024-04-24T08:36:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.914+0000 7f3aa3e00700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 0x7f3a88071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.915+0000 7f3aa3e00700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 0x7f3a88071170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a94005950 tx=0x7f3a940058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:08.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:08.919+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3a8c059120 con 0x7f3aa4073a00 2024-04-24T08:36:09.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.229+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3aa4062380 con 0x7f3aa4073a00 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.229+0000 7f3aa1600700 1 -- 172.21.15.90:0/2877119467 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f3a8c05c7e0 con 0x7f3aa4073a00 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:09.231 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 12 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:36:09.232 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 msgr2=0x7f3a88071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 0x7f3a88071170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a94005950 tx=0x7f3a940058e0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 msgr2=0x7f3aa419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c00be20 comp rx=0 tx=0).stop 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 shutdown_connections 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3a8806ecc0 0x7f3a88071170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3aa40730f0 0x7f3aa419c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 --2- 172.21.15.90:0/2877119467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa4073a00 0x7f3aa419c6e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 >> 172.21.15.90:0/2877119467 conn(0x7f3aa40fbb10 msgr2=0x7f3aa4102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.232+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 shutdown_connections 2024-04-24T08:36:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.233+0000 7f3aa9600700 1 -- 172.21.15.90:0/2877119467 wait complete. 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3227211264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 msgr2=0x7fbb541006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3227211264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb541006a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009910 tx=0x7fbb48009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3227211264 shutdown_connections 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3227211264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3227211264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb541006a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3227211264 >> 172.21.15.90:0/3227211264 conn(0x7fbb540fbbb0 msgr2=0x7fbb540fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3227211264 shutdown_connections 2024-04-24T08:36:09.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.391+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3227211264 wait complete. 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.392+0000 7fbb5ae00700 1 Processor -- start 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.392+0000 7fbb5ae00700 1 -- start start 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb5ae00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb5ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb5ae00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54198ae0 con 0x7fbb541002d0 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb5ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5419c870 con 0x7fbb54100c70 2024-04-24T08:36:09.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57898/0 (socket says 172.21.15.90:57898) 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 -- 172.21.15.90:0/3515265767 learned_addr learned my addr 172.21.15.90:0/3515265767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59000700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 -- 172.21.15.90:0/3515265767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 msgr2=0x7fbb54198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59a00700 1 -- 172.21.15.90:0/3515265767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480095f0 con 0x7fbb541002d0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.393+0000 7fbb59000700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb59a00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800f690 tx=0x7fbb4800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb4801d070 con 0x7fbb541002d0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb480050d0 con 0x7fbb541002d0 2024-04-24T08:36:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5419caf0 con 0x7fbb541002d0 2024-04-24T08:36:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb48022470 con 0x7fbb541002d0 2024-04-24T08:36:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.394+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5419cfe0 con 0x7fbb541002d0 2024-04-24T08:36:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.396+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb541043c0 con 0x7fbb541002d0 2024-04-24T08:36:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.396+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbb48003680 con 0x7fbb541002d0 2024-04-24T08:36:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.397+0000 7fbb52000700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 0x7fbb3c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.397+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbb48090320 con 0x7fbb541002d0 2024-04-24T08:36:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.398+0000 7fbb59000700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 0x7fbb3c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.398+0000 7fbb59000700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 0x7fbb3c071170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb541994e0 tx=0x7fbb40009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:09.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.402+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbb480590d0 con 0x7fbb541002d0 2024-04-24T08:36:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:09 smithi149 ceph-mon[66490]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:09 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2877119467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:09.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:09 smithi090 ceph-mon[85282]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:09.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:09 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2877119467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:09.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.678+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb54103da0 con 0x7fbb541002d0 2024-04-24T08:36:09.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.678+0000 7fbb52000700 1 -- 172.21.15.90:0/3515265767 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1710 (secure 0 0 0) 0x7fbb4805c790 con 0x7fbb541002d0 2024-04-24T08:36:09.680 INFO:teuthology.orchestra.run.smithi090.stdout:e26 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 26 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:36:08.621894+0000 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:36:09.681 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 85 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24237} 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:36:09.682 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{0:24237} state up:reconnect seq 237 join_fscid=1 addr [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:09.683 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{-1:34494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.681+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 msgr2=0x7fbb3c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.681+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 0x7fbb3c071170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb541994e0 tx=0x7fbb40009260 comp rx=0 tx=0).stop 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.681+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 msgr2=0x7fbb54197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.681+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800f690 tx=0x7fbb4800f770 comp rx=0 tx=0).stop 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 shutdown_connections 2024-04-24T08:36:09.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbb3c06ecc0 0x7fbb3c071170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb541002d0 0x7fbb54197ec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 --2- 172.21.15.90:0/3515265767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb54100c70 0x7fbb54198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 >> 172.21.15.90:0/3515265767 conn(0x7fbb540fbbb0 msgr2=0x7fbb540fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 shutdown_connections 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.682+0000 7fbb5ae00700 1 -- 172.21.15.90:0/3515265767 wait complete. 2024-04-24T08:36:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-24T08:36:09.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.835+0000 7f4e1cc00700 1 -- 172.21.15.90:0/2737363775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e18103040 msgr2=0x7f4e18107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.835+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/2737363775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e18103040 0x7f4e18107010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009910 tx=0x7f4e04009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:09.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 -- 172.21.15.90:0/2737363775 shutdown_connections 2024-04-24T08:36:09.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/2737363775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e18103040 0x7f4e18107010 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/2737363775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e181026a0 0x7f4e18102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 -- 172.21.15.90:0/2737363775 >> 172.21.15.90:0/2737363775 conn(0x7f4e180fdf40 msgr2=0x7f4e18100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 -- 172.21.15.90:0/2737363775 shutdown_connections 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.836+0000 7f4e1cc00700 1 -- 172.21.15.90:0/2737363775 wait complete. 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.837+0000 7f4e1cc00700 1 Processor -- start 2024-04-24T08:36:09.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.837+0000 7f4e1cc00700 1 -- start start 2024-04-24T08:36:09.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e1cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e1cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e18103040 0x7f4e18198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e1cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e18198be0 con 0x7f4e181026a0 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e1cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e18198d20 con 0x7f4e18103040 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38342/0 (socket says 172.21.15.90:38342) 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 -- 172.21.15.90:0/3522421887 learned_addr learned my addr 172.21.15.90:0/3522421887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15400700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e18103040 0x7f4e18198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 -- 172.21.15.90:0/3522421887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e18103040 msgr2=0x7f4e18198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e18103040 0x7f4e18198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:09.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.838+0000 7f4e15e00700 1 -- 172.21.15.90:0/3522421887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040095f0 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e15e00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c930 tx=0x7f4e0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e08007ad0 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e0800ce80 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1819cb70 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e08018630 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.839+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1819d0c0 con 0x7f4e181026a0 2024-04-24T08:36:09.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.840+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1810aa10 con 0x7f4e181026a0 2024-04-24T08:36:09.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.841+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4e08007c30 con 0x7f4e181026a0 2024-04-24T08:36:09.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.842+0000 7f4e0ea00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 0x7f4dfc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:09.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.842+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4e0808ee40 con 0x7f4e181026a0 2024-04-24T08:36:09.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.842+0000 7f4e15400700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 0x7f4dfc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:09.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.843+0000 7f4e15400700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 0x7f4dfc071170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e18199670 tx=0x7f4e0401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:09.846+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4e08057bd0 con 0x7f4e181026a0 2024-04-24T08:36:10.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.096+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e1819d390 con 0x7f4dfc06ecc0 2024-04-24T08:36:10.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.098+0000 7f4e0ea00700 1 -- 172.21.15.90:0/3522421887 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e1819d390 con 0x7f4dfc06ecc0 2024-04-24T08:36:10.099 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:36:10.099 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:36:10.099 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "14/23 daemons upgraded", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:36:10.100 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 msgr2=0x7f4dfc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 0x7f4dfc071170 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e18199670 tx=0x7f4e0401a040 comp rx=0 tx=0).stop 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 msgr2=0x7f4e18198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c930 tx=0x7f4e0800cc40 comp rx=0 tx=0).stop 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 shutdown_connections 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dfc06ecc0 0x7f4dfc071170 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e181026a0 0x7f4e18198050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 --2- 172.21.15.90:0/3522421887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e18103040 0x7f4e18198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.101+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 >> 172.21.15.90:0/3522421887 conn(0x7f4e180fdf40 msgr2=0x7f4e18100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:10.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.102+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 shutdown_connections 2024-04-24T08:36:10.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.102+0000 7f4e1cc00700 1 -- 172.21.15.90:0/3522421887 wait complete. 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 -- 172.21.15.90:0/1199161250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 msgr2=0x7f4dd81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 --2- 172.21.15.90:0/1199161250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd81008b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009910 tx=0x7f4dcc009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 -- 172.21.15.90:0/1199161250 shutdown_connections 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 --2- 172.21.15.90:0/1199161250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 --2- 172.21.15.90:0/1199161250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd81008b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 -- 172.21.15.90:0/1199161250 >> 172.21.15.90:0/1199161250 conn(0x7f4dd80fbda0 msgr2=0x7f4dd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 -- 172.21.15.90:0/1199161250 shutdown_connections 2024-04-24T08:36:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.257+0000 7f4de0800700 1 -- 172.21.15.90:0/1199161250 wait complete. 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.258+0000 7f4de0800700 1 Processor -- start 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.258+0000 7f4de0800700 1 -- start start 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4de0800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4de0800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4de0800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8073ad0 con 0x7f4dd81004e0 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4de0800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8073c10 con 0x7f4dd8100e80 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4ddde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4ddde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38360/0 (socket says 172.21.15.90:38360) 2024-04-24T08:36:10.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.259+0000 7f4ddde00700 1 -- 172.21.15.90:0/2528920100 learned_addr learned my addr 172.21.15.90:0/2528920100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddd400700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddde00700 1 -- 172.21.15.90:0/2528920100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 msgr2=0x7f4dd8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddde00700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddde00700 1 -- 172.21.15.90:0/2528920100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0095f0 con 0x7f4dd81004e0 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddd400700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:36:10.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4ddde00700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b5c0 tx=0x7f4dcc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dcc01d070 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dcc005e00 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8073e90 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dcc017400 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.260+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd81a23f0 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.261+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd81087f0 con 0x7f4dd81004e0 2024-04-24T08:36:10.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.262+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4dcc00f460 con 0x7f4dd81004e0 2024-04-24T08:36:10.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.263+0000 7f4dd2a00700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 0x7f4dc0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:10.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.264+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4dcc090350 con 0x7f4dd81004e0 2024-04-24T08:36:10.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.264+0000 7f4ddd400700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 0x7f4dc0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:10.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.264+0000 7f4ddd400700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 0x7f4dc0071170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005fd0 tx=0x7f4dc8005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:10.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.269+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4dcc0590e0 con 0x7f4dd81004e0 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: reconnect by client.24275 192.168.0.1:0/4244548942 after 0 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: reconnect by client.14534 192.168.0.1:0/2302411015 after 0.000999981 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:reconnect 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:reconnect} 2 up:standby 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3515265767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:36:10.456 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:10 smithi149 ceph-mon[66490]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: reconnect by client.24275 192.168.0.1:0/4244548942 after 0 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: reconnect by client.14534 192.168.0.1:0/2302411015 after 0.000999981 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:reconnect 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:reconnect} 2 up:standby 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3515265767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:36:10.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:10 smithi090 ceph-mon[85282]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:10.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.588+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4dd80745b0 con 0x7f4dd81004e0 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.588+0000 7f4dd2a00700 1 -- 172.21.15.90:0/2528920100 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f4dcc05c7a0 con 0x7f4dd81004e0 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is degraded 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:36:10.589 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 msgr2=0x7f4dc0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 0x7f4dc0071170 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005fd0 tx=0x7f4dc8005ee0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 msgr2=0x7f4dd8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b5c0 tx=0x7f4dcc00be20 comp rx=0 tx=0).stop 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 shutdown_connections 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4dc006ecc0 0x7f4dc0071170 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dd81004e0 0x7f4dd8073050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 --2- 172.21.15.90:0/2528920100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4dd8100e80 0x7f4dd8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 >> 172.21.15.90:0/2528920100 conn(0x7f4dd80fbda0 msgr2=0x7f4dd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.590+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 shutdown_connections 2024-04-24T08:36:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:10.591+0000 7f4de0800700 1 -- 172.21.15.90:0/2528920100 wait complete. 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:rejoin 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:rejoin} 2 up:standby 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: daemon mds.cephfs.smithi149.zwjtmw is now active in filesystem cephfs as rank 0 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2528920100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:11.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:11.544 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:11 smithi090 ceph-mon[85282]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T08:36:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:rejoin 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi149.zwjtmw=up:rejoin} 2 up:standby 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: daemon mds.cephfs.smithi149.zwjtmw is now active in filesystem cephfs as rank 0 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2528920100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:11.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:11 smithi149 ceph-mon[66490]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T08:36:12.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:12 smithi090 ceph-mon[85282]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:36:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:12 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:active 2024-04-24T08:36:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:12 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.149:6824/4101562674,v1:172.21.15.149:6825/4101562674] up:boot 2024-04-24T08:36:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:12 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi149.zwjtmw=up:active} 3 up:standby 2024-04-24T08:36:12.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:12 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:12.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:12 smithi149 ceph-mon[66490]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:36:12.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:12 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.149:6826/2080624312,v1:172.21.15.149:6827/2080624312] up:active 2024-04-24T08:36:12.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:12 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.149:6824/4101562674,v1:172.21.15.149:6825/4101562674] up:boot 2024-04-24T08:36:12.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:12 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi149.zwjtmw=up:active} 3 up:standby 2024-04-24T08:36:12.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:12 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:13.573 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:13 smithi149 ceph-mon[66490]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-04-24T08:36:13.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:13 smithi090 ceph-mon[85282]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-04-24T08:36:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:16 smithi149 ceph-mon[66490]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T08:36:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:16.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:16 smithi090 ceph-mon[85282]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T08:36:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:17.605 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:17.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:17.606 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:18.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:18 smithi149 ceph-mon[66490]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T08:36:18.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:18 smithi090 ceph-mon[85282]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T08:36:19.625 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:19 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:19.625 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:19 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:19.625 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:19 smithi149 ceph-mon[66490]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-04-24T08:36:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:19 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:19 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:19.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:19 smithi090 ceph-mon[85282]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-04-24T08:36:22.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:21 smithi149 ceph-mon[66490]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-04-24T08:36:22.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:21 smithi090 ceph-mon[85282]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-04-24T08:36:24.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:24 smithi149 ceph-mon[66490]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-24T08:36:24.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:24 smithi090 ceph-mon[85282]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:25.938 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:25.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:25.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:25 smithi149 ceph-mon[66490]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-24T08:36:26.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:26.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:25 smithi090 ceph-mon[85282]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: Upgrade: Updating mds.cephfs.smithi149.zwjtmw 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:28.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:28 smithi149 ceph-mon[66490]: Deploying daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:36:28.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-04-24T08:36:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: Upgrade: Updating mds.cephfs.smithi149.zwjtmw 2024-04-24T08:36:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.zwjtmw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:36:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:28.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:28 smithi090 ceph-mon[85282]: Deploying daemon mds.cephfs.smithi149.zwjtmw on smithi149 2024-04-24T08:36:30.490 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:30 smithi149 ceph-mon[66490]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-04-24T08:36:30.490 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:30.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:30 smithi090 ceph-mon[85282]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-04-24T08:36:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:30.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:30 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[85278]: 2024-04-24T08:36:30.490+0000 7f9cc0800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: osdmap e86: 6 total, 6 up, 6 in 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: Standby daemon mds.cephfs.smithi090.qbyvpv assigned to filesystem cephfs as rank 0 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:replay} 2 up:standby 2024-04-24T08:36:31.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: osdmap e86: 6 total, 6 up, 6 in 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: Standby daemon mds.cephfs.smithi090.qbyvpv assigned to filesystem cephfs as rank 0 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:replay} 2 up:standby 2024-04-24T08:36:31.533 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:32.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:32 smithi149 ceph-mon[66490]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:36:32.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:32 smithi090 ceph-mon[85282]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-04-24T08:36:34.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:34 smithi149 ceph-mon[66490]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T08:36:34.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:34 smithi090 ceph-mon[85282]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T08:36:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:36 smithi149 ceph-mon[66490]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-24T08:36:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:36 smithi090 ceph-mon[85282]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-24T08:36:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:37 smithi149 ceph-mon[66490]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T08:36:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:37 smithi090 ceph-mon[85282]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.149:6826/1676173850,v1:172.21.15.149:6827/1676173850] up:boot 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:replay} 3 up:standby 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:40.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:39 smithi149 ceph-mon[66490]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T08:36:40.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.149:6826/1676173850,v1:172.21.15.149:6827/1676173850] up:boot 2024-04-24T08:36:40.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:replay} 3 up:standby 2024-04-24T08:36:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:40.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:39 smithi090 ceph-mon[85282]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T08:36:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 -- 172.21.15.90:0/76870727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 msgr2=0x7fbd44106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 --2- 172.21.15.90:0/76870727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd44106ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009910 tx=0x7fbd30009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 -- 172.21.15.90:0/76870727 shutdown_connections 2024-04-24T08:36:40.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 --2- 172.21.15.90:0/76870727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd44106ff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:40.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 --2- 172.21.15.90:0/76870727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd4410c8f0 0x7fbd4410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:40.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 -- 172.21.15.90:0/76870727 >> 172.21.15.90:0/76870727 conn(0x7fbd44076060 msgr2=0x7fbd44078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:40.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 -- 172.21.15.90:0/76870727 shutdown_connections 2024-04-24T08:36:40.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.756+0000 7fbd4a200700 1 -- 172.21.15.90:0/76870727 wait complete. 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 Processor -- start 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 -- start start 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd4410c8f0 0x7fbd4419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4419d100 con 0x7fbd44106b80 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.758+0000 7fbd4a200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441a0e90 con 0x7fbd4410c8f0 2024-04-24T08:36:40.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41486/0 (socket says 172.21.15.90:41486) 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 -- 172.21.15.90:0/865250646 learned_addr learned my addr 172.21.15.90:0/865250646 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 -- 172.21.15.90:0/865250646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd4410c8f0 msgr2=0x7fbd4419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd4410c8f0 0x7fbd4419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 -- 172.21.15.90:0/865250646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300095f0 con 0x7fbd44106b80 2024-04-24T08:36:40.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd43400700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ba50 tx=0x7fbd3400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.759+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3400ca00 con 0x7fbd44106b80 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.760+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd3400cb60 con 0x7fbd44106b80 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.760+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd340075a0 con 0x7fbd44106b80 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.760+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd441a1170 con 0x7fbd44106b80 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.760+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd441a16c0 con 0x7fbd44106b80 2024-04-24T08:36:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.761+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd44109b20 con 0x7fbd44106b80 2024-04-24T08:36:40.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.762+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbd3400ccd0 con 0x7fbd44106b80 2024-04-24T08:36:40.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.763+0000 7fbd3be00700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 0x7fbd28071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:40.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.763+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd34006e40 con 0x7fbd44106b80 2024-04-24T08:36:40.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.763+0000 7fbd42a00700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 0x7fbd28071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:40.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.764+0000 7fbd42a00700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 0x7fbd28071170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000b5c0 tx=0x7fbd30005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:40.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.768+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbd340573f0 con 0x7fbd44106b80 2024-04-24T08:36:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:40.998+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd441a1a50 con 0x7fbd2806ecc0 2024-04-24T08:36:41.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.001+0000 7fbd3be00700 1 -- 172.21.15.90:0/865250646 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbd441a1a50 con 0x7fbd2806ecc0 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 msgr2=0x7fbd28071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 0x7fbd28071170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000b5c0 tx=0x7fbd30005960 comp rx=0 tx=0).stop 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 msgr2=0x7fbd4419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ba50 tx=0x7fbd3400be10 comp rx=0 tx=0).stop 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 shutdown_connections 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd2806ecc0 0x7fbd28071170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd44106b80 0x7fbd4419c4e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 --2- 172.21.15.90:0/865250646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd4410c8f0 0x7fbd4419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.004+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 >> 172.21.15.90:0/865250646 conn(0x7fbd44076060 msgr2=0x7fbd44077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.005+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 shutdown_connections 2024-04-24T08:36:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.005+0000 7fbd4a200700 1 -- 172.21.15.90:0/865250646 wait complete. 2024-04-24T08:36:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 -- 172.21.15.90:0/1188929264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c1004e0 msgr2=0x7f060c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 --2- 172.21.15.90:0/1188929264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c1004e0 0x7f060c1008b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009870 tx=0x7f05fc009b80 comp rx=0 tx=0).stop 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 -- 172.21.15.90:0/1188929264 shutdown_connections 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 --2- 172.21.15.90:0/1188929264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c100e80 0x7f060c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 --2- 172.21.15.90:0/1188929264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c1004e0 0x7f060c1008b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.163+0000 7f0611400700 1 -- 172.21.15.90:0/1188929264 >> 172.21.15.90:0/1188929264 conn(0x7f060c0fbda0 msgr2=0x7f060c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.164+0000 7f0611400700 1 -- 172.21.15.90:0/1188929264 shutdown_connections 2024-04-24T08:36:41.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.164+0000 7f0611400700 1 -- 172.21.15.90:0/1188929264 wait complete. 2024-04-24T08:36:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 Processor -- start 2024-04-24T08:36:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 -- start start 2024-04-24T08:36:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 0x7f060c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c198cc0 con 0x7f060c1004e0 2024-04-24T08:36:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.165+0000 7f0611400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c19ca50 con 0x7f060c100e80 2024-04-24T08:36:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41504/0 (socket says 172.21.15.90:41504) 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 -- 172.21.15.90:0/2700984435 learned_addr learned my addr 172.21.15.90:0/2700984435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f0609e00700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 0x7f060c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 -- 172.21.15.90:0/2700984435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 msgr2=0x7f060c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 0x7f060c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f060a800700 1 -- 172.21.15.90:0/2700984435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80095f0 con 0x7f060c1004e0 2024-04-24T08:36:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.166+0000 7f0609e00700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 0x7f060c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.167+0000 7f060a800700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009230 tx=0x7f05fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.167+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc01c070 con 0x7f060c1004e0 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.167+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05fc00fcd0 con 0x7f060c1004e0 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.167+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc022dd0 con 0x7f060c1004e0 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.167+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc009520 con 0x7f060c1004e0 2024-04-24T08:36:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.168+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c19cfe0 con 0x7f060c1004e0 2024-04-24T08:36:41.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.169+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c1045d0 con 0x7f060c1004e0 2024-04-24T08:36:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.170+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f05fc020070 con 0x7f060c1004e0 2024-04-24T08:36:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.171+0000 7f0603400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 0x7f05f0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.171+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f05fc090710 con 0x7f060c1004e0 2024-04-24T08:36:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.172+0000 7f0609e00700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 0x7f05f0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.172+0000 7f0609e00700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 0x7f05f0071170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f060c1996c0 tx=0x7f05f8009310 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:41.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.179+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f05fc059640 con 0x7f060c1004e0 2024-04-24T08:36:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:41 smithi090 ceph-mon[85282]: reconnect by client.14534 192.168.0.1:0/2302411015 after 0.00199997 2024-04-24T08:36:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:41 smithi090 ceph-mon[85282]: reconnect by client.24275 192.168.0.1:0/4244548942 after 0.00199997 2024-04-24T08:36:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:41 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:reconnect 2024-04-24T08:36:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:41 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:reconnect} 3 up:standby 2024-04-24T08:36:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.414+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f060c19d310 con 0x7f05f006ecc0 2024-04-24T08:36:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.417+0000 7f0603400700 1 -- 172.21.15.90:0/2700984435 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f060c19d310 con 0x7f05f006ecc0 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.419+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 msgr2=0x7f05f0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.419+0000 7f0611400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 0x7f05f0071170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f060c1996c0 tx=0x7f05f8009310 comp rx=0 tx=0).stop 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.419+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 msgr2=0x7f060c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.419+0000 7f0611400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009230 tx=0x7f05fc0047b0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.419+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 shutdown_connections 2024-04-24T08:36:41.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f05f006ecc0 0x7f05f0071170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c1004e0 0x7f060c1980a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 --2- 172.21.15.90:0/2700984435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c100e80 0x7f060c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 >> 172.21.15.90:0/2700984435 conn(0x7f060c0fbda0 msgr2=0x7f060c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 shutdown_connections 2024-04-24T08:36:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.420+0000 7f0611400700 1 -- 172.21.15.90:0/2700984435 wait complete. 2024-04-24T08:36:41.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:41 smithi149 ceph-mon[66490]: reconnect by client.14534 192.168.0.1:0/2302411015 after 0.00199997 2024-04-24T08:36:41.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:41 smithi149 ceph-mon[66490]: reconnect by client.24275 192.168.0.1:0/4244548942 after 0.00199997 2024-04-24T08:36:41.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:41 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:reconnect 2024-04-24T08:36:41.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:41 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:reconnect} 3 up:standby 2024-04-24T08:36:41.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.576+0000 7fdc52a00700 1 -- 172.21.15.90:0/3511082821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 msgr2=0x7fdc4c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.576+0000 7fdc52a00700 1 --2- 172.21.15.90:0/3511082821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c108840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009910 tx=0x7fdc3c009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 -- 172.21.15.90:0/3511082821 shutdown_connections 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 --2- 172.21.15.90:0/3511082821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 --2- 172.21.15.90:0/3511082821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c108840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 -- 172.21.15.90:0/3511082821 >> 172.21.15.90:0/3511082821 conn(0x7fdc4c0fdf90 msgr2=0x7fdc4c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 -- 172.21.15.90:0/3511082821 shutdown_connections 2024-04-24T08:36:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.577+0000 7fdc52a00700 1 -- 172.21.15.90:0/3511082821 wait complete. 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 Processor -- start 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 -- start start 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c198e10 con 0x7fdc4c108470 2024-04-24T08:36:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.578+0000 7fdc52a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19cba0 con 0x7fdc4c102470 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54256/0 (socket says 172.21.15.90:54256) 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4be00700 1 -- 172.21.15.90:0/1988385009 learned_addr learned my addr 172.21.15.90:0/1988385009 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4b400700 1 -- 172.21.15.90:0/1988385009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 msgr2=0x7fdc4c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4b400700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.579+0000 7fdc4b400700 1 -- 172.21.15.90:0/1988385009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0095f0 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc4b400700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c198730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400d8d0 tx=0x7fdc3400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3400f860 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc3400fe80 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c19ce80 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3400e5e0 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.580+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c19d3d0 con 0x7fdc4c108470 2024-04-24T08:36:41.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.581+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c10a9e0 con 0x7fdc4c108470 2024-04-24T08:36:41.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.582+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdc34010460 con 0x7fdc4c108470 2024-04-24T08:36:41.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.583+0000 7fdc48c00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 0x7fdc30071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:41.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.583+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc3408eb60 con 0x7fdc4c108470 2024-04-24T08:36:41.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.583+0000 7fdc4be00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 0x7fdc30071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:41.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.584+0000 7fdc4be00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 0x7fdc30071170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c006010 tx=0x7fdc3c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:41.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.588+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdc34057820 con 0x7fdc4c108470 2024-04-24T08:36:41.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.843+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc4c1995c0 con 0x7fdc3006ecc0 2024-04-24T08:36:41.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.857+0000 7fdc48c00700 1 -- 172.21.15.90:0/1988385009 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdc4c1995c0 con 0x7fdc3006ecc0 2024-04-24T08:36:41.857 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:36:41.857 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 56s ago 23m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:36:41.857 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (24m) 56s ago 24m 9466k - 18.2.0 dc2bc1663786 a4ddf5319ed3 2024-04-24T08:36:41.857 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (22m) 25s ago 22m 9248k - 18.2.0 dc2bc1663786 f1a6ad92ce74 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (5m) 56s ago 23m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (5m) 25s ago 22m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (9m) 56s ago 23m 73.8M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (64s) 56s ago 16m 10.9M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 247c152849b1 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (91s) 56s ago 16m 12.0M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (31s) 25s ago 16m 11.0M - 19.0.0-3116-g1f81fb39 b98aea5aa06f c05a5e07bc14 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 starting - - - - 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (12m) 56s ago 25m 511M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (11m) 25s ago 22m 453M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (7m) 56s ago 25m 56.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (7m) 25s ago 22m 44.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (10m) 56s ago 23m 8040k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:36:41.858 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (10m) 25s ago 22m 8010k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (5m) 56s ago 21m 93.4M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (4m) 56s ago 20m 87.5M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (3m) 56s ago 20m 76.2M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (3m) 25s ago 19m 99.5M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (2m) 25s ago 18m 94.9M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (118s) 25s ago 18m 86.8M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:36:41.859 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (10m) 56s ago 23m 76.0M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.859+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 msgr2=0x7fdc30071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.859+0000 7fdc52a00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 0x7fdc30071170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c006010 tx=0x7fdc3c01a040 comp rx=0 tx=0).stop 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.859+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 msgr2=0x7fdc4c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.859+0000 7fdc52a00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c198730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400d8d0 tx=0x7fdc3400dbe0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 shutdown_connections 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdc3006ecc0 0x7fdc30071170 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc4c102470 0x7fdc4c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 --2- 172.21.15.90:0/1988385009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc4c108470 0x7fdc4c198730 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:41.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 >> 172.21.15.90:0/1988385009 conn(0x7fdc4c0fdf90 msgr2=0x7fdc4c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:41.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 shutdown_connections 2024-04-24T08:36:41.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:41.860+0000 7fdc52a00700 1 -- 172.21.15.90:0/1988385009 wait complete. 2024-04-24T08:36:42.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.010+0000 7f065c200700 1 -- 172.21.15.90:0/1010853191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0654073090 msgr2=0x7f0654073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.010+0000 7f065c200700 1 --2- 172.21.15.90:0/1010853191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0654073090 0x7f0654073460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0648009940 tx=0x7f0648009c50 comp rx=0 tx=0).stop 2024-04-24T08:36:42.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 -- 172.21.15.90:0/1010853191 shutdown_connections 2024-04-24T08:36:42.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 --2- 172.21.15.90:0/1010853191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06540739a0 0x7f065410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 --2- 172.21.15.90:0/1010853191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0654073090 0x7f0654073460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 -- 172.21.15.90:0/1010853191 >> 172.21.15.90:0/1010853191 conn(0x7f06540fbda0 msgr2=0x7f06540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:42.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 -- 172.21.15.90:0/1010853191 shutdown_connections 2024-04-24T08:36:42.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.011+0000 7f065c200700 1 -- 172.21.15.90:0/1010853191 wait complete. 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 Processor -- start 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 -- start start 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0654073090 0x7f0654198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0654198da0 con 0x7f06540739a0 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.012+0000 7f065c200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065419cb30 con 0x7f0654073090 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41546/0 (socket says 172.21.15.90:41546) 2024-04-24T08:36:42.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 -- 172.21.15.90:0/1596337626 learned_addr learned my addr 172.21.15.90:0/1596337626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:42.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0659800700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0654073090 0x7f0654198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 -- 172.21.15.90:0/1596337626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0654073090 msgr2=0x7f0654198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0654073090 0x7f0654198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 -- 172.21.15.90:0/1596337626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480095f0 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.013+0000 7f0658e00700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f064000d8d0 tx=0x7f064000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.014+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06400098a0 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.014+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0640010460 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.014+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064000f5f0 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.014+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065419ce10 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.014+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065419d360 con 0x7f06540739a0 2024-04-24T08:36:42.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.015+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0654109ce0 con 0x7f06540739a0 2024-04-24T08:36:42.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.016+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0640009a00 con 0x7f06540739a0 2024-04-24T08:36:42.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.017+0000 7f0652000700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 0x7f063c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.017+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f064008ed20 con 0x7f06540739a0 2024-04-24T08:36:42.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.017+0000 7f0659800700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 0x7f063c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.018+0000 7f0659800700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 0x7f063c071170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0648006010 tx=0x7f0648005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.022+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0640057a60 con 0x7f06540739a0 2024-04-24T08:36:42.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.340+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0654199580 con 0x7f06540739a0 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.341+0000 7f0652000700 1 -- 172.21.15.90:0/1596337626 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f064005b120 con 0x7f06540739a0 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 14 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:36:42.343 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 msgr2=0x7f063c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 0x7f063c071170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0648006010 tx=0x7f0648005a90 comp rx=0 tx=0).stop 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 msgr2=0x7f06541986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f064000d8d0 tx=0x7f064000dbe0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 shutdown_connections 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f063c06ecc0 0x7f063c071170 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0654073090 0x7f0654198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 --2- 172.21.15.90:0/1596337626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06540739a0 0x7f06541986c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.344+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 >> 172.21.15.90:0/1596337626 conn(0x7f06540fbda0 msgr2=0x7f0654106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:42.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.345+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 shutdown_connections 2024-04-24T08:36:42.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.345+0000 7f065c200700 1 -- 172.21.15.90:0/1596337626 wait complete. 2024-04-24T08:36:42.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:42 smithi149 ceph-mon[66490]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:42.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:42 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:rejoin 2024-04-24T08:36:42.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:42 smithi149 ceph-mon[66490]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:rejoin} 3 up:standby 2024-04-24T08:36:42.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:42 smithi149 ceph-mon[66490]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-24T08:36:42.463 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:42 smithi149 ceph-mon[66490]: daemon mds.cephfs.smithi090.qbyvpv is now active in filesystem cephfs as rank 0 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.499+0000 7fd0ade00700 1 -- 172.21.15.90:0/3342422475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a8106b80 msgr2=0x7fd0a8106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.499+0000 7fd0ade00700 1 --2- 172.21.15.90:0/3342422475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a8106b80 0x7fd0a8106ff0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd094009910 tx=0x7fd094009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 -- 172.21.15.90:0/3342422475 shutdown_connections 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 --2- 172.21.15.90:0/3342422475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a8106b80 0x7fd0a8106ff0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 --2- 172.21.15.90:0/3342422475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a810c8f0 0x7fd0a810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 -- 172.21.15.90:0/3342422475 >> 172.21.15.90:0/3342422475 conn(0x7fd0a8076060 msgr2=0x7fd0a8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:42.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:42 smithi090 ceph-mon[85282]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:42.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:42 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:rejoin 2024-04-24T08:36:42.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:42 smithi090 ceph-mon[85282]: fsmap cephfs:1/1 {0=cephfs.smithi090.qbyvpv=up:rejoin} 3 up:standby 2024-04-24T08:36:42.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:42 smithi090 ceph-mon[85282]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-24T08:36:42.501 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:42 smithi090 ceph-mon[85282]: daemon mds.cephfs.smithi090.qbyvpv is now active in filesystem cephfs as rank 0 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 -- 172.21.15.90:0/3342422475 shutdown_connections 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.500+0000 7fd0ade00700 1 -- 172.21.15.90:0/3342422475 wait complete. 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.501+0000 7fd0ade00700 1 Processor -- start 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.502+0000 7fd0ade00700 1 -- start start 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.502+0000 7fd0ade00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8106b80 0x7fd0a819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.502+0000 7fd0ade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.502+0000 7fd0ade00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a819cfd0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.502+0000 7fd0ade00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a819d110 con 0x7fd0a8106b80 2024-04-24T08:36:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41566/0 (socket says 172.21.15.90:41566) 2024-04-24T08:36:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 -- 172.21.15.90:0/439290903 learned_addr learned my addr 172.21.15.90:0/439290903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6a00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8106b80 0x7fd0a819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 -- 172.21.15.90:0/439290903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8106b80 msgr2=0x7fd0a819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8106b80 0x7fd0a819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.503+0000 7fd0a6000700 1 -- 172.21.15.90:0/439290903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940095f0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0a6000700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd09400b5c0 tx=0x7fd09400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09401d070 con 0x7fd0a810c8f0 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd09400f460 con 0x7fd0a810c8f0 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd094017630 con 0x7fd0a810c8f0 2024-04-24T08:36:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a81a0f00 con 0x7fd0a810c8f0 2024-04-24T08:36:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.504+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a81a13f0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.505+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a810ecc0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.507+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd09400f5d0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.508+0000 7fd0aca00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 0x7fd08c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.508+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd0940903b0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.508+0000 7fd0a6a00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 0x7fd08c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.509+0000 7fd0a6a00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 0x7fd08c071170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ac50 tx=0x7fd09800a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.514+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd0940590f0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.787+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0a804f2a0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.788+0000 7fd0aca00700 1 -- 172.21.15.90:0/439290903 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1939 (secure 0 0 0) 0x7fd09405c7b0 con 0x7fd0a810c8f0 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:e34 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:36:42.789 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 34 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:36:42.091184+0000 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 86 2024-04-24T08:36:42.790 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34462} 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34462 members: 34462 2024-04-24T08:36:42.791 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{0:34462} state up:active seq 26 join_fscid=1 addr [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{-1:34494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/4101562674,v1:172.21.15.149:6825/4101562674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:42.792 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6826/1676173850,v1:172.21.15.149:6827/1676173850] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 msgr2=0x7fd08c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 0x7fd08c071170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ac50 tx=0x7fd09800a380 comp rx=0 tx=0).stop 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 msgr2=0x7fd0a819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd09400b5c0 tx=0x7fd09400bd50 comp rx=0 tx=0).stop 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 shutdown_connections 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd08c06ecc0 0x7fd08c071170 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8106b80 0x7fd0a819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 --2- 172.21.15.90:0/439290903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0a810c8f0 0x7fd0a819c980 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 >> 172.21.15.90:0/439290903 conn(0x7fd0a8076060 msgr2=0x7fd0a8077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 shutdown_connections 2024-04-24T08:36:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.792+0000 7fd0ade00700 1 -- 172.21.15.90:0/439290903 wait complete. 2024-04-24T08:36:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 34 2024-04-24T08:36:42.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 -- 172.21.15.90:0/707862856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 msgr2=0x7fb278106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 --2- 172.21.15.90:0/707862856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb278106ff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb264009910 tx=0x7fb264009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 -- 172.21.15.90:0/707862856 shutdown_connections 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 --2- 172.21.15.90:0/707862856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb278106ff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 --2- 172.21.15.90:0/707862856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 -- 172.21.15.90:0/707862856 >> 172.21.15.90:0/707862856 conn(0x7fb278076060 msgr2=0x7fb278078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 -- 172.21.15.90:0/707862856 shutdown_connections 2024-04-24T08:36:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.940+0000 7fb27f200700 1 -- 172.21.15.90:0/707862856 wait complete. 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.941+0000 7fb27f200700 1 Processor -- start 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb27f200700 1 -- start start 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb27f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb27f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb27f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27819d100 con 0x7fb278106b80 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb27f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a0e90 con 0x7fb27810c8f0 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb277e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb277e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41592/0 (socket says 172.21.15.90:41592) 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.942+0000 7fb277e00700 1 -- 172.21.15.90:0/2241321337 learned_addr learned my addr 172.21.15.90:0/2241321337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:42.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277400700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277e00700 1 -- 172.21.15.90:0/2241321337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 msgr2=0x7fb27819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277e00700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277e00700 1 -- 172.21.15.90:0/2241321337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2640095f0 con 0x7fb278106b80 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277400700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb277e00700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb26800b4b0 tx=0x7fb26800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb268011800 con 0x7fb278106b80 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb268011e20 con 0x7fb278106b80 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2781a1170 con 0x7fb278106b80 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb26800f2e0 con 0x7fb278106b80 2024-04-24T08:36:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.943+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2781a16c0 con 0x7fb278106b80 2024-04-24T08:36:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.945+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb278109b20 con 0x7fb278106b80 2024-04-24T08:36:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.946+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb268010380 con 0x7fb278106b80 2024-04-24T08:36:42.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.946+0000 7fb274c00700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 0x7fb25c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:42.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.947+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb26808e9e0 con 0x7fb278106b80 2024-04-24T08:36:42.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.947+0000 7fb277400700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 0x7fb25c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:42.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.947+0000 7fb277400700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 0x7fb25c071170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb264006010 tx=0x7fb26400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:42.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:42.951+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb268057870 con 0x7fb278106b80 2024-04-24T08:36:43.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.191+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2781a1990 con 0x7fb25c06ecc0 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.193+0000 7fb274c00700 1 -- 172.21.15.90:0/2241321337 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb2781a1990 con 0x7fb25c06ecc0 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:36:43.194 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "15/23 daemons upgraded", 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:36:43.195 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:36:43.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 msgr2=0x7fb25c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 0x7fb25c071170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb264006010 tx=0x7fb26400b540 comp rx=0 tx=0).stop 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 msgr2=0x7fb27819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb26800b4b0 tx=0x7fb26800b870 comp rx=0 tx=0).stop 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 shutdown_connections 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb25c06ecc0 0x7fb25c071170 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb278106b80 0x7fb27819c4e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 --2- 172.21.15.90:0/2241321337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb27810c8f0 0x7fb27819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 >> 172.21.15.90:0/2241321337 conn(0x7fb278076060 msgr2=0x7fb278077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 shutdown_connections 2024-04-24T08:36:43.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.197+0000 7fb27f200700 1 -- 172.21.15.90:0/2241321337 wait complete. 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 -- 172.21.15.90:0/1547907220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 msgr2=0x7f9f08104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 --2- 172.21.15.90:0/1547907220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08104e50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009910 tx=0x7f9ef4009c20 comp rx=0 tx=0).stop 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 -- 172.21.15.90:0/1547907220 shutdown_connections 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 --2- 172.21.15.90:0/1547907220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08104e50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 --2- 172.21.15.90:0/1547907220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f081008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.356+0000 7f9f0e800700 1 -- 172.21.15.90:0/1547907220 >> 172.21.15.90:0/1547907220 conn(0x7f9f080fbda0 msgr2=0x7f9f080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:43.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:active 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1596337626' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:43.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:43 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/439290903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:36:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.357+0000 7f9f0e800700 1 -- 172.21.15.90:0/1547907220 shutdown_connections 2024-04-24T08:36:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.357+0000 7f9f0e800700 1 -- 172.21.15.90:0/1547907220 wait complete. 2024-04-24T08:36:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 Processor -- start 2024-04-24T08:36:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 -- start start 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f08198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08198d30 con 0x7f9f08100e80 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.358+0000 7f9f0e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0819cac0 con 0x7f9f081004e0 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41622/0 (socket says 172.21.15.90:41622) 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 -- 172.21.15.90:0/2496420209 learned_addr learned my addr 172.21.15.90:0/2496420209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f07400700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f08198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 -- 172.21.15.90:0/2496420209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 msgr2=0x7f9f08198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f08198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f06a00700 1 -- 172.21.15.90:0/2496420209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40095f0 con 0x7f9f08100e80 2024-04-24T08:36:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.359+0000 7f9f07400700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f08198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:36:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9f06a00700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400bda0 tx=0x7f9ef400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef401d070 con 0x7f9f08100e80 2024-04-24T08:36:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ef400f460 con 0x7f9f08100e80 2024-04-24T08:36:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef4017610 con 0x7f9f08100e80 2024-04-24T08:36:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0819cd40 con 0x7f9f08100e80 2024-04-24T08:36:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.360+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0819d230 con 0x7f9f08100e80 2024-04-24T08:36:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.361+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f081045d0 con 0x7f9f08100e80 2024-04-24T08:36:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.362+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9ef400fa40 con 0x7f9f08100e80 2024-04-24T08:36:43.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.363+0000 7f9effe00700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 0x7f9eec071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:36:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.363+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9ef4090330 con 0x7f9f08100e80 2024-04-24T08:36:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.364+0000 7f9f07400700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 0x7f9eec071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:36:43.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.365+0000 7f9f07400700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 0x7f9eec071170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005fd0 tx=0x7f9ef8005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:36:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.368+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9ef4059070 con 0x7f9f08100e80 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: mds.? [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] up:active 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1596337626' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:43.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:43 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/439290903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:36:43.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.686+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9f08199510 con 0x7f9f08100e80 2024-04-24T08:36:43.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.687+0000 7f9effe00700 1 -- 172.21.15.90:0/2496420209 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9ef405c730 con 0x7f9f08100e80 2024-04-24T08:36:43.688 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:36:43.688 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:36:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:36:43.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 msgr2=0x7f9eec071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 0x7f9eec071170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005fd0 tx=0x7f9ef8005ee0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 msgr2=0x7f9f08198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400bda0 tx=0x7f9ef400be80 comp rx=0 tx=0).stop 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 shutdown_connections 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eec06ecc0 0x7f9eec071170 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f081004e0 0x7f9f08198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 --2- 172.21.15.90:0/2496420209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f08100e80 0x7f9f08198650 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 >> 172.21.15.90:0/2496420209 conn(0x7f9f080fbda0 msgr2=0x7f9f080fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.690+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 shutdown_connections 2024-04-24T08:36:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:36:43.691+0000 7f9f0e800700 1 -- 172.21.15.90:0/2496420209 wait complete. 2024-04-24T08:36:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:44 smithi149 ceph-mon[66490]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:44 smithi149 ceph-mon[66490]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T08:36:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:44 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2496420209' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:36:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:44.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:44 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:44.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:44 smithi090 ceph-mon[85282]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:36:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:44 smithi090 ceph-mon[85282]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T08:36:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:44 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2496420209' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:36:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:44 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:36:46.520 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:46 smithi149 ceph-mon[66490]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T08:36:46.520 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:46.520 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:46 smithi090 ceph-mon[85282]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T08:36:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:48.507 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:48 smithi149 ceph-mon[66490]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T08:36:48.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:48 smithi090 ceph-mon[85282]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T08:36:50.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:50 smithi149 ceph-mon[66490]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-24T08:36:50.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:50 smithi090 ceph-mon[85282]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-24T08:36:51.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:51 smithi149 ceph-mon[66490]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-24T08:36:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:51 smithi090 ceph-mon[85282]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all mds 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.lumztb"}]': finished 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.qbyvpv"}]': finished 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:53.028 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.eazcmc"}]': finished 2024-04-24T08:36:53.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:53.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.zwjtmw"}]': finished 2024-04-24T08:36:53.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T08:36:53.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:52 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:36:53.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all mds 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.lumztb"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.lumztb"}]': finished 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.qbyvpv"}]: dispatch 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.qbyvpv"}]': finished 2024-04-24T08:36:53.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.eazcmc"}]: dispatch 2024-04-24T08:36:53.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.eazcmc"}]': finished 2024-04-24T08:36:53.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.zwjtmw"}]: dispatch 2024-04-24T08:36:53.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.zwjtmw"}]': finished 2024-04-24T08:36:53.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T08:36:53.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:52 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T08:36:54.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T08:36:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:54 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:54 smithi149 ceph-mon[66490]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-24T08:36:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:54 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-24T08:36:54.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:54 smithi149 ceph-mon[66490]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-24T08:36:54.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T08:36:54.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:54 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:54.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:54 smithi090 ceph-mon[85282]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-24T08:36:54.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:54 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-24T08:36:54.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:54 smithi090 ceph-mon[85282]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-24T08:36:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-24T08:36:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:36:55.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all rgw 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.294 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:55 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 3 up:standby 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all rgw 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:55.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:55 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:36:56.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:56 smithi090 ceph-mon[85282]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-24T08:36:56.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:56 smithi149 ceph-mon[66490]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-24T08:36:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: Upgrade: Updating ceph-exporter.smithi090 (1/2) 2024-04-24T08:36:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:36:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:36:57.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:36:57 smithi090 ceph-mon[85282]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: Upgrade: Updating ceph-exporter.smithi090 (1/2) 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-24T08:36:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:36:57 smithi149 ceph-mon[66490]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-24T08:37:00.521 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:00 smithi090 ceph-mon[85282]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-24T08:37:00.522 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:00 smithi149 ceph-mon[66490]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-24T08:37:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:02.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:02 smithi149 ceph-mon[66490]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-04-24T08:37:02.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:02 smithi090 ceph-mon[85282]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-04-24T08:37:04.010 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:04.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:03 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:04.011 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:03 smithi149 ceph-mon[66490]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-04-24T08:37:04.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:04.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:03 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:04.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:03 smithi090 ceph-mon[85282]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-04-24T08:37:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: Upgrade: Updating ceph-exporter.smithi149 (2/2) 2024-04-24T08:37:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:37:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:06.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: Deploying daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:37:06.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:06 smithi149 ceph-mon[66490]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: Upgrade: Updating ceph-exporter.smithi149 (2/2) 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi149", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: Deploying daemon ceph-exporter.smithi149 on smithi149 2024-04-24T08:37:06.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:06 smithi090 ceph-mon[85282]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-04-24T08:37:08.526 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:08 smithi090 ceph-mon[85282]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-24T08:37:08.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:08 smithi149 ceph-mon[66490]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-24T08:37:09.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:09 smithi090 ceph-mon[85282]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T08:37:09.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:09 smithi149 ceph-mon[66490]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T08:37:12.240 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:12.240 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:12.240 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:11 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:12.240 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:11 smithi090 ceph-mon[85282]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-04-24T08:37:12.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:12.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:12.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:11 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:12.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:11 smithi149 ceph-mon[66490]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-04-24T08:37:13.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 -- 172.21.15.90:0/3257521868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ff3c0 msgr2=0x7fed9c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:13.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 --2- 172.21.15.90:0/3257521868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c0ff790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009910 tx=0x7fed8c009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:13.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 -- 172.21.15.90:0/3257521868 shutdown_connections 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 --2- 172.21.15.90:0/3257521868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ffd60 0x7fed9c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 --2- 172.21.15.90:0/3257521868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c0ff790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 -- 172.21.15.90:0/3257521868 >> 172.21.15.90:0/3257521868 conn(0x7fed9c0fac60 msgr2=0x7fed9c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 -- 172.21.15.90:0/3257521868 shutdown_connections 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.864+0000 7feda1800700 1 -- 172.21.15.90:0/3257521868 wait complete. 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.865+0000 7feda1800700 1 Processor -- start 2024-04-24T08:37:13.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.865+0000 7feda1800700 1 -- start start 2024-04-24T08:37:13.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7feda1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:13.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7feda1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:13.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7feda1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c198c80 con 0x7fed9c0ffd60 2024-04-24T08:37:13.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7feda1800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c19ca10 con 0x7fed9c0ff3c0 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7fed9a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7fed9a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45208/0 (socket says 172.21.15.90:45208) 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7fed9a000700 1 -- 172.21.15.90:0/774577022 learned_addr learned my addr 172.21.15.90:0/774577022 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.866+0000 7fed9aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed9a000700 1 -- 172.21.15.90:0/774577022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ff3c0 msgr2=0x7fed9c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:13.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed9a000700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed9a000700 1 -- 172.21.15.90:0/774577022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0095f0 con 0x7fed9c0ffd60 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed9a000700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e8f0 tx=0x7fed8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed88009790 con 0x7fed9c0ffd60 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed8800c810 con 0x7fed9c0ffd60 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed88005580 con 0x7fed9c0ffd60 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.867+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c19ccf0 con 0x7fed9c0ffd60 2024-04-24T08:37:13.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.868+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c19d240 con 0x7fed9c0ffd60 2024-04-24T08:37:13.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.869+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c04f2a0 con 0x7fed9c0ffd60 2024-04-24T08:37:13.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.870+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fed8800c980 con 0x7fed9c0ffd60 2024-04-24T08:37:13.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.871+0000 7fed93400700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 0x7fed80071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:13.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.871+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed88014070 con 0x7fed9c0ffd60 2024-04-24T08:37:13.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.871+0000 7fed9aa00700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 0x7fed80071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:13.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.872+0000 7fed9aa00700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 0x7fed80071170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009fd0 tx=0x7fed8c005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:13.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:13.876+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fed88057e70 con 0x7fed9c0ffd60 2024-04-24T08:37:14.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.144+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed9c19d620 con 0x7fed8006ecc0 2024-04-24T08:37:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.146+0000 7fed93400700 1 -- 172.21.15.90:0/774577022 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fed9c19d620 con 0x7fed8006ecc0 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 msgr2=0x7fed80071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 0x7fed80071170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009fd0 tx=0x7fed8c005a90 comp rx=0 tx=0).stop 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 msgr2=0x7fed9c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e8f0 tx=0x7fed8800ec00 comp rx=0 tx=0).stop 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 shutdown_connections 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fed8006ecc0 0x7fed80071170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed9c0ff3c0 0x7fed9c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 --2- 172.21.15.90:0/774577022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c0ffd60 0x7fed9c1985a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.150+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 >> 172.21.15.90:0/774577022 conn(0x7fed9c0fac60 msgr2=0x7fed9c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:14.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.151+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 shutdown_connections 2024-04-24T08:37:14.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.151+0000 7feda1800700 1 -- 172.21.15.90:0/774577022 wait complete. 2024-04-24T08:37:14.168 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:37:14.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 -- 172.21.15.90:0/1257989164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 msgr2=0x7f3ae810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 --2- 172.21.15.90:0/1257989164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae810c5e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009940 tx=0x7f3ad4009c50 comp rx=0 tx=0).stop 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 -- 172.21.15.90:0/1257989164 shutdown_connections 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 --2- 172.21.15.90:0/1257989164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae810c5e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 --2- 172.21.15.90:0/1257989164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae8073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 -- 172.21.15.90:0/1257989164 >> 172.21.15.90:0/1257989164 conn(0x7f3ae80fbda0 msgr2=0x7f3ae80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 -- 172.21.15.90:0/1257989164 shutdown_connections 2024-04-24T08:37:14.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.312+0000 7f3aef600700 1 -- 172.21.15.90:0/1257989164 wait complete. 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.313+0000 7f3aef600700 1 Processor -- start 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.313+0000 7f3aef600700 1 -- start start 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3aef600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3aef600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3aef600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8198dd0 con 0x7f3ae80739a0 2024-04-24T08:37:14.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3aef600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae819cb60 con 0x7f3ae8073090 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3aecc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45212/0 (socket says 172.21.15.90:45212) 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.314+0000 7f3ae7e00700 1 -- 172.21.15.90:0/1726692568 learned_addr learned my addr 172.21.15.90:0/1726692568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae7e00700 1 -- 172.21.15.90:0/1726692568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 msgr2=0x7f3ae81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae7e00700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae7e00700 1 -- 172.21.15.90:0/1726692568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40095f0 con 0x7f3ae80739a0 2024-04-24T08:37:14.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3aecc00700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae7e00700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000c00 tx=0x7f3ad400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad401d070 con 0x7f3ae80739a0 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ad4003d10 con 0x7f3ae80739a0 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad400fe90 con 0x7f3ae80739a0 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.315+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae819cde0 con 0x7f3ae80739a0 2024-04-24T08:37:14.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.316+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae819d220 con 0x7f3ae80739a0 2024-04-24T08:37:14.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.317+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae8109ce0 con 0x7f3ae80739a0 2024-04-24T08:37:14.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.318+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ad400f6f0 con 0x7f3ae80739a0 2024-04-24T08:37:14.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.319+0000 7f3ae5600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 0x7f3ad0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.319+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3ad40900f0 con 0x7f3ae80739a0 2024-04-24T08:37:14.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.319+0000 7f3aecc00700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 0x7f3ad0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.320+0000 7f3aecc00700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 0x7f3ad0071170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3adc011490 tx=0x7f3adc009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:14.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.325+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ad4058e30 con 0x7f3ae80739a0 2024-04-24T08:37:14.419 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:14 smithi090 ceph-mon[85282]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 1 op/s 2024-04-24T08:37:14.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:14 smithi149 ceph-mon[66490]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 1 op/s 2024-04-24T08:37:14.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.591+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ae819d4f0 con 0x7f3ad006ecc0 2024-04-24T08:37:14.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.594+0000 7f3ae5600700 1 -- 172.21.15.90:0/1726692568 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f3ae819d4f0 con 0x7f3ad006ecc0 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 msgr2=0x7f3ad0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 0x7f3ad0071170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3adc011490 tx=0x7f3adc009260 comp rx=0 tx=0).stop 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 msgr2=0x7f3ae81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000c00 tx=0x7f3ad400bb10 comp rx=0 tx=0).stop 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 shutdown_connections 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3ad006ecc0 0x7f3ad0071170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8073090 0x7f3ae81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 --2- 172.21.15.90:0/1726692568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae80739a0 0x7f3ae81986f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.597+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 >> 172.21.15.90:0/1726692568 conn(0x7f3ae80fbda0 msgr2=0x7f3ae8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.598+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 shutdown_connections 2024-04-24T08:37:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.598+0000 7f3aef600700 1 -- 172.21.15.90:0/1726692568 wait complete. 2024-04-24T08:37:14.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 -- 172.21.15.90:0/1509231273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 msgr2=0x7fb9140689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 --2- 172.21.15.90:0/1509231273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb9140689b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb900009910 tx=0x7fb900009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 -- 172.21.15.90:0/1509231273 shutdown_connections 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 --2- 172.21.15.90:0/1509231273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb9140689b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 --2- 172.21.15.90:0/1509231273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 0x7fb914106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 -- 172.21.15.90:0/1509231273 >> 172.21.15.90:0/1509231273 conn(0x7fb914074fa0 msgr2=0x7fb9140753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.762+0000 7fb918e00700 1 -- 172.21.15.90:0/1509231273 shutdown_connections 2024-04-24T08:37:14.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.763+0000 7fb918e00700 1 -- 172.21.15.90:0/1509231273 wait complete. 2024-04-24T08:37:14.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.764+0000 7fb918e00700 1 Processor -- start 2024-04-24T08:37:14.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.764+0000 7fb918e00700 1 -- start start 2024-04-24T08:37:14.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb918e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb918e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 0x7fb91419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb918e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91419c520 con 0x7fb914068540 2024-04-24T08:37:14.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb918e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141959d0 con 0x7fb9141061b0 2024-04-24T08:37:14.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb913400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb913400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45228/0 (socket says 172.21.15.90:45228) 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb913400700 1 -- 172.21.15.90:0/1016876587 learned_addr learned my addr 172.21.15.90:0/1016876587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.765+0000 7fb912a00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 0x7fb91419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.766+0000 7fb913400700 1 -- 172.21.15.90:0/1016876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 msgr2=0x7fb91419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.766+0000 7fb913400700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 0x7fb91419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.766+0000 7fb913400700 1 -- 172.21.15.90:0/1016876587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000095f0 con 0x7fb914068540 2024-04-24T08:37:14.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.766+0000 7fb913400700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb90400e920 tx=0x7fb90400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:14.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.766+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90400c9d0 con 0x7fb914068540 2024-04-24T08:37:14.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.767+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb904010430 con 0x7fb914068540 2024-04-24T08:37:14.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.767+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb914195cb0 con 0x7fb914068540 2024-04-24T08:37:14.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.767+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb914196280 con 0x7fb914068540 2024-04-24T08:37:14.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.767+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb904018630 con 0x7fb914068540 2024-04-24T08:37:14.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.768+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4005320 con 0x7fb914068540 2024-04-24T08:37:14.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.769+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb90400cb30 con 0x7fb914068540 2024-04-24T08:37:14.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.771+0000 7fb90be00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 0x7fb8f8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:14.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.771+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb904014070 con 0x7fb914068540 2024-04-24T08:37:14.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.771+0000 7fb912a00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 0x7fb8f8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:14.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.772+0000 7fb912a00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 0x7fb8f8071170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb91410abe0 tx=0x7fb900005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:14.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:14.775+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb904053ff0 con 0x7fb914068540 2024-04-24T08:37:15.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.026+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8f4000bf0 con 0x7fb8f806ecc0 2024-04-24T08:37:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.059+0000 7fb90be00700 1 -- 172.21.15.90:0/1016876587 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb8f4000bf0 con 0x7fb8f806ecc0 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (10m) 89s ago 24m 14.1M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 starting - - - - 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (4s) 0s ago 23m 6937k - 19.0.0-3116-g1f81fb39 b98aea5aa06f d4efef4d6133 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (6m) 89s ago 24m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (6m) 0s ago 23m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (9m) 89s ago 23m 73.8M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (97s) 89s ago 16m 10.9M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 247c152849b1 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (2m) 89s ago 17m 12.0M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (65s) 0s ago 16m 12.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f c05a5e07bc14 2024-04-24T08:37:15.061 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (36s) 0s ago 16m 12.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f e28402bceb34 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (13m) 89s ago 26m 511M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (11m) 0s ago 22m 453M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (8m) 89s ago 26m 56.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (7m) 0s ago 22m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (11m) 89s ago 24m 8040k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (11m) 0s ago 22m 7822k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (5m) 89s ago 21m 93.4M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (5m) 89s ago 21m 87.5M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (4m) 89s ago 20m 76.2M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (3m) 0s ago 20m 102M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (3m) 0s ago 19m 97.1M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (2m) 0s ago 18m 88.7M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:37:15.062 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (10m) 89s ago 23m 76.0M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 msgr2=0x7fb8f8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 0x7fb8f8071170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb91410abe0 tx=0x7fb900005a90 comp rx=0 tx=0).stop 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 msgr2=0x7fb91419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb90400e920 tx=0x7fb90400ec30 comp rx=0 tx=0).stop 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 shutdown_connections 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb8f806ecc0 0x7fb8f8071170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb914068540 0x7fb91419b950 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 --2- 172.21.15.90:0/1016876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9141061b0 0x7fb91419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.062+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 >> 172.21.15.90:0/1016876587 conn(0x7fb914074fa0 msgr2=0x7fb9140fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:15.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.063+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 shutdown_connections 2024-04-24T08:37:15.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.063+0000 7fb918e00700 1 -- 172.21.15.90:0/1016876587 wait complete. 2024-04-24T08:37:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 -- 172.21.15.90:0/926512046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90480687a0 msgr2=0x7f9048068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 --2- 172.21.15.90:0/926512046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90480687a0 0x7f9048068b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f903800b0a0 tx=0x7f903800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 -- 172.21.15.90:0/926512046 shutdown_connections 2024-04-24T08:37:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 --2- 172.21.15.90:0/926512046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9048069140 0x7f9048104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 --2- 172.21.15.90:0/926512046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90480687a0 0x7f9048068b70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 -- 172.21.15.90:0/926512046 >> 172.21.15.90:0/926512046 conn(0x7f9048075130 msgr2=0x7f9048075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:15.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 -- 172.21.15.90:0/926512046 shutdown_connections 2024-04-24T08:37:15.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.217+0000 7f904d200700 1 -- 172.21.15.90:0/926512046 wait complete. 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.218+0000 7f904d200700 1 Processor -- start 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.219+0000 7f904d200700 1 -- start start 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.219+0000 7f904d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90480687a0 0x7f90480ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.219+0000 7f904d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.219+0000 7f904d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048100720 con 0x7f9048069140 2024-04-24T08:37:15.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.219+0000 7f904d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048100860 con 0x7f90480687a0 2024-04-24T08:37:15.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.220+0000 7f903fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.220+0000 7f903fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45240/0 (socket says 172.21.15.90:45240) 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.220+0000 7f903fe00700 1 -- 172.21.15.90:0/819504486 learned_addr learned my addr 172.21.15.90:0/819504486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.220+0000 7f9045e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90480687a0 0x7f90480ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.221+0000 7f903fe00700 1 -- 172.21.15.90:0/819504486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90480687a0 msgr2=0x7f90480ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.221+0000 7f903fe00700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90480687a0 0x7f90480ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.221+0000 7f903fe00700 1 -- 172.21.15.90:0/819504486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038009d00 con 0x7f9048069140 2024-04-24T08:37:15.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.221+0000 7f903fe00700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f903000c8a0 tx=0x7f903000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:15.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.222+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90300078e0 con 0x7f9048069140 2024-04-24T08:37:15.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.222+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f903000f450 con 0x7f9048069140 2024-04-24T08:37:15.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.222+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90481a1ff0 con 0x7f9048069140 2024-04-24T08:37:15.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.222+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903000e5e0 con 0x7f9048069140 2024-04-24T08:37:15.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.222+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90481a2400 con 0x7f9048069140 2024-04-24T08:37:15.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.223+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90481087e0 con 0x7f9048069140 2024-04-24T08:37:15.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.225+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f903000f5c0 con 0x7f9048069140 2024-04-24T08:37:15.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.226+0000 7f903ea00700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 0x7f902c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.226+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f903008edb0 con 0x7f9048069140 2024-04-24T08:37:15.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.226+0000 7f9045e00700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 0x7f902c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.227+0000 7f9045e00700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 0x7f902c071170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f903800b830 tx=0x7f9038000f40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:15.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.231+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9030057af0 con 0x7f9048069140 2024-04-24T08:37:15.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:15 smithi090 ceph-mon[85282]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:15.293 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:15 smithi149 ceph-mon[66490]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:15.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:15.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.615+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f904804f2a0 con 0x7f9048069140 2024-04-24T08:37:15.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.616+0000 7f903ea00700 1 -- 172.21.15.90:0/819504486 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f903005b1b0 con 0x7f9048069140 2024-04-24T08:37:15.617 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:37:15.618 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:37:15.619 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 14 2024-04-24T08:37:15.619 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:37:15.619 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:37:15.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 msgr2=0x7f902c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 0x7f902c071170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f903800b830 tx=0x7f9038000f40 comp rx=0 tx=0).stop 2024-04-24T08:37:15.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 msgr2=0x7f90481001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f903000c8a0 tx=0x7f903000cbb0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 shutdown_connections 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f902c06ecc0 0x7f902c071170 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90480687a0 0x7f90480ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 --2- 172.21.15.90:0/819504486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9048069140 0x7f90481001e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 >> 172.21.15.90:0/819504486 conn(0x7f9048075130 msgr2=0x7f9048104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 shutdown_connections 2024-04-24T08:37:15.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.619+0000 7f904d200700 1 -- 172.21.15.90:0/819504486 wait complete. 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 -- 172.21.15.90:0/364620462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 msgr2=0x7fc8a0105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/364620462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0105600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc890009910 tx=0x7fc890009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 -- 172.21.15.90:0/364620462 shutdown_connections 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/364620462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 0x7fc8a00689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/364620462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0105600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.783+0000 7fc8a4c00700 1 -- 172.21.15.90:0/364620462 >> 172.21.15.90:0/364620462 conn(0x7fc8a0075130 msgr2=0x7fc8a0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:15.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.784+0000 7fc8a4c00700 1 -- 172.21.15.90:0/364620462 shutdown_connections 2024-04-24T08:37:15.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.784+0000 7fc8a4c00700 1 -- 172.21.15.90:0/364620462 wait complete. 2024-04-24T08:37:15.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 Processor -- start 2024-04-24T08:37:15.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 -- start start 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 0x7fc8a0195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0196a00 con 0x7fc8a0105230 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.785+0000 7fc8a4c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a019a790 con 0x7fc8a0068540 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45256/0 (socket says 172.21.15.90:45256) 2024-04-24T08:37:15.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 -- 172.21.15.90:0/541553003 learned_addr learned my addr 172.21.15.90:0/541553003 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89de00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 0x7fc8a0195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 -- 172.21.15.90:0/541553003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 msgr2=0x7fc8a0195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 0x7fc8a0195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.786+0000 7fc89d400700 1 -- 172.21.15.90:0/541553003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900095f0 con 0x7fc8a0105230 2024-04-24T08:37:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.787+0000 7fc89d400700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00d8d0 tx=0x7fc88c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:15.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.787+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc88c009940 con 0x7fc8a0105230 2024-04-24T08:37:15.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.787+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc88c010460 con 0x7fc8a0105230 2024-04-24T08:37:15.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.787+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc88c00f5f0 con 0x7fc8a0105230 2024-04-24T08:37:15.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.788+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a019aa70 con 0x7fc8a0105230 2024-04-24T08:37:15.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.788+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a019afc0 con 0x7fc8a0105230 2024-04-24T08:37:15.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.789+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a004f2a0 con 0x7fc8a0105230 2024-04-24T08:37:15.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.790+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc88c0105d0 con 0x7fc8a0105230 2024-04-24T08:37:15.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.792+0000 7fc896a00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 0x7fc884071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:15.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.792+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc88c08e850 con 0x7fc8a0105230 2024-04-24T08:37:15.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.792+0000 7fc89de00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 0x7fc884071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:15.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.793+0000 7fc89de00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 0x7fc884071170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc89001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:15.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:15.797+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc88c057630 con 0x7fc8a0105230 2024-04-24T08:37:16.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.086+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc8a0197140 con 0x7fc8a0105230 2024-04-24T08:37:16.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.087+0000 7fc896a00700 1 -- 172.21.15.90:0/541553003 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1966 (secure 0 0 0) 0x7fc88c0207c0 con 0x7fc8a0105230 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:e37 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 37 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:37:16.089 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:36:54.127064+0000 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 86 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:37:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34462} 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34462 members: 34462 2024-04-24T08:37:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{0:34462} state up:active seq 26 join_fscid=1 addr [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:34494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/4101562674,v1:172.21.15.149:6825/4101562674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6826/1676173850,v1:172.21.15.149:6827/1676173850] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:37:16.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 msgr2=0x7fc884071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 0x7fc884071170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc89001a040 comp rx=0 tx=0).stop 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 msgr2=0x7fc8a0196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00d8d0 tx=0x7fc88c00dc90 comp rx=0 tx=0).stop 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 shutdown_connections 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc88406ecc0 0x7fc884071170 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8a0068540 0x7fc8a0195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 --2- 172.21.15.90:0/541553003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8a0105230 0x7fc8a0196320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.090+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 >> 172.21.15.90:0/541553003 conn(0x7fc8a0075130 msgr2=0x7fc8a00fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.091+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 shutdown_connections 2024-04-24T08:37:16.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.091+0000 7fc8a4c00700 1 -- 172.21.15.90:0/541553003 wait complete. 2024-04-24T08:37:16.094 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.250+0000 7fd65c000700 1 -- 172.21.15.90:0/4087780764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6540687a0 msgr2=0x7fd654068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.250+0000 7fd65c000700 1 --2- 172.21.15.90:0/4087780764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6540687a0 0x7fd654068b70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd648009910 tx=0x7fd648009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.250+0000 7fd65c000700 1 -- 172.21.15.90:0/4087780764 shutdown_connections 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.251+0000 7fd65c000700 1 --2- 172.21.15.90:0/4087780764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd654069140 0x7fd654104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.251+0000 7fd65c000700 1 --2- 172.21.15.90:0/4087780764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6540687a0 0x7fd654068b70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.251+0000 7fd65c000700 1 -- 172.21.15.90:0/4087780764 >> 172.21.15.90:0/4087780764 conn(0x7fd654075130 msgr2=0x7fd654075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:16.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.251+0000 7fd65c000700 1 -- 172.21.15.90:0/4087780764 shutdown_connections 2024-04-24T08:37:16.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.251+0000 7fd65c000700 1 -- 172.21.15.90:0/4087780764 wait complete. 2024-04-24T08:37:16.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 Processor -- start 2024-04-24T08:37:16.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 -- start start 2024-04-24T08:37:16.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6540687a0 0x7fd6540ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd654100720 con 0x7fd654069140 2024-04-24T08:37:16.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.252+0000 7fd65c000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd654100860 con 0x7fd6540687a0 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45276/0 (socket says 172.21.15.90:45276) 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 -- 172.21.15.90:0/836830648 learned_addr learned my addr 172.21.15.90:0/836830648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd659600700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6540687a0 0x7fd6540ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 -- 172.21.15.90:0/836830648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6540687a0 msgr2=0x7fd6540ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6540687a0 0x7fd6540ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.253+0000 7fd658c00700 1 -- 172.21.15.90:0/836830648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480095f0 con 0x7fd654069140 2024-04-24T08:37:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd658c00700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd64000d900 tx=0x7fd64000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:16.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd640009900 con 0x7fd654069140 2024-04-24T08:37:16.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd640010460 con 0x7fd654069140 2024-04-24T08:37:16.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd64000f5d0 con 0x7fd654069140 2024-04-24T08:37:16.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6541a20d0 con 0x7fd654069140 2024-04-24T08:37:16.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.254+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6541a24e0 con 0x7fd654069140 2024-04-24T08:37:16.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.256+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654108820 con 0x7fd654069140 2024-04-24T08:37:16.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.257+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd6400105d0 con 0x7fd654069140 2024-04-24T08:37:16.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.258+0000 7fd652000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 0x7fd63c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.258+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd64008ed00 con 0x7fd654069140 2024-04-24T08:37:16.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.258+0000 7fd659600700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 0x7fd63c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.259+0000 7fd659600700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 0x7fd63c071170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd64801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:16.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.263+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd640092030 con 0x7fd654069140 2024-04-24T08:37:16.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:16 smithi090 ceph-mon[85282]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:16.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:16 smithi090 ceph-mon[85282]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:16.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:16 smithi090 ceph-mon[85282]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 2 op/s 2024-04-24T08:37:16.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:16 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/819504486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:16.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:16 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/541553003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:37:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:16 smithi149 ceph-mon[66490]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:16 smithi149 ceph-mon[66490]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:16 smithi149 ceph-mon[66490]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 2 op/s 2024-04-24T08:37:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:16 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/819504486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:16 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/541553003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:37:16.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.535+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd654061190 con 0x7fd63c06ecc0 2024-04-24T08:37:16.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.536+0000 7fd652000700 1 -- 172.21.15.90:0/836830648 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fd654061190 con 0x7fd63c06ecc0 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "osd", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "mds" 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T08:37:16.538 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "17/23 daemons upgraded", 2024-04-24T08:37:16.539 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-24T08:37:16.539 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:37:16.539 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:37:16.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 msgr2=0x7fd63c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 0x7fd63c071170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd64801a040 comp rx=0 tx=0).stop 2024-04-24T08:37:16.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 msgr2=0x7fd6541001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd64000d900 tx=0x7fd64000dcc0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 shutdown_connections 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fd63c06ecc0 0x7fd63c071170 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6540687a0 0x7fd6540ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 --2- 172.21.15.90:0/836830648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd654069140 0x7fd6541001e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 >> 172.21.15.90:0/836830648 conn(0x7fd654075130 msgr2=0x7fd6540ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 shutdown_connections 2024-04-24T08:37:16.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.539+0000 7fd65c000700 1 -- 172.21.15.90:0/836830648 wait complete. 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 -- 172.21.15.90:0/783912570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 msgr2=0x7f485c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 --2- 172.21.15.90:0/783912570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c10ca30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f484c009910 tx=0x7f484c009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 -- 172.21.15.90:0/783912570 shutdown_connections 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 --2- 172.21.15.90:0/783912570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 0x7f485c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 --2- 172.21.15.90:0/783912570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c10ca30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 -- 172.21.15.90:0/783912570 >> 172.21.15.90:0/783912570 conn(0x7f485c075e90 msgr2=0x7f485c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 -- 172.21.15.90:0/783912570 shutdown_connections 2024-04-24T08:37:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.693+0000 7f4861c00700 1 -- 172.21.15.90:0/783912570 wait complete. 2024-04-24T08:37:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 Processor -- start 2024-04-24T08:37:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 -- start start 2024-04-24T08:37:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 0x7f485c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c19cf00 con 0x7f485c10c660 2024-04-24T08:37:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.695+0000 7f4861c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1a0c90 con 0x7f485c106b80 2024-04-24T08:37:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45296/0 (socket says 172.21.15.90:45296) 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 -- 172.21.15.90:0/1012610095 learned_addr learned my addr 172.21.15.90:0/1012610095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485be00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 0x7f485c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 -- 172.21.15.90:0/1012610095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 msgr2=0x7f485c19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 0x7f485c19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 -- 172.21.15.90:0/1012610095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0095f0 con 0x7f485c10c660 2024-04-24T08:37:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.696+0000 7f485b400700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f484800e940 tx=0x7f484800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.697+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484800cba0 con 0x7f485c10c660 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.697+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f484800ee90 con 0x7f485c10c660 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.697+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48480055d0 con 0x7f485c10c660 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.697+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c1a0f70 con 0x7f485c10c660 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.697+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c1a14c0 con 0x7f485c10c660 2024-04-24T08:37:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.698+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c10eb40 con 0x7f485c10c660 2024-04-24T08:37:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.699+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4848004750 con 0x7f485c10c660 2024-04-24T08:37:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.700+0000 7f4858c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 0x7f4840071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.700+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4848014070 con 0x7f485c10c660 2024-04-24T08:37:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.700+0000 7f485be00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 0x7f4840071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.701+0000 7f485be00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 0x7f4840071170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f484c0059d0 tx=0x7f484c005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:16.706+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4848057f60 con 0x7f485c10c660 2024-04-24T08:37:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.049+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f485c062380 con 0x7f485c10c660 2024-04-24T08:37:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.050+0000 7f4858c00700 1 -- 172.21.15.90:0/1012610095 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f484805b620 con 0x7f485c10c660 2024-04-24T08:37:17.052 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:37:17.052 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:37:17.052 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:37:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 msgr2=0x7f4840071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 0x7f4840071170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f484c0059d0 tx=0x7f484c005940 comp rx=0 tx=0).stop 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 msgr2=0x7f485c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f484800e940 tx=0x7f484800ec50 comp rx=0 tx=0).stop 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 shutdown_connections 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f484006ecc0 0x7f4840071170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485c106b80 0x7f485c19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 --2- 172.21.15.90:0/1012610095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f485c10c660 0x7f485c19c820 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 >> 172.21.15.90:0/1012610095 conn(0x7f485c075e90 msgr2=0x7f485c0779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 shutdown_connections 2024-04-24T08:37:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:17.053+0000 7f4861c00700 1 -- 172.21.15.90:0/1012610095 wait complete. 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1012610095' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-04-24T08:37:17.890 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.891 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:17 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1012610095' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:17.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:17 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:20.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:20 smithi149 ceph-mon[66490]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:20.527 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:20 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:20.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:20 smithi090 ceph-mon[85282]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:20.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:20.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:20 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:21.390 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:21 smithi090 ceph-mon[85282]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:21.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:21 smithi149 ceph-mon[66490]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:23 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:23 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:23 smithi149 ceph-mon[66490]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:23.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:23.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:23 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:23.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:23 smithi090 ceph-mon[85282]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T08:37:26.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:25 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:26.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T08:37:26.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:25 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-24T08:37:27.044 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]': finished 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi149"}]: dispatch 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi149"}]': finished 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all iscsi 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all nfs 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Setting container_image for all nvmeof 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.045 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.046 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Finalizing container_image settings 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T08:37:27.047 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: Upgrade: Complete! 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:27.049 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:27.050 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: fsmap cephfs:1 {0=cephfs.smithi090.qbyvpv=up:active} 1 up:standby-replay 2 up:standby 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]': finished 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi149"}]: dispatch 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi149"}]': finished 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all iscsi 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all nfs 2024-04-24T08:37:27.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Setting container_image for all nvmeof 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Finalizing container_image settings 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T08:37:27.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T08:37:27.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: Upgrade: Complete! 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T08:37:27.282 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:37:27.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:28.260 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:27 smithi090 ceph-mon[85282]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:28.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:27 smithi149 ceph-mon[66490]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:29.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:29 smithi149 ceph-mon[66490]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:29 smithi090 ceph-mon[85282]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:30.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:30.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:37:31.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:31 smithi149 ceph-mon[66490]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:31 smithi090 ceph-mon[85282]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:33.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:33 smithi090 ceph-mon[85282]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:33 smithi149 ceph-mon[66490]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:35 smithi149 ceph-mon[66490]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:35 smithi090 ceph-mon[85282]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:37.717 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:37 smithi149 ceph-mon[66490]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:37 smithi090 ceph-mon[85282]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:39.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:39 smithi149 ceph-mon[66490]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:39 smithi090 ceph-mon[85282]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:41 smithi149 ceph-mon[66490]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:41.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:41 smithi090 ceph-mon[85282]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:43.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:43 smithi090 ceph-mon[85282]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:43 smithi149 ceph-mon[66490]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:37:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:46 smithi149 ceph-mon[66490]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:46 smithi090 ceph-mon[85282]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:47.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 -- 172.21.15.90:0/1548493807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 msgr2=0x7fa600106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 --2- 172.21.15.90:0/1548493807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa600106ff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009910 tx=0x7fa5ec009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 -- 172.21.15.90:0/1548493807 shutdown_connections 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 --2- 172.21.15.90:0/1548493807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa600106ff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 --2- 172.21.15.90:0/1548493807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 0x7fa60010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 -- 172.21.15.90:0/1548493807 >> 172.21.15.90:0/1548493807 conn(0x7fa600076060 msgr2=0x7fa600078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 -- 172.21.15.90:0/1548493807 shutdown_connections 2024-04-24T08:37:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.227+0000 7fa607a00700 1 -- 172.21.15.90:0/1548493807 wait complete. 2024-04-24T08:37:47.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.228+0000 7fa607a00700 1 Processor -- start 2024-04-24T08:37:47.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa607a00700 1 -- start start 2024-04-24T08:37:47.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa607a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa607a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 0x7fa60019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa607a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60019d060 con 0x7fa600106b80 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa607a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001a0df0 con 0x7fa60010c8f0 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa605000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa605000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40924/0 (socket says 172.21.15.90:40924) 2024-04-24T08:37:47.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.229+0000 7fa605000700 1 -- 172.21.15.90:0/332431983 learned_addr learned my addr 172.21.15.90:0/332431983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:47.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa5ffe00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 0x7fa60019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:47.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa605000700 1 -- 172.21.15.90:0/332431983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 msgr2=0x7fa60019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:47.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa605000700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 0x7fa60019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa605000700 1 -- 172.21.15.90:0/332431983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0095f0 con 0x7fa600106b80 2024-04-24T08:37:47.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa605000700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d8d0 tx=0x7fa5f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.230+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f4009940 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.231+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001a10d0 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.231+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5f4010460 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.231+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f400f5f0 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.231+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001a1620 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.232+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60010ecc0 con 0x7fa600106b80 2024-04-24T08:37:47.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.233+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa5f40105d0 con 0x7fa600106b80 2024-04-24T08:37:47.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.234+0000 7fa5fd600700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 0x7fa5e8071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:47.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.234+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa5f408ef70 con 0x7fa600106b80 2024-04-24T08:37:47.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.234+0000 7fa5ffe00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 0x7fa5e8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:47.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.235+0000 7fa5ffe00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 0x7fa5e8071170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa60019da60 tx=0x7fa5ec01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:47.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.239+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5f4057c30 con 0x7fa600106b80 2024-04-24T08:37:47.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.485+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6001a1840 con 0x7fa5e806ecc0 2024-04-24T08:37:47.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:47 smithi090 ceph-mon[85282]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:47.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.488+0000 7fa5fd600700 1 -- 172.21.15.90:0/332431983 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa6001a1840 con 0x7fa5e806ecc0 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 msgr2=0x7fa5e8071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 0x7fa5e8071170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa60019da60 tx=0x7fa5ec01a040 comp rx=0 tx=0).stop 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 msgr2=0x7fa60019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d8d0 tx=0x7fa5f400dc90 comp rx=0 tx=0).stop 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 shutdown_connections 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa5e806ecc0 0x7fa5e8071170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa600106b80 0x7fa60019c440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.490+0000 7fa607a00700 1 --2- 172.21.15.90:0/332431983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa60010c8f0 0x7fa60019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:47.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.491+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 >> 172.21.15.90:0/332431983 conn(0x7fa600076060 msgr2=0x7fa600077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:47.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.491+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 shutdown_connections 2024-04-24T08:37:47.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:47.491+0000 7fa607a00700 1 -- 172.21.15.90:0/332431983 wait complete. 2024-04-24T08:37:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:47 smithi149 ceph-mon[66490]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:37:48.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch ps' 2024-04-24T08:37:48.754 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:48 smithi149 ceph-mon[66490]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:48.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:48 smithi090 ceph-mon[85282]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:48.866 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:37:49.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:49 smithi090 ceph-mon[85282]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:49 smithi149 ceph-mon[66490]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 -- 172.21.15.90:0/2896641601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 msgr2=0x7fb1c810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 --2- 172.21.15.90:0/2896641601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c810ccc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009910 tx=0x7fb1b8009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 -- 172.21.15.90:0/2896641601 shutdown_connections 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 --2- 172.21.15.90:0/2896641601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 0x7fb1c8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 --2- 172.21.15.90:0/2896641601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c810ccc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 -- 172.21.15.90:0/2896641601 >> 172.21.15.90:0/2896641601 conn(0x7fb1c8076060 msgr2=0x7fb1c8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 -- 172.21.15.90:0/2896641601 shutdown_connections 2024-04-24T08:37:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.963+0000 7fb1cce00700 1 -- 172.21.15.90:0/2896641601 wait complete. 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.964+0000 7fb1cce00700 1 Processor -- start 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.964+0000 7fb1cce00700 1 -- start start 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1cce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 0x7fb1c819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1cce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1cce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819d100 con 0x7fb1c810c8f0 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1cce00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81a0e90 con 0x7fb1c8106b80 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1c5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1c5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39784/0 (socket says 172.21.15.90:39784) 2024-04-24T08:37:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1c5400700 1 -- 172.21.15.90:0/732999573 learned_addr learned my addr 172.21.15.90:0/732999573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1c5e00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 0x7fb1c819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.965+0000 7fb1c5400700 1 -- 172.21.15.90:0/732999573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 msgr2=0x7fb1c819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1c5400700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 0x7fb1c819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1c5400700 1 -- 172.21.15.90:0/732999573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80095f0 con 0x7fb1c810c8f0 2024-04-24T08:37:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1c5400700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b4b0 tx=0x7fb1b400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b4011800 con 0x7fb1c810c8f0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c81a1170 con 0x7fb1c810c8f0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b4011e20 con 0x7fb1c810c8f0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b401a400 con 0x7fb1c810c8f0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.966+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c81a16c0 con 0x7fb1c810c8f0 2024-04-24T08:37:50.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.968+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c8109b20 con 0x7fb1c810c8f0 2024-04-24T08:37:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.969+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb1b4010380 con 0x7fb1c810c8f0 2024-04-24T08:37:50.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.970+0000 7fb1bea00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:50.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.970+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb1b408ea30 con 0x7fb1c810c8f0 2024-04-24T08:37:50.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.970+0000 7fb1c5e00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:50.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.971+0000 7fb1c5e00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 0x7fb1ac071170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8006010 tx=0x7fb1b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:50.975+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb1b4057790 con 0x7fb1c810c8f0 2024-04-24T08:37:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.215+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1c81a1990 con 0x7fb1ac06ecc0 2024-04-24T08:37:51.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.247+0000 7fb1bea00700 1 -- 172.21.15.90:0/732999573 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb1c81a1990 con 0x7fb1ac06ecc0 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (10m) 33s ago 24m 13.5M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (49s) 33s ago 25m 7856k - 19.0.0-3116-g1f81fb39 b98aea5aa06f a7884c73d8bf 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (40s) 36s ago 23m 6937k - 19.0.0-3116-g1f81fb39 b98aea5aa06f d4efef4d6133 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (7m) 33s ago 25m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:37:51.250 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (6m) 36s ago 23m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (10m) 33s ago 24m 73.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (2m) 33s ago 17m 162M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 247c152849b1 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (2m) 33s ago 17m 88.3M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (101s) 36s ago 17m 12.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f c05a5e07bc14 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (72s) 36s ago 17m 12.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f e28402bceb34 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (13m) 33s ago 26m 512M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (12m) 36s ago 23m 453M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (8m) 33s ago 26m 58.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (8m) 36s ago 23m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (12m) 33s ago 25m 7834k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:37:51.251 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (11m) 36s ago 23m 7822k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (6m) 33s ago 22m 99.4M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (5m) 33s ago 21m 100M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (5m) 33s ago 21m 86.7M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (4m) 36s ago 20m 102M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (3m) 36s ago 20m 97.1M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (3m) 36s ago 19m 88.7M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:37:51.252 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (11m) 33s ago 24m 76.7M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.250+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 msgr2=0x7fb1ac071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.250+0000 7fb1cce00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 0x7fb1ac071170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8006010 tx=0x7fb1b801a040 comp rx=0 tx=0).stop 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 msgr2=0x7fb1c819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b4b0 tx=0x7fb1b400b870 comp rx=0 tx=0).stop 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 shutdown_connections 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1c8106b80 0x7fb1c819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 --2- 172.21.15.90:0/732999573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1c810c8f0 0x7fb1c819ca20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 >> 172.21.15.90:0/732999573 conn(0x7fb1c8076060 msgr2=0x7fb1c8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:51.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 shutdown_connections 2024-04-24T08:37:51.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:51.251+0000 7fb1cce00700 1 -- 172.21.15.90:0/732999573 wait complete. 2024-04-24T08:37:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:51 smithi090 ceph-mon[85282]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:51.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:51 smithi090 ceph-mon[85282]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:51 smithi149 ceph-mon[66490]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:51.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:51 smithi149 ceph-mon[66490]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:52.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch upgrade status' 2024-04-24T08:37:52.442 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:37:53.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:53 smithi090 ceph-mon[85282]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:53 smithi149 ceph-mon[66490]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:54.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 -- 172.21.15.90:0/1699966219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72241003f0 msgr2=0x7f72241007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 --2- 172.21.15.90:0/1699966219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72241003f0 0x7f72241007c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7214009910 tx=0x7f7214009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 -- 172.21.15.90:0/1699966219 shutdown_connections 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 --2- 172.21.15.90:0/1699966219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7224101c50 0x7f72241020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 --2- 172.21.15.90:0/1699966219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72241003f0 0x7f72241007c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 -- 172.21.15.90:0/1699966219 >> 172.21.15.90:0/1699966219 conn(0x7f72240fbbf0 msgr2=0x7f72240fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 -- 172.21.15.90:0/1699966219 shutdown_connections 2024-04-24T08:37:54.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.563+0000 7f7223e00700 1 -- 172.21.15.90:0/1699966219 wait complete. 2024-04-24T08:37:54.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.564+0000 7f7223e00700 1 Processor -- start 2024-04-24T08:37:54.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.564+0000 7f7223e00700 1 -- start start 2024-04-24T08:37:54.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7223e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72241003f0 0x7f7224073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7223e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7223e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224073ad0 con 0x7f7224101c50 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7223e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224073c10 con 0x7f72241003f0 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39806/0 (socket says 172.21.15.90:39806) 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 -- 172.21.15.90:0/3924997775 learned_addr learned my addr 172.21.15.90:0/3924997775 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:54.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222a00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72241003f0 0x7f7224073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 -- 172.21.15.90:0/3924997775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72241003f0 msgr2=0x7f7224073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72241003f0 0x7f7224073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.565+0000 7f7222000700 1 -- 172.21.15.90:0/3924997775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140095f0 con 0x7f7224101c50 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7222000700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f721000d8d0 tx=0x7f721000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7210009940 con 0x7f7224101c50 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7210010460 con 0x7f7224101c50 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72241a1e80 con 0x7f7224101c50 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721000f5f0 con 0x7f7224101c50 2024-04-24T08:37:54.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.566+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72241a2310 con 0x7f7224101c50 2024-04-24T08:37:54.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.567+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7224062380 con 0x7f7224101c50 2024-04-24T08:37:54.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.569+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f72100105d0 con 0x7f7224101c50 2024-04-24T08:37:54.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.570+0000 7f7228a00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 0x7f7208071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.570+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f721008ee60 con 0x7f7224101c50 2024-04-24T08:37:54.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.570+0000 7f7222a00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 0x7f7208071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.571+0000 7f7222a00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 0x7f7208071170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f721401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:54.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.575+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7210057ba0 con 0x7f7224101c50 2024-04-24T08:37:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.861+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72240fcb40 con 0x7f720806ecc0 2024-04-24T08:37:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.863+0000 7f7228a00700 1 -- 172.21.15.90:0/3924997775 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f72240fcb40 con 0x7f720806ecc0 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": null, 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": false, 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "", 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": null, 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-24T08:37:54.864 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:37:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.866+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 msgr2=0x7f7208071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.866+0000 7f7223e00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 0x7f7208071170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f721401a040 comp rx=0 tx=0).stop 2024-04-24T08:37:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 msgr2=0x7f7224073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f721000d8d0 tx=0x7f721000dc90 comp rx=0 tx=0).stop 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 shutdown_connections 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f720806ecc0 0x7f7208071170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72241003f0 0x7f7224073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 --2- 172.21.15.90:0/3924997775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7224101c50 0x7f7224073590 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 >> 172.21.15.90:0/3924997775 conn(0x7f72240fbbf0 msgr2=0x7f72240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 shutdown_connections 2024-04-24T08:37:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:54.867+0000 7f7223e00700 1 -- 172.21.15.90:0/3924997775 wait complete. 2024-04-24T08:37:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:55 smithi090 ceph-mon[85282]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:55 smithi090 ceph-mon[85282]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:55 smithi149 ceph-mon[66490]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:37:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:55 smithi149 ceph-mon[66490]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:37:55.920 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph health detail' 2024-04-24T08:37:56.327 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:37:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:57 smithi090 ceph-mon[85282]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:57.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:57 smithi149 ceph-mon[66490]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:37:58.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 -- 172.21.15.90:0/3174008320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3664106b80 msgr2=0x7f3664106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 --2- 172.21.15.90:0/3174008320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3664106b80 0x7f3664106ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3650009910 tx=0x7f3650009c20 comp rx=0 tx=0).stop 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 -- 172.21.15.90:0/3174008320 shutdown_connections 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 --2- 172.21.15.90:0/3174008320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3664106b80 0x7f3664106ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 --2- 172.21.15.90:0/3174008320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366410c8f0 0x7f366410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 -- 172.21.15.90:0/3174008320 >> 172.21.15.90:0/3174008320 conn(0x7f3664076060 msgr2=0x7f3664078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 -- 172.21.15.90:0/3174008320 shutdown_connections 2024-04-24T08:37:58.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.460+0000 7f366a800700 1 -- 172.21.15.90:0/3174008320 wait complete. 2024-04-24T08:37:58.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.461+0000 7f366a800700 1 Processor -- start 2024-04-24T08:37:58.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.461+0000 7f366a800700 1 -- start start 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f366a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3664106b80 0x7f366419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f366a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f366a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419cfd0 con 0x7f366410c8f0 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f366a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419d110 con 0x7f3664106b80 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f3662a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f3662a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39820/0 (socket says 172.21.15.90:39820) 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f3662a00700 1 -- 172.21.15.90:0/512118059 learned_addr learned my addr 172.21.15.90:0/512118059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.462+0000 7f3663400700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3664106b80 0x7f366419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f3662a00700 1 -- 172.21.15.90:0/512118059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3664106b80 msgr2=0x7f366419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f3662a00700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3664106b80 0x7f366419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f3662a00700 1 -- 172.21.15.90:0/512118059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500095f0 con 0x7f366410c8f0 2024-04-24T08:37:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f3662a00700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f365000b5c0 tx=0x7f365000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365001d070 con 0x7f366410c8f0 2024-04-24T08:37:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365000f460 con 0x7f366410c8f0 2024-04-24T08:37:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36641a0f00 con 0x7f366410c8f0 2024-04-24T08:37:58.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3650017610 con 0x7f366410c8f0 2024-04-24T08:37:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.463+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36641a13f0 con 0x7f366410c8f0 2024-04-24T08:37:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.465+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366410ecc0 con 0x7f366410c8f0 2024-04-24T08:37:58.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.466+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f365000f5d0 con 0x7f366410c8f0 2024-04-24T08:37:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.467+0000 7f365be00700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 0x7f3648071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.467+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f36500904c0 con 0x7f366410c8f0 2024-04-24T08:37:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.467+0000 7f3663400700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 0x7f3648071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:58.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.468+0000 7f3663400700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 0x7f3648071170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3654005ea0 tx=0x7f3654005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:58.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.472+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3650059200 con 0x7f366410c8f0 2024-04-24T08:37:58.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.790+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f366404f2a0 con 0x7f366410c8f0 2024-04-24T08:37:58.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.790+0000 7f365be00700 1 -- 172.21.15.90:0/512118059 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f365005c8c0 con 0x7f366410c8f0 2024-04-24T08:37:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:37:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T08:37:58.791 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T08:37:58.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 msgr2=0x7f3648071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:58.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 0x7f3648071170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3654005ea0 tx=0x7f3654005e30 comp rx=0 tx=0).stop 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 msgr2=0x7f366419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f365000b5c0 tx=0x7f365000be00 comp rx=0 tx=0).stop 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 shutdown_connections 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f364806ecc0 0x7f3648071170 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3664106b80 0x7f366419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 --2- 172.21.15.90:0/512118059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366410c8f0 0x7f366419c980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 >> 172.21.15.90:0/512118059 conn(0x7f3664076060 msgr2=0x7f3664077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 shutdown_connections 2024-04-24T08:37:58.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:37:58.794+0000 7f366a800700 1 -- 172.21.15.90:0/512118059 wait complete. 2024-04-24T08:37:59.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:58 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/512118059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:37:59.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:58 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/512118059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T08:37:59.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions' 2024-04-24T08:38:00.035 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:38:00.189 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:37:59 smithi090 ceph-mon[85282]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:00.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:37:59 smithi149 ceph-mon[66490]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:01.248 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:01.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:02.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:01 smithi090 ceph-mon[85282]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:02.143 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:01 smithi149 ceph-mon[66490]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 -- 172.21.15.90:0/3353784751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 msgr2=0x7fbd78107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 --2- 172.21.15.90:0/3353784751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd78107010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd60009c20 comp rx=0 tx=0).stop 2024-04-24T08:38:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 -- 172.21.15.90:0/3353784751 shutdown_connections 2024-04-24T08:38:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 --2- 172.21.15.90:0/3353784751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd78107010 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 --2- 172.21.15.90:0/3353784751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 0x7fbd78102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 -- 172.21.15.90:0/3353784751 >> 172.21.15.90:0/3353784751 conn(0x7fbd780fdf40 msgr2=0x7fbd78100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.168+0000 7fbd7de00700 1 -- 172.21.15.90:0/3353784751 shutdown_connections 2024-04-24T08:38:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.169+0000 7fbd7de00700 1 -- 172.21.15.90:0/3353784751 wait complete. 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 Processor -- start 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 -- start start 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 0x7fbd78198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd78198ea0 con 0x7fbd78103040 2024-04-24T08:38:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd7de00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819cc30 con 0x7fbd781026a0 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.170+0000 7fbd6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43516/0 (socket says 172.21.15.90:43516) 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 -- 172.21.15.90:0/708074748 learned_addr learned my addr 172.21.15.90:0/708074748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd76a00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 0x7fbd78198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 -- 172.21.15.90:0/708074748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 msgr2=0x7fbd78198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 0x7fbd78198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 -- 172.21.15.90:0/708074748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600095f0 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.171+0000 7fbd6fe00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000bda0 tx=0x7fbd6000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.172+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6001d070 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.172+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd60004fb0 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.172+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6000f460 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.172+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7819ceb0 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.172+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7819d3a0 con 0x7fbd78103040 2024-04-24T08:38:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.173+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7810a960 con 0x7fbd78103040 2024-04-24T08:38:02.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.174+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbd60022800 con 0x7fbd78103040 2024-04-24T08:38:02.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.175+0000 7fbd7ca00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 0x7fbd5c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:02.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.175+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd600903e0 con 0x7fbd78103040 2024-04-24T08:38:02.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.175+0000 7fbd76a00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 0x7fbd5c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:02.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.176+0000 7fbd76a00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 0x7fbd5c0711c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd68008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:02.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.180+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbd600592f0 con 0x7fbd78103040 2024-04-24T08:38:02.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.505+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd7804f2a0 con 0x7fbd78103040 2024-04-24T08:38:02.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.505+0000 7fbd7ca00700 1 -- 172.21.15.90:0/708074748 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbd6005c9b0 con 0x7fbd78103040 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:38:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 14 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:38:02.507 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:38:02.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 msgr2=0x7fbd5c0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 0x7fbd5c0711c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd68008040 comp rx=0 tx=0).stop 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 msgr2=0x7fbd781987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000bda0 tx=0x7fbd6000be80 comp rx=0 tx=0).stop 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 shutdown_connections 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbd5c06ed10 0x7fbd5c0711c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd781026a0 0x7fbd78198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 --2- 172.21.15.90:0/708074748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd78103040 0x7fbd781987c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 >> 172.21.15.90:0/708074748 conn(0x7fbd780fdf40 msgr2=0x7fbd780ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:02.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 shutdown_connections 2024-04-24T08:38:02.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:38:02.509+0000 7fbd7de00700 1 -- 172.21.15.90:0/708074748 wait complete. 2024-04-24T08:38:03.114 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:02 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/708074748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:38:03.161 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-24T08:38:03.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:02 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/708074748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:38:03.567 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:38:03.994 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:03 smithi090 ceph-mon[85282]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:04.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:03 smithi149 ceph-mon[66490]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:05.373 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:05 smithi090 ceph-mon[85282]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:05.611 INFO:teuthology.orchestra.run.smithi090.stdout:wait for servicemap items w/ changing names to refresh 2024-04-24T08:38:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:05 smithi149 ceph-mon[66490]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:06.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'sleep 60' 2024-04-24T08:38:06.744 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:38:07.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:07 smithi090 ceph-mon[85282]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:07.579 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:07 smithi149 ceph-mon[66490]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:09.630 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:09 smithi090 ceph-mon[85282]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:09 smithi149 ceph-mon[66490]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:11 smithi149 ceph-mon[66490]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:11 smithi090 ceph-mon[85282]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:13.678 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:13 smithi090 ceph-mon[85282]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:13 smithi149 ceph-mon[66490]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:15.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:16 smithi149 ceph-mon[66490]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:16 smithi090 ceph-mon[85282]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:17 smithi149 ceph-mon[66490]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:17.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:17 smithi090 ceph-mon[85282]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:19 smithi149 ceph-mon[66490]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:19 smithi090 ceph-mon[85282]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:22.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:21 smithi149 ceph-mon[66490]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:22.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:21 smithi090 ceph-mon[85282]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:23.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:23 smithi090 ceph-mon[85282]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:23 smithi149 ceph-mon[66490]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:25.770 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:25 smithi090 ceph-mon[85282]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:25.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:25 smithi149 ceph-mon[66490]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:27.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:26 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:27.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:26 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:28.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:27 smithi149 ceph-mon[66490]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:28.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:27 smithi090 ceph-mon[85282]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:38:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:38:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:38:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:38:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:38:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:38:30.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:29 smithi149 ceph-mon[66490]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:30.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:29 smithi090 ceph-mon[85282]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:31.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:31.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:32.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:31 smithi149 ceph-mon[66490]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:32.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:31 smithi090 ceph-mon[85282]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:33.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:33 smithi090 ceph-mon[85282]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:33 smithi149 ceph-mon[66490]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:35 smithi149 ceph-mon[66490]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:35 smithi090 ceph-mon[85282]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:37.613 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:37 smithi149 ceph-mon[66490]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:37 smithi090 ceph-mon[85282]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:39 smithi149 ceph-mon[66490]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:39 smithi090 ceph-mon[85282]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:41 smithi149 ceph-mon[66490]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:41.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:41 smithi090 ceph-mon[85282]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:43.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:43 smithi090 ceph-mon[85282]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:43 smithi149 ceph-mon[66490]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:38:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:38:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:46 smithi149 ceph-mon[66490]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:46 smithi090 ceph-mon[85282]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:47 smithi149 ceph-mon[66490]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:47.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:47 smithi090 ceph-mon[85282]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:49 smithi149 ceph-mon[66490]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:49 smithi090 ceph-mon[85282]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:51 smithi149 ceph-mon[66490]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:51 smithi090 ceph-mon[85282]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:53.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:53 smithi090 ceph-mon[85282]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:53 smithi149 ceph-mon[66490]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:55 smithi149 ceph-mon[66490]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:55 smithi090 ceph-mon[85282]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:57.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:57 smithi149 ceph-mon[66490]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:57.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:57 smithi090 ceph-mon[85282]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:38:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:38:59 smithi149 ceph-mon[66490]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:38:59.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:38:59 smithi090 ceph-mon[85282]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:01 smithi149 ceph-mon[66490]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:01 smithi090 ceph-mon[85282]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:03.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:03 smithi090 ceph-mon[85282]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:03 smithi149 ceph-mon[66490]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:05 smithi149 ceph-mon[66490]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:05.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:05 smithi090 ceph-mon[85282]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:07 smithi149 ceph-mon[66490]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:07.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:07 smithi090 ceph-mon[85282]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:39:09.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph orch ps' 2024-04-24T08:39:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:09 smithi149 ceph-mon[66490]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:09 smithi090 ceph-mon[85282]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:09.885 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:39:11.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:11 smithi090 ceph-mon[85282]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:11 smithi149 ceph-mon[66490]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:12.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 -- 172.21.15.90:0/2526167517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 msgr2=0x7fbff8073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:12.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 --2- 172.21.15.90:0/2526167517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8073480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b0d0 tx=0x7fbff000b3e0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 -- 172.21.15.90:0/2526167517 shutdown_connections 2024-04-24T08:39:12.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 --2- 172.21.15.90:0/2526167517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 0x7fbff810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 --2- 172.21.15.90:0/2526167517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8073480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 -- 172.21.15.90:0/2526167517 >> 172.21.15.90:0/2526167517 conn(0x7fbff80fbda0 msgr2=0x7fbff80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:12.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 -- 172.21.15.90:0/2526167517 shutdown_connections 2024-04-24T08:39:12.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.046+0000 7fc000800700 1 -- 172.21.15.90:0/2526167517 wait complete. 2024-04-24T08:39:12.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 Processor -- start 2024-04-24T08:39:12.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 -- start start 2024-04-24T08:39:12.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:12.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 0x7fbff8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8198e30 con 0x7fbff80730b0 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.047+0000 7fc000800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819cbc0 con 0x7fbff80739c0 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36050/0 (socket says 172.21.15.90:36050) 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 -- 172.21.15.90:0/574091633 learned_addr learned my addr 172.21.15.90:0/574091633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffd400700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 0x7fbff8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 -- 172.21.15.90:0/574091633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 msgr2=0x7fbff8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 0x7fbff8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 -- 172.21.15.90:0/574091633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009d00 con 0x7fbff80730b0 2024-04-24T08:39:12.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.048+0000 7fbffde00700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b0a0 tx=0x7fbff0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.049+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff000e040 con 0x7fbff80730b0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.049+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbff0007b90 con 0x7fbff80730b0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.049+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff001d740 con 0x7fbff80730b0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.049+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff819ce40 con 0x7fbff80730b0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.049+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff819d330 con 0x7fbff80730b0 2024-04-24T08:39:12.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.050+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff8109cc0 con 0x7fbff80730b0 2024-04-24T08:39:12.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.051+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbff0007d00 con 0x7fbff80730b0 2024-04-24T08:39:12.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.052+0000 7fbfeea00700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 0x7fbfe4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:12.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.052+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbff0090690 con 0x7fbff80730b0 2024-04-24T08:39:12.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.053+0000 7fbffd400700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 0x7fbfe4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:12.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.053+0000 7fbffd400700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 0x7fbfe4071170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005950 tx=0x7fbfe80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:12.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.058+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbff00593d0 con 0x7fbff80730b0 2024-04-24T08:39:12.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.302+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 --> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbff8199680 con 0x7fbfe406ecc0 2024-04-24T08:39:12.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.335+0000 7fbfeea00700 1 -- 172.21.15.90:0/574091633 <== mgr.34100 v2:172.21.15.90:6800/3216343886 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbff8199680 con 0x7fbfe406ecc0 2024-04-24T08:39:12.335 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:39:12.335 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (12m) 115s ago 26m 13.5M - 0.25.0 c8568f914cd2 07398a90628a 2024-04-24T08:39:12.335 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (2m) 115s ago 26m 7856k - 19.0.0-3116-g1f81fb39 b98aea5aa06f a7884c73d8bf 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi149 smithi149 running (2m) 117s ago 25m 6937k - 19.0.0-3116-g1f81fb39 b98aea5aa06f d4efef4d6133 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (8m) 115s ago 26m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec0696033de2 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi149 smithi149 running (8m) 117s ago 25m 6660k - 19.0.0-3116-g1f81fb39 b98aea5aa06f ba56bc5af6e6 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (11m) 115s ago 25m 73.0M - 9.4.12 2bacad6d85d8 f837474014ab 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.lumztb smithi090 running (3m) 115s ago 18m 162M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 247c152849b1 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.qbyvpv smithi090 running (4m) 115s ago 19m 88.3M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5f6bf5a8361a 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.eazcmc smithi149 running (3m) 117s ago 18m 12.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f c05a5e07bc14 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi149.zwjtmw smithi149 running (2m) 117s ago 18m 12.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f e28402bceb34 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.jsrfxo smithi090 *:8443,9283,8765 running (15m) 115s ago 27m 512M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bd29ba4b5e28 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi149.vavftl smithi149 *:8443,9283,8765 running (13m) 117s ago 24m 453M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 1377d4db259c 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (10m) 115s ago 28m 58.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb68ff47c36 2024-04-24T08:39:12.336 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi149 smithi149 running (9m) 117s ago 24m 44.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f f99b1e7df199 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (13m) 115s ago 26m 7834k - 1.5.0 0da6a335fe13 365bd4421979 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi149 smithi149 *:9100 running (13m) 117s ago 24m 7822k - 1.5.0 0da6a335fe13 6812387d8978 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 115s ago 23m 99.4M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f a5db84b0ac4b 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 115s ago 23m 100M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8bb613c07b47 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (6m) 115s ago 22m 86.7M 1216M 19.0.0-3116-g1f81fb39 b98aea5aa06f 88a827e60f78 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi149 running (5m) 117s ago 21m 102M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 17e278364e03 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi149 running (5m) 117s ago 21m 97.1M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3298f7699768 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (4m) 117s ago 20m 88.7M 4096M 19.0.0-3116-g1f81fb39 b98aea5aa06f ecd6e5e644e3 2024-04-24T08:39:12.337 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (12m) 115s ago 25m 76.7M - 2.43.0 a07b618ecd1d e737c460f76d 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 msgr2=0x7fbfe4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 0x7fbfe4071170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005950 tx=0x7fbfe80058e0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 msgr2=0x7fbff8198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b0a0 tx=0x7fbff0003ce0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 shutdown_connections 2024-04-24T08:39:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbfe406ecc0 0x7fbfe4071170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbff80730b0 0x7fbff8198210 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 --2- 172.21.15.90:0/574091633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbff80739c0 0x7fbff8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 >> 172.21.15.90:0/574091633 conn(0x7fbff80fbda0 msgr2=0x7fbff8106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 shutdown_connections 2024-04-24T08:39:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:12.338+0000 7fc000800700 1 -- 172.21.15.90:0/574091633 wait complete. 2024-04-24T08:39:13.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions' 2024-04-24T08:39:13.438 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:39:13.598 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:13 smithi090 ceph-mon[85282]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:13.598 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:13 smithi090 ceph-mon[85282]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:13 smithi149 ceph-mon[66490]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:13 smithi149 ceph-mon[66490]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:15.291 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 -- 172.21.15.90:0/3220456470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9430103e30 msgr2=0x7f94301042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 --2- 172.21.15.90:0/3220456470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9430103e30 0x7f94301042a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f941c009910 tx=0x7f941c009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 -- 172.21.15.90:0/3220456470 shutdown_connections 2024-04-24T08:39:15.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 --2- 172.21.15.90:0/3220456470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9430103e30 0x7f94301042a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 --2- 172.21.15.90:0/3220456470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94301025d0 0x7f94301029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 -- 172.21.15.90:0/3220456470 >> 172.21.15.90:0/3220456470 conn(0x7f94300fddb0 msgr2=0x7f94301001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 -- 172.21.15.90:0/3220456470 shutdown_connections 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.537+0000 7f942fe00700 1 -- 172.21.15.90:0/3220456470 wait complete. 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.538+0000 7f942fe00700 1 Processor -- start 2024-04-24T08:39:15.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.538+0000 7f942fe00700 1 -- start start 2024-04-24T08:39:15.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9430103e30 0x7f9430198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430198ab0 con 0x7f94301025d0 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430198bf0 con 0x7f9430103e30 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36060/0 (socket says 172.21.15.90:36060) 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942ea00700 1 -- 172.21.15.90:0/2878040632 learned_addr learned my addr 172.21.15.90:0/2878040632 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.539+0000 7f942e000700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9430103e30 0x7f9430198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942ea00700 1 -- 172.21.15.90:0/2878040632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9430103e30 msgr2=0x7f9430198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942ea00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9430103e30 0x7f9430198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942ea00700 1 -- 172.21.15.90:0/2878040632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0095f0 con 0x7f94301025d0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942ea00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f942000c930 tx=0x7f942000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9420007ad0 con 0x7f94301025d0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9420007c30 con 0x7f94301025d0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9420018610 con 0x7f94301025d0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943019ca40 con 0x7f94301025d0 2024-04-24T08:39:15.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.540+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943019cf40 con 0x7f94301025d0 2024-04-24T08:39:15.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.541+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943010a890 con 0x7f94301025d0 2024-04-24T08:39:15.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.543+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f942000f450 con 0x7f94301025d0 2024-04-24T08:39:15.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.543+0000 7f9427400700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 0x7f9414071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:15.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.544+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f942008f170 con 0x7f94301025d0 2024-04-24T08:39:15.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.544+0000 7f942e000700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 0x7f9414071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:15.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.545+0000 7f942e000700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 0x7f9414071170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9430199540 tx=0x7f941c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:15.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.549+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9420057e30 con 0x7f94301025d0 2024-04-24T08:39:15.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.874+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f943010a270 con 0x7f94301025d0 2024-04-24T08:39:15.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.875+0000 7f9427400700 1 -- 172.21.15.90:0/2878040632 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f942005b4f0 con 0x7f94301025d0 2024-04-24T08:39:15.876 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T08:39:15.876 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 2 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 6 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 4 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)": 14 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T08:39:15.877 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T08:39:15.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 msgr2=0x7f9414071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 0x7f9414071170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9430199540 tx=0x7f941c01a040 comp rx=0 tx=0).stop 2024-04-24T08:39:15.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 msgr2=0x7f9430197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f942000c930 tx=0x7f942000ccf0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 shutdown_connections 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f941406ecc0 0x7f9414071170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94301025d0 0x7f9430197f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 --2- 172.21.15.90:0/2878040632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9430103e30 0x7f9430198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 >> 172.21.15.90:0/2878040632 conn(0x7f94300fddb0 msgr2=0x7f94300fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 shutdown_connections 2024-04-24T08:39:15.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:15.878+0000 7f942fe00700 1 -- 172.21.15.90:0/2878040632 wait complete. 2024-04-24T08:39:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:16 smithi149 ceph-mon[66490]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:16 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2878040632' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:16 smithi090 ceph-mon[85282]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:16 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2878040632' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:16.622 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-24T08:39:17.049 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:39:17.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:17 smithi090 ceph-mon[85282]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:17 smithi149 ceph-mon[66490]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 -- 172.21.15.90:0/3180770582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b8073c00 msgr2=0x7ff8b810c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 --2- 172.21.15.90:0/3180770582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b8073c00 0x7ff8b810c420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009910 tx=0x7ff8a4009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 -- 172.21.15.90:0/3180770582 shutdown_connections 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 --2- 172.21.15.90:0/3180770582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b8073c00 0x7ff8b810c420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 --2- 172.21.15.90:0/3180770582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b80732f0 0x7ff8b80736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 -- 172.21.15.90:0/3180770582 >> 172.21.15.90:0/3180770582 conn(0x7ff8b80fbbc0 msgr2=0x7ff8b80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 -- 172.21.15.90:0/3180770582 shutdown_connections 2024-04-24T08:39:19.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.302+0000 7ff8bda00700 1 -- 172.21.15.90:0/3180770582 wait complete. 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.303+0000 7ff8bda00700 1 Processor -- start 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.303+0000 7ff8bda00700 1 -- start start 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8bda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8bda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b8073c00 0x7ff8b8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8bda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8198b30 con 0x7ff8b80732f0 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8bda00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b819c8c0 con 0x7ff8b8073c00 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36072/0 (socket says 172.21.15.90:36072) 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7e00700 1 -- 172.21.15.90:0/2520667751 learned_addr learned my addr 172.21.15.90:0/2520667751 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7400700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b8073c00 0x7ff8b8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7e00700 1 -- 172.21.15.90:0/2520667751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b8073c00 msgr2=0x7ff8b8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:19.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.304+0000 7ff8b7e00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b8073c00 0x7ff8b8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8b7e00700 1 -- 172.21.15.90:0/2520667751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40095f0 con 0x7ff8b80732f0 2024-04-24T08:39:19.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8b7e00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c8a0 tx=0x7ff8a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:19.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a800cea0 con 0x7ff8b80732f0 2024-04-24T08:39:19.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8a8004830 con 0x7ff8b80732f0 2024-04-24T08:39:19.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b819cba0 con 0x7ff8b80732f0 2024-04-24T08:39:19.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a80055b0 con 0x7ff8b80732f0 2024-04-24T08:39:19.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.305+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b819d0f0 con 0x7ff8b80732f0 2024-04-24T08:39:19.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.307+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b8109b20 con 0x7ff8b80732f0 2024-04-24T08:39:19.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.308+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff8a80049a0 con 0x7ff8b80732f0 2024-04-24T08:39:19.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.308+0000 7ff8b4c00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 0x7ff89c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:19.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.309+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff8a808eb40 con 0x7ff8b80732f0 2024-04-24T08:39:19.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.309+0000 7ff8b7400700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 0x7ff89c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:19.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.309+0000 7ff8b7400700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 0x7ff89c071170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400b5c0 tx=0x7ff8a4005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:19.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.314+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff8a8057880 con 0x7ff8b80732f0 2024-04-24T08:39:19.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:19 smithi090 ceph-mon[85282]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.627+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8b819d3c0 con 0x7ff8b80732f0 2024-04-24T08:39:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.628+0000 7ff8b4c00700 1 -- 172.21.15.90:0/2520667751 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff8a801d070 con 0x7ff8b80732f0 2024-04-24T08:39:19.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 msgr2=0x7ff89c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:19.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 0x7ff89c071170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400b5c0 tx=0x7ff8a4005fd0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 msgr2=0x7ff8b8197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c8a0 tx=0x7ff8a800cc60 comp rx=0 tx=0).stop 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 shutdown_connections 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff89c06ecc0 0x7ff89c071170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8b80732f0 0x7ff8b8197f10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 --2- 172.21.15.90:0/2520667751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8b8073c00 0x7ff8b8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 >> 172.21.15.90:0/2520667751 conn(0x7ff8b80fbbc0 msgr2=0x7ff8b8106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.631+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 shutdown_connections 2024-04-24T08:39:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:19.632+0000 7ff8bda00700 1 -- 172.21.15.90:0/2520667751 wait complete. 2024-04-24T08:39:19.648 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-24T08:39:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:19 smithi149 ceph-mon[66490]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:20.428 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -e sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-24T08:39:20.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:20 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2520667751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:20.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:20 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2520667751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:20.850 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:39:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:21 smithi149 ceph-mon[66490]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:21.777 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:21 smithi090 ceph-mon[85282]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.854+0000 7fc7d6800700 1 -- 172.21.15.90:0/2234543163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d0100e80 msgr2=0x7fc7d0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.854+0000 7fc7d6800700 1 --2- 172.21.15.90:0/2234543163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d0100e80 0x7fc7d0104e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009910 tx=0x7fc7b8009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 -- 172.21.15.90:0/2234543163 shutdown_connections 2024-04-24T08:39:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 --2- 172.21.15.90:0/2234543163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d0100e80 0x7fc7d0104e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:22.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 --2- 172.21.15.90:0/2234543163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d01004e0 0x7fc7d01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:22.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 -- 172.21.15.90:0/2234543163 >> 172.21.15.90:0/2234543163 conn(0x7fc7d00fbda0 msgr2=0x7fc7d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:22.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 -- 172.21.15.90:0/2234543163 shutdown_connections 2024-04-24T08:39:22.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.855+0000 7fc7d6800700 1 -- 172.21.15.90:0/2234543163 wait complete. 2024-04-24T08:39:22.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 Processor -- start 2024-04-24T08:39:22.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 -- start start 2024-04-24T08:39:22.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:22.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d0100e80 0x7fc7d0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0073ad0 con 0x7fc7d01004e0 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.856+0000 7fc7d6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0073c10 con 0x7fc7d0100e80 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49564/0 (socket says 172.21.15.90:49564) 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 -- 172.21.15.90:0/651211153 learned_addr learned my addr 172.21.15.90:0/651211153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7c6a00700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d0100e80 0x7fc7d0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 -- 172.21.15.90:0/651211153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d0100e80 msgr2=0x7fc7d0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d0100e80 0x7fc7d0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 -- 172.21.15.90:0/651211153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80095f0 con 0x7fc7d01004e0 2024-04-24T08:39:22.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.857+0000 7fc7cf400700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d8d0 tx=0x7fc7c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.858+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c0009940 con 0x7fc7d01004e0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.858+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7c0010460 con 0x7fc7d01004e0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.858+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d01a2100 con 0x7fc7d01004e0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.858+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c000f5f0 con 0x7fc7d01004e0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.858+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d01a2510 con 0x7fc7d01004e0 2024-04-24T08:39:22.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.859+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d0108850 con 0x7fc7d01004e0 2024-04-24T08:39:22.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.860+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc7c00105d0 con 0x7fc7d01004e0 2024-04-24T08:39:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.861+0000 7fc7ccc00700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 0x7fc7b4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.861+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc7c008edc0 con 0x7fc7d01004e0 2024-04-24T08:39:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.861+0000 7fc7c6a00700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 0x7fc7b4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:22.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.862+0000 7fc7c6a00700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 0x7fc7b4071170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0074840 tx=0x7fc7b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:22.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:22.866+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc7c0057ba0 con 0x7fc7d01004e0 2024-04-24T08:39:23.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.190+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc7d0108230 con 0x7fc7d01004e0 2024-04-24T08:39:23.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.192+0000 7fc7ccc00700 1 -- 172.21.15.90:0/651211153 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc7c005b260 con 0x7fc7d01004e0 2024-04-24T08:39:23.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 msgr2=0x7fc7b4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:23.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 0x7fc7b4071170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0074840 tx=0x7fc7b801a040 comp rx=0 tx=0).stop 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 msgr2=0x7fc7d0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d8d0 tx=0x7fc7c000dc90 comp rx=0 tx=0).stop 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 shutdown_connections 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fc7b406ecc0 0x7fc7b4071170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d01004e0 0x7fc7d0073050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 --2- 172.21.15.90:0/651211153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7d0100e80 0x7fc7d0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 >> 172.21.15.90:0/651211153 conn(0x7fc7d00fbda0 msgr2=0x7fc7d00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 shutdown_connections 2024-04-24T08:39:23.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:23.195+0000 7fc7d6800700 1 -- 172.21.15.90:0/651211153 wait complete. 2024-04-24T08:39:23.214 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)" 2024-04-24T08:39:23.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:23 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/651211153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:23.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:23 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/651211153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T08:39:23.949 DEBUG:teuthology.parallel:result is None 2024-04-24T08:39:23.949 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:39:23.962 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:39:23.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10d437de-0212-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T08:39:24.377 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-24T08:39:24.525 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:24 smithi090 ceph-mon[85282]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:24.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:24 smithi149 ceph-mon[66490]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:25.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:25 smithi149 ceph-mon[66490]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:25.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:25 smithi090 ceph-mon[85282]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 -- 172.21.15.90:0/1632214366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 msgr2=0x7f227c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 --2- 172.21.15.90:0/1632214366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c068b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f226c009910 tx=0x7f226c009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 -- 172.21.15.90:0/1632214366 shutdown_connections 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 --2- 172.21.15.90:0/1632214366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 0x7f227c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 --2- 172.21.15.90:0/1632214366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c068b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 -- 172.21.15.90:0/1632214366 >> 172.21.15.90:0/1632214366 conn(0x7f227c075130 msgr2=0x7f227c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 -- 172.21.15.90:0/1632214366 shutdown_connections 2024-04-24T08:39:26.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.561+0000 7f2282a00700 1 -- 172.21.15.90:0/1632214366 wait complete. 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.562+0000 7f2282a00700 1 Processor -- start 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f2282a00700 1 -- start start 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f2282a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f2282a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 0x7f227c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f2282a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c100720 con 0x7f227c0687a0 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f2282a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c100860 con 0x7f227c069140 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f227be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f227be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49582/0 (socket says 172.21.15.90:49582) 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.563+0000 7f227be00700 1 -- 172.21.15.90:0/862389153 learned_addr learned my addr 172.21.15.90:0/862389153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f227b400700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 0x7f227c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:26.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f227be00700 1 -- 172.21.15.90:0/862389153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 msgr2=0x7f227c1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:26.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f227be00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 0x7f227c1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f227be00700 1 -- 172.21.15.90:0/862389153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0095f0 con 0x7f227c0687a0 2024-04-24T08:39:26.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f227be00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f226c006010 tx=0x7f226c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:26.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226c01d070 con 0x7f227c0687a0 2024-04-24T08:39:26.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f226c004fb0 con 0x7f227c0687a0 2024-04-24T08:39:26.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.564+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226c00f690 con 0x7f227c0687a0 2024-04-24T08:39:26.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.565+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c100ae0 con 0x7f227c0687a0 2024-04-24T08:39:26.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.565+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c1a2390 con 0x7f227c0687a0 2024-04-24T08:39:26.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.566+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c108790 con 0x7f227c0687a0 2024-04-24T08:39:26.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.567+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f226c0228a0 con 0x7f227c0687a0 2024-04-24T08:39:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.568+0000 7f2278c00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 0x7f2260071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.568+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f226c0906b0 con 0x7f227c0687a0 2024-04-24T08:39:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.569+0000 7f227b400700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 0x7f2260071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.569+0000 7f227b400700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 0x7f2260071170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2268005ea0 tx=0x7f2268005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.574+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f226c059490 con 0x7f227c0687a0 2024-04-24T08:39:26.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.853+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f227c1012a0 con 0x7f227c0687a0 2024-04-24T08:39:26.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.854+0000 7f2278c00700 1 -- 172.21.15.90:0/862389153 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1966 (secure 0 0 0) 0x7f226c05cb50 con 0x7f227c0687a0 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:e38 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 38 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T08:20:07.120167+0000 2024-04-24T08:39:26.859 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T08:37:25.249017+0000 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 86 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:in 0 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34462} 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T08:39:26.860 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data enabled 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34462 members: 34462 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.qbyvpv{0:34462} state up:active seq 26 join_fscid=1 addr [v2:172.21.15.90:6826/1385436580,v1:172.21.15.90:6827/1385436580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.lumztb{0:34494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/4088472812,v1:172.21.15.90:6829/4088472812] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T08:39:26.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:26.862 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.eazcmc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/4101562674,v1:172.21.15.149:6825/4101562674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:39:26.862 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi149.zwjtmw{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6826/1676173850,v1:172.21.15.149:6827/1676173850] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 msgr2=0x7f2260071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 0x7f2260071170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2268005ea0 tx=0x7f2268005e30 comp rx=0 tx=0).stop 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 msgr2=0x7f227c0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f226c006010 tx=0x7f226c00be20 comp rx=0 tx=0).stop 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 shutdown_connections 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f226006ecc0 0x7f2260071170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f227c0687a0 0x7f227c0ffca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 --2- 172.21.15.90:0/862389153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c069140 0x7f227c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:26.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 >> 172.21.15.90:0/862389153 conn(0x7f227c075130 msgr2=0x7f227c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:26.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 shutdown_connections 2024-04-24T08:39:26.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:26.857+0000 7f2282a00700 1 -- 172.21.15.90:0/862389153 wait complete. 2024-04-24T08:39:26.864 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 38 2024-04-24T08:39:27.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:26 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/862389153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:39:27.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:26 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/862389153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:39:27.619 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-24T08:39:27.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 -- 172.21.15.90:0/2945914306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fef70 msgr2=0x7f1298102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 --2- 172.21.15.90:0/2945914306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fef70 0x7f1298102f20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1284009880 tx=0x7f128402bcf0 comp rx=0 tx=0).stop 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 -- 172.21.15.90:0/2945914306 shutdown_connections 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 --2- 172.21.15.90:0/2945914306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fef70 0x7f1298102f20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 --2- 172.21.15.90:0/2945914306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fe5d0 0x7f12980fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.841+0000 7f1297e00640 1 -- 172.21.15.90:0/2945914306 >> 172.21.15.90:0/2945914306 conn(0x7f1298076280 msgr2=0x7f1298076680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.842+0000 7f1297e00640 1 -- 172.21.15.90:0/2945914306 shutdown_connections 2024-04-24T08:39:27.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.842+0000 7f1297e00640 1 -- 172.21.15.90:0/2945914306 wait complete. 2024-04-24T08:39:27.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 Processor -- start 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 -- start start 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fef70 0x7f12981956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298195d80 con 0x7f12980fe5d0 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1297e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298199b50 con 0x7f12980fef70 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1296a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1296a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49592/0 (socket says 172.21.15.90:49592) 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.843+0000 7f1296a00640 1 -- 172.21.15.90:0/1788589873 learned_addr learned my addr 172.21.15.90:0/1788589873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1296000640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fef70 0x7f12981956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1296a00640 1 -- 172.21.15.90:0/1788589873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fef70 msgr2=0x7f12981956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1296a00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fef70 0x7f12981956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1296a00640 1 -- 172.21.15.90:0/1788589873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1284009560 con 0x7f12980fe5d0 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1296a00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f128800b420 tx=0x7f128800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:27.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1288011800 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1288011e20 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f128800f2e0 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1298199e30 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.844+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129819a390 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.845+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129804e850 con 0x7f12980fe5d0 2024-04-24T08:39:27.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.846+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1288010380 con 0x7f12980fe5d0 2024-04-24T08:39:27.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.847+0000 7f128f400640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 0x7f127c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:27.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.847+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f128808e8d0 con 0x7f12980fe5d0 2024-04-24T08:39:27.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.848+0000 7f1296000640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 0x7f127c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:27.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.848+0000 7f1296000640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 0x7f127c0711a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1284036040 tx=0x7f1284005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:27.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:27.852+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1288057590 con 0x7f12980fe5d0 2024-04-24T08:39:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.058+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f129810b820 con 0x7f12980fe5d0 2024-04-24T08:39:28.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.060+0000 7f128f400640 1 -- 172.21.15.90:0/1788589873 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f1288055e60 con 0x7f12980fe5d0 2024-04-24T08:39:28.062 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:28.062 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:36:30.490085+0000","last_up_change":"2024-04-24T08:34:51.191821+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T08:20:06.040754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T08:20:06.663119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6803","nonce":4221900826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6805","nonce":4221900826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6809","nonce":4221900826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6807","nonce":4221900826}]},"public_addr":"172.21.15.90:6803/4221900826","cluster_addr":"172.21.15.90:6805/4221900826","heartbeat_back_addr":"172.21.15.90:6809/4221900826","heartbeat_front_addr":"172.21.15.90:6807/4221900826","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6811","nonce":1858244546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6813","nonce":1858244546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6817","nonce":1858244546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6815","nonce":1858244546}]},"public_addr":"172.21.15.90:6811/1858244546","cluster_addr":"172.21.15.90:6813/1858244546","heartbeat_back_addr":"172.21.15.90:6817/1858244546","heartbeat_front_addr":"172.21.15.90:6815/1858244546","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6819","nonce":701071143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6821","nonce":701071143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6825","nonce":701071143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6823","nonce":701071143}]},"public_addr":"172.21.15.90:6819/701071143","cluster_addr":"172.21.15.90:6821/701071143","heartbeat_back_addr":"172.21.15.90:6825/701071143","heartbeat_front_addr":"172.21.15.90:6823/701071143","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6801","nonce":4083464090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6803","nonce":4083464090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6807","nonce":4083464090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6805","nonce":4083464090}]},"public_addr":"172.21.15.149:6801/4083464090","cluster_addr":"172.21.15.149:6803/4083464090","heartbeat_back_addr":"172.21.15.149:6807/4083464090","heartbeat_front_addr":"172.21.15.149:6805/4083464090","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6809","nonce":3688072354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6811","nonce":3688072354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6815","nonce":3688072354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6813","nonce":3688072354}]},"public_addr":"172.21.15.149:6809/3688072354","cluster_addr":"172.21.15.149:6811/3688072354","heartbeat_back_addr":"172.21.15.149:6815/3688072354","heartbeat_front_addr":"172.21.15.149:6813/3688072354","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6817","nonce":1799649429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6819","nonce":1799649429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6823","nonce":1799649429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6821","nonce":1799649429}]},"public_addr":"172.21.15.149:6817/1799649429","cluster_addr":"172.21.15.149:6819/1799649429","heartbeat_back_addr":"172.21.15.149:6823/1799649429","heartbeat_front_addr":"172.21.15.149:6821/1799649429","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6826/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6825/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.149:6824/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3635708408":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/903772982":"2024-04-25T08:29:15.008628+0000","172.21.15.149:0/3314646633":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6826/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3128177159":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.149:6827/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6828/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.149:0/3752867478":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6827/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/2244430099":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.149:0/3222289279":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/2953349914":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/563018427":"2024-04-25T08:23:20.611807+0000","172.21.15.149:6829/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.149:0/683486625":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/182534323":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/78008447":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3406148041":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/152802986":"2024-04-25T08:29:15.008628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:39:28.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 msgr2=0x7f127c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 0x7f127c0711a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1284036040 tx=0x7f1284005fd0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 msgr2=0x7f12981951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f128800b420 tx=0x7f128800b8e0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 shutdown_connections 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f127c06ecf0 0x7f127c0711a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12980fef70 0x7f12981956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 --2- 172.21.15.90:0/1788589873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12980fe5d0 0x7f12981951b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 >> 172.21.15.90:0/1788589873 conn(0x7f1298076280 msgr2=0x7f1298102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 shutdown_connections 2024-04-24T08:39:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.063+0000 7f1297e00640 1 -- 172.21.15.90:0/1788589873 wait complete. 2024-04-24T08:39:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:16:48.618419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T08:20:06.040754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T08:20:06.663119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:39:28.085 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:39:28.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:27 smithi149 ceph-mon[66490]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.286+0000 7f2d6f200640 1 -- 172.21.15.90:0/2066258185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 msgr2=0x7f2d68101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.286+0000 7f2d6f200640 1 --2- 172.21.15.90:0/2066258185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68101b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009880 tx=0x7f2d5802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 -- 172.21.15.90:0/2066258185 shutdown_connections 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 --2- 172.21.15.90:0/2066258185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68101b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 --2- 172.21.15.90:0/2066258185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d68107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 -- 172.21.15.90:0/2066258185 >> 172.21.15.90:0/2066258185 conn(0x7f2d680fd1b0 msgr2=0x7f2d680ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 -- 172.21.15.90:0/2066258185 shutdown_connections 2024-04-24T08:39:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.287+0000 7f2d6f200640 1 -- 172.21.15.90:0/2066258185 wait complete. 2024-04-24T08:39:28.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 Processor -- start 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 -- start start 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d681978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68197f40 con 0x7f2d681016b0 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.288+0000 7f2d6f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819bd10 con 0x7f2d681076c0 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49610/0 (socket says 172.21.15.90:49610) 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 -- 172.21.15.90:0/651527284 learned_addr learned my addr 172.21.15.90:0/651527284 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6d400640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d681978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 -- 172.21.15.90:0/651527284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 msgr2=0x7f2d681978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d681978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 -- 172.21.15.90:0/651527284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58009560 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6d400640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d681978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.289+0000 7f2d6de00640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c8f0 tx=0x7f2d5c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.290+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5c007b80 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.290+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d5c007ce0 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.290+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6819bff0 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.290+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5c018870 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.290+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6819c550 con 0x7f2d681016b0 2024-04-24T08:39:28.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:27 smithi090 ceph-mon[85282]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:39:28.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.291+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6804e000 con 0x7f2d681016b0 2024-04-24T08:39:28.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.292+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2d5c0189d0 con 0x7f2d681016b0 2024-04-24T08:39:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.293+0000 7f2d62a00640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 0x7f2d500710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.293+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d5c08efe0 con 0x7f2d681016b0 2024-04-24T08:39:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.293+0000 7f2d6d400640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 0x7f2d500710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.294+0000 7f2d6d400640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 0x7f2d500710e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d58036040 tx=0x7f2d5802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.298+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2d5c057d60 con 0x7f2d681016b0 2024-04-24T08:39:28.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.491+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d681986c0 con 0x7f2d681016b0 2024-04-24T08:39:28.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.491+0000 7f2d62a00640 1 -- 172.21.15.90:0/651527284 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d5c05b420 con 0x7f2d681016b0 2024-04-24T08:39:28.492 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 msgr2=0x7f2d500710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 0x7f2d500710e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d58036040 tx=0x7f2d5802cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 msgr2=0x7f2d68197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c8f0 tx=0x7f2d5c00cdb0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 shutdown_connections 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f2d5006ec30 0x7f2d500710e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d681076c0 0x7f2d681978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 --2- 172.21.15.90:0/651527284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d681016b0 0x7f2d68197370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 >> 172.21.15.90:0/651527284 conn(0x7f2d680fd1b0 msgr2=0x7f2d680feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 shutdown_connections 2024-04-24T08:39:28.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.495+0000 7f2d6f200640 1 -- 172.21.15.90:0/651527284 wait complete. 2024-04-24T08:39:28.512 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 -- 172.21.15.90:0/2825758051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072f10 msgr2=0x7fef4c10b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 --2- 172.21.15.90:0/2825758051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072f10 0x7fef4c10b7b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef38009880 tx=0x7fef3802bd70 comp rx=0 tx=0).stop 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 -- 172.21.15.90:0/2825758051 shutdown_connections 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 --2- 172.21.15.90:0/2825758051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072f10 0x7fef4c10b7b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 --2- 172.21.15.90:0/2825758051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072600 0x7fef4c0729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 -- 172.21.15.90:0/2825758051 >> 172.21.15.90:0/2825758051 conn(0x7fef4c0fafb0 msgr2=0x7fef4c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 -- 172.21.15.90:0/2825758051 shutdown_connections 2024-04-24T08:39:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.684+0000 7fef51e00640 1 -- 172.21.15.90:0/2825758051 wait complete. 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 Processor -- start 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 -- start start 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 0x7fef4c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c197f80 con 0x7fef4c072600 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.685+0000 7fef51e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c19bd50 con 0x7fef4c072f10 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49626/0 (socket says 172.21.15.90:49626) 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 -- 172.21.15.90:0/557069268 learned_addr learned my addr 172.21.15.90:0/557069268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef4be00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 0x7fef4c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 -- 172.21.15.90:0/557069268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 msgr2=0x7fef4c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 0x7fef4c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 -- 172.21.15.90:0/557069268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009560 con 0x7fef4c072600 2024-04-24T08:39:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef4be00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 0x7fef4c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.686+0000 7fef50a00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e970 tx=0x7fef4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.687+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4000cc90 con 0x7fef4c072600 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.687+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef4000cdf0 con 0x7fef4c072600 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.687+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef40010430 con 0x7fef4c072600 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.687+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c19c030 con 0x7fef4c072600 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.687+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c19c540 con 0x7fef4c072600 2024-04-24T08:39:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.688+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c04e000 con 0x7fef4c072600 2024-04-24T08:39:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.690+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fef400046c0 con 0x7fef4c072600 2024-04-24T08:39:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.691+0000 7fef49600640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 0x7fef340710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.691+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fef40014070 con 0x7fef4c072600 2024-04-24T08:39:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.691+0000 7fef4be00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 0x7fef340710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.692+0000 7fef4be00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 0x7fef340710e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef4c1989c0 tx=0x7fef3802cbd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.700+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef40058200 con 0x7fef4c072600 2024-04-24T08:39:28.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.896+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fef4c108e20 con 0x7fef4c072600 2024-04-24T08:39:28.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.897+0000 7fef49600640 1 -- 172.21.15.90:0/557069268 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fef4005b8c0 con 0x7fef4c072600 2024-04-24T08:39:28.898 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.899+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 msgr2=0x7fef340710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.899+0000 7fef51e00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 0x7fef340710e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef4c1989c0 tx=0x7fef3802cbd0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.899+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 msgr2=0x7fef4c1973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.899+0000 7fef51e00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e970 tx=0x7fef4000ed70 comp rx=0 tx=0).stop 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 shutdown_connections 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef3406ec30 0x7fef340710e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef4c072f10 0x7fef4c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 --2- 172.21.15.90:0/557069268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef4c072600 0x7fef4c1973b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 >> 172.21.15.90:0/557069268 conn(0x7fef4c0fafb0 msgr2=0x7fef4c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 shutdown_connections 2024-04-24T08:39:28.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:28.900+0000 7fef51e00640 1 -- 172.21.15.90:0/557069268 wait complete. 2024-04-24T08:39:28.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T08:39:29.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 -- 172.21.15.90:0/2148430602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 msgr2=0x7fbdd8105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 --2- 172.21.15.90:0/2148430602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8105990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009880 tx=0x7fbdc802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 -- 172.21.15.90:0/2148430602 shutdown_connections 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 --2- 172.21.15.90:0/2148430602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8067760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 --2- 172.21.15.90:0/2148430602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8105990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 -- 172.21.15.90:0/2148430602 >> 172.21.15.90:0/2148430602 conn(0x7fbdd80743d0 msgr2=0x7fbdd80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 -- 172.21.15.90:0/2148430602 shutdown_connections 2024-04-24T08:39:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.114+0000 7fbddca00640 1 -- 172.21.15.90:0/2148430602 wait complete. 2024-04-24T08:39:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.115+0000 7fbddca00640 1 Processor -- start 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.115+0000 7fbddca00640 1 -- start start 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbddca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbddca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbddca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8197f10 con 0x7fbdd81055c0 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbddca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819bce0 con 0x7fbdd8067310 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49648/0 (socket says 172.21.15.90:49648) 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 -- 172.21.15.90:0/1512359499 learned_addr learned my addr 172.21.15.90:0/1512359499 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdd6800640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 -- 172.21.15.90:0/1512359499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 msgr2=0x7fbdd8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.116+0000 7fbdcfe00640 1 -- 172.21.15.90:0/1512359499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8009560 con 0x7fbdd81055c0 2024-04-24T08:39:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbdd6800640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:39:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbdcfe00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000d6f0 tx=0x7fbdc000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc0009a30 con 0x7fbdd81055c0 2024-04-24T08:39:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc00054e0 con 0x7fbdd81055c0 2024-04-24T08:39:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc0010460 con 0x7fbdd81055c0 2024-04-24T08:39:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd819bfc0 con 0x7fbdd81055c0 2024-04-24T08:39:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.117+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd819c520 con 0x7fbdd81055c0 2024-04-24T08:39:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.118+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd804e000 con 0x7fbdd81055c0 2024-04-24T08:39:29.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.120+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbdc0009b90 con 0x7fbdd81055c0 2024-04-24T08:39:29.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.121+0000 7fbdcf400640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 0x7fbdbc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.121+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbdc008e9a0 con 0x7fbdd81055c0 2024-04-24T08:39:29.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.121+0000 7fbdd6800640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 0x7fbdbc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.122+0000 7fbdd6800640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 0x7fbdbc0710e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:29.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.126+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbdc0057780 con 0x7fbdd81055c0 2024-04-24T08:39:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:28 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1788589873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:28 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:28 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/651527284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:29.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:28 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/557069268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:28 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1788589873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:28 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:28 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/651527284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:29.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:28 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/557069268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:29.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.324+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fbdd810c720 con 0x7fbdd81055c0 2024-04-24T08:39:29.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.325+0000 7fbdcf400640 1 -- 172.21.15.90:0/1512359499 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbdc0020020 con 0x7fbdd81055c0 2024-04-24T08:39:29.326 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:29.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 msgr2=0x7fbdbc0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 0x7fbdbc0710e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc802cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 msgr2=0x7fbdd8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000d6f0 tx=0x7fbdc000dbb0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 shutdown_connections 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fbdbc06ec30 0x7fbdbc0710e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdd81055c0 0x7fbdd8197880 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 --2- 172.21.15.90:0/1512359499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdd8067310 0x7fbdd8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 >> 172.21.15.90:0/1512359499 conn(0x7fbdd80743d0 msgr2=0x7fbdd8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.327+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 shutdown_connections 2024-04-24T08:39:29.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.328+0000 7fbddca00640 1 -- 172.21.15.90:0/1512359499 wait complete. 2024-04-24T08:39:29.346 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 -- 172.21.15.90:0/533110799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 msgr2=0x7f69c0109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 --2- 172.21.15.90:0/533110799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0109420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009880 tx=0x7f69a802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 -- 172.21.15.90:0/533110799 shutdown_connections 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 --2- 172.21.15.90:0/533110799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0109420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 --2- 172.21.15.90:0/533110799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 0x7f69c01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 -- 172.21.15.90:0/533110799 >> 172.21.15.90:0/533110799 conn(0x7f69c00f9e30 msgr2=0x7f69c00fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.530+0000 7f69c5e00640 1 -- 172.21.15.90:0/533110799 shutdown_connections 2024-04-24T08:39:29.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.531+0000 7f69c5e00640 1 -- 172.21.15.90:0/533110799 wait complete. 2024-04-24T08:39:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.531+0000 7f69c5e00640 1 Processor -- start 2024-04-24T08:39:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c5e00640 1 -- start start 2024-04-24T08:39:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 0x7f69c01972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c5e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0197eb0 con 0x7f69c0100f30 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c5e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c019bc80 con 0x7f69c0100620 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49668/0 (socket says 172.21.15.90:49668) 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69b7e00640 1 -- 172.21.15.90:0/3816573727 learned_addr learned my addr 172.21.15.90:0/3816573727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.532+0000 7f69c4a00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 0x7f69c01972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69b7e00640 1 -- 172.21.15.90:0/3816573727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 msgr2=0x7f69c01972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69b7e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 0x7f69c01972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69b7e00640 1 -- 172.21.15.90:0/3816573727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8009560 con 0x7f69c0100f30 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69b7e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f69a802c220 tx=0x7f69a802c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a80035f0 con 0x7f69c0100f30 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69a8003c10 con 0x7f69c0100f30 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a8005660 con 0x7f69c0100f30 2024-04-24T08:39:29.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c019bf00 con 0x7f69c0100f30 2024-04-24T08:39:29.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.533+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c019c460 con 0x7f69c0100f30 2024-04-24T08:39:29.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.535+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c004e850 con 0x7f69c0100f30 2024-04-24T08:39:29.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.535+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f69a8004ed0 con 0x7f69c0100f30 2024-04-24T08:39:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.536+0000 7f69be000640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 0x7f69a40711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.536+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f69a80b1a60 con 0x7f69c0100f30 2024-04-24T08:39:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.537+0000 7f69c4a00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 0x7f69a40711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.537+0000 7f69c4a00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 0x7f69a40711a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005fd0 tx=0x7f69b0005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:29.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.541+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f69a807a840 con 0x7f69c0100f30 2024-04-24T08:39:29.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.790+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69c0108e10 con 0x7f69c0100f30 2024-04-24T08:39:29.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.791+0000 7f69be000640 1 -- 172.21.15.90:0/3816573727 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5221 (secure 0 0 0) 0x7f69a8043090 con 0x7f69c0100f30 2024-04-24T08:39:29.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:29.792 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:37:25.249017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34494":{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34462,"qdb_cluster":[34462]},"id":1}]} 2024-04-24T08:39:29.792 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 38 2024-04-24T08:39:29.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 msgr2=0x7f69a40711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 0x7f69a40711a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005fd0 tx=0x7f69b0005850 comp rx=0 tx=0).stop 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 msgr2=0x7f69c0197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f69a802c220 tx=0x7f69a802c7c0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 shutdown_connections 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f69a406ecf0 0x7f69a40711a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c0100f30 0x7f69c0197820 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 --2- 172.21.15.90:0/3816573727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c0100620 0x7f69c01972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 >> 172.21.15.90:0/3816573727 conn(0x7f69c00f9e30 msgr2=0x7f69c0105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.793+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 shutdown_connections 2024-04-24T08:39:29.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:29.794+0000 7f69c5e00640 1 -- 172.21.15.90:0/3816573727 wait complete. 2024-04-24T08:39:29.811 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-04-24T08:39:29.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-24T08:39:30.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 -- 172.21.15.90:0/1691623258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81481054c0 msgr2=0x7f8148105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 --2- 172.21.15.90:0/1691623258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81481054c0 0x7f8148105890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8138009880 tx=0x7f813802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 -- 172.21.15.90:0/1691623258 shutdown_connections 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 --2- 172.21.15.90:0/1691623258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81480ff4b0 0x7f81480ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 --2- 172.21.15.90:0/1691623258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81481054c0 0x7f8148105890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.003+0000 7f8147e00640 1 -- 172.21.15.90:0/1691623258 >> 172.21.15.90:0/1691623258 conn(0x7f81480fafb0 msgr2=0x7f81480fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.004+0000 7f8147e00640 1 -- 172.21.15.90:0/1691623258 shutdown_connections 2024-04-24T08:39:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.004+0000 7f8147e00640 1 -- 172.21.15.90:0/1691623258 wait complete. 2024-04-24T08:39:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1512359499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:30.458 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:39:30.459 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:29 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3816573727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:39:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.004+0000 7f8147e00640 1 Processor -- start 2024-04-24T08:39:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8147e00640 1 -- start start 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8147e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8147e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 0x7f81481923d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8147e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148195fa0 con 0x7f81480ff4b0 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8147e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148192910 con 0x7f81481054c0 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8146a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8146a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35112/0 (socket says 172.21.15.90:35112) 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.005+0000 7f8146a00640 1 -- 172.21.15.90:0/821393553 learned_addr learned my addr 172.21.15.90:0/821393553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f813e000640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 0x7f81481923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8146a00640 1 -- 172.21.15.90:0/821393553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 msgr2=0x7f81481923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8146a00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 0x7f81481923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8146a00640 1 -- 172.21.15.90:0/821393553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138009560 con 0x7f81480ff4b0 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f813e000640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 0x7f81481923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8146a00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8138000c00 tx=0x7f8138004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81380037c0 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8138003de0 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148192b90 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8138034460 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.006+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81481a5b00 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.008+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814804e000 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.008+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8138003920 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.009+0000 7f813fe00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 0x7f812c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.010+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f81380b1390 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.010+0000 7f813e000640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 0x7f812c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.010+0000 7f813e000640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 0x7f812c0710e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8148074410 tx=0x7f8130005d90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.014+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f813807a190 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.261+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f81481933c0 con 0x7f81480ff4b0 2024-04-24T08:39:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.262+0000 7f813fe00640 1 -- 172.21.15.90:0/821393553 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v38) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f813807d850 con 0x7f81480ff4b0 2024-04-24T08:39:30.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:30.464 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14494,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/13627278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":13627278},{"type":"v1","addr":"172.21.15.90:6827","nonce":13627278}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 18 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 msgr2=0x7f812c0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 0x7f812c0710e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8148074410 tx=0x7f8130005d90 comp rx=0 tx=0).stop 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 msgr2=0x7f8148191e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8138000c00 tx=0x7f8138004270 comp rx=0 tx=0).stop 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 shutdown_connections 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f812c06ec30 0x7f812c0710e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81481054c0 0x7f81481923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 --2- 172.21.15.90:0/821393553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81480ff4b0 0x7f8148191e90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 >> 172.21.15.90:0/821393553 conn(0x7f81480fafb0 msgr2=0x7f8148072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 shutdown_connections 2024-04-24T08:39:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.265+0000 7f8147e00640 1 -- 172.21.15.90:0/821393553 wait complete. 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1512359499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:39:30.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:29 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3816573727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:39:30.467 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-04-24T08:39:30.467 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-24T08:39:30.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 -- 172.21.15.90:0/4060393957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d41054c0 msgr2=0x7f18d4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 --2- 172.21.15.90:0/4060393957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d41054c0 0x7f18d4105890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009c20 tx=0x7f18c402c1b0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 -- 172.21.15.90:0/4060393957 shutdown_connections 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 --2- 172.21.15.90:0/4060393957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d40ff4b0 0x7f18d40ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 --2- 172.21.15.90:0/4060393957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d41054c0 0x7f18d4105890 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 -- 172.21.15.90:0/4060393957 >> 172.21.15.90:0/4060393957 conn(0x7f18d40fafb0 msgr2=0x7f18d40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 -- 172.21.15.90:0/4060393957 shutdown_connections 2024-04-24T08:39:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.677+0000 7f18d3200640 1 -- 172.21.15.90:0/4060393957 wait complete. 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.678+0000 7f18d3200640 1 Processor -- start 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.678+0000 7f18d3200640 1 -- start start 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d41054c0 0x7f18d4072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4072fe0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4073150 con 0x7f18d41054c0 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35124/0 (socket says 172.21.15.90:35124) 2024-04-24T08:39:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 -- 172.21.15.90:0/204718767 learned_addr learned my addr 172.21.15.90:0/204718767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 -- 172.21.15.90:0/204718767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d41054c0 msgr2=0x7f18d4072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d41054c0 0x7f18d4072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.679+0000 7f18d1e00640 1 -- 172.21.15.90:0/204718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4009900 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18d1e00640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18c4037040 tx=0x7f18c4032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4034820 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18c4003e30 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c403f440 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d40733d0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.680+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41a15e0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.681+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d404e000 con 0x7f18d40ff4b0 2024-04-24T08:39:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.682+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f18c403b040 con 0x7f18d40ff4b0 2024-04-24T08:39:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.683+0000 7f18cb400640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 0x7f18b80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.683+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f18c40b1e30 con 0x7f18d40ff4b0 2024-04-24T08:39:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.684+0000 7f18c9400640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 0x7f18b80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:30.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.684+0000 7f18c9400640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 0x7f18b80710e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005fd0 tx=0x7f18bc005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.688+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f18c407abb0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.927+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f18d40733d0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.928+0000 7f18cb400640 1 -- 172.21.15.90:0/204718767 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v38) v1 ==== 107+0+4867 (secure 0 0 0) 0x7f18d40733d0 con 0x7f18d40ff4b0 2024-04-24T08:39:30.930 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:30.930 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14494,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/13627278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":13627278},{"type":"v1","addr":"172.21.15.90:6827","nonce":13627278}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34430,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4061836021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4061836021},{"type":"v1","addr":"172.21.15.90:6829","nonce":4061836021}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:30.930 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 msgr2=0x7f18b80710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 0x7f18b80710e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005fd0 tx=0x7f18bc005850 comp rx=0 tx=0).stop 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 msgr2=0x7f18d4072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18c4037040 tx=0x7f18c4032b60 comp rx=0 tx=0).stop 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 shutdown_connections 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f18b806ec30 0x7f18b80710e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18d41054c0 0x7f18d4072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 --2- 172.21.15.90:0/204718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18d40ff4b0 0x7f18d4072560 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 >> 172.21.15.90:0/204718767 conn(0x7f18d40fafb0 msgr2=0x7f18d40fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 shutdown_connections 2024-04-24T08:39:30.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:30.931+0000 7f18d3200640 1 -- 172.21.15.90:0/204718767 wait complete. 2024-04-24T08:39:30.951 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-04-24T08:39:30.951 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-24T08:39:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 -- 172.21.15.90:0/42371510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41016b0 msgr2=0x7f54d4101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 --2- 172.21.15.90:0/42371510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41016b0 0x7f54d4101b20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009880 tx=0x7f54c002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 -- 172.21.15.90:0/42371510 shutdown_connections 2024-04-24T08:39:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 --2- 172.21.15.90:0/42371510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41016b0 0x7f54d4101b20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 --2- 172.21.15.90:0/42371510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54d41076c0 0x7f54d4107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 -- 172.21.15.90:0/42371510 >> 172.21.15.90:0/42371510 conn(0x7f54d40fd1b0 msgr2=0x7f54d40ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 -- 172.21.15.90:0/42371510 shutdown_connections 2024-04-24T08:39:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.149+0000 7f54db000640 1 -- 172.21.15.90:0/42371510 wait complete. 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.150+0000 7f54db000640 1 Processor -- start 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.150+0000 7f54db000640 1 -- start start 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54db000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54d41016b0 0x7f54d4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54db000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54db000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4198010 con 0x7f54d41076c0 2024-04-24T08:39:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54db000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d419bde0 con 0x7f54d41016b0 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35136/0 (socket says 172.21.15.90:35136) 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 -- 172.21.15.90:0/562957753 learned_addr learned my addr 172.21.15.90:0/562957753 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 -- 172.21.15.90:0/562957753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54d41016b0 msgr2=0x7f54d4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54d41016b0 0x7f54d4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.151+0000 7f54d9200640 1 -- 172.21.15.90:0/562957753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c0009560 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54d9200640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c00037c0 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54c0003de0 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c0034460 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d419c060 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.152+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d419c5e0 con 0x7f54d41076c0 2024-04-24T08:39:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.153+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d404e000 con 0x7f54d41076c0 2024-04-24T08:39:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.154+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f54c0003920 con 0x7f54d41076c0 2024-04-24T08:39:31.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.156+0000 7f54d2000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 0x7f54bc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.156+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f54c00b1390 con 0x7f54d41076c0 2024-04-24T08:39:31.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.156+0000 7f54d9c00640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 0x7f54bc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.157+0000 7f54d9c00640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 0x7f54bc0710e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54d41027f0 tx=0x7f54c8006c90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.161+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f54c007a190 con 0x7f54d41076c0 2024-04-24T08:39:31.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:31.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:30 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/821393553' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T08:39:31.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:30 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/204718767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T08:39:31.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:31.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:30 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/821393553' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T08:39:31.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:30 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/204718767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T08:39:31.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.403+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f54d4198790 con 0x7f54d41076c0 2024-04-24T08:39:31.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.404+0000 7f54d2000640 1 -- 172.21.15.90:0/562957753 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+4094 (secure 0 0 0) 0x7f54c007d850 con 0x7f54d41076c0 2024-04-24T08:39:31.405 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:31.405 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34430,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4061836021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4061836021},{"type":"v1","addr":"172.21.15.90:6829","nonce":4061836021}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:31.405 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 20 2024-04-24T08:39:31.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 msgr2=0x7f54bc0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 0x7f54bc0710e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54d41027f0 tx=0x7f54c8006c90 comp rx=0 tx=0).stop 2024-04-24T08:39:31.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 msgr2=0x7f54d4197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0004270 comp rx=0 tx=0).stop 2024-04-24T08:39:31.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 shutdown_connections 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f54bc06ec30 0x7f54bc0710e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54d41076c0 0x7f54d4197980 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 --2- 172.21.15.90:0/562957753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54d41016b0 0x7f54d4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.406+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 >> 172.21.15.90:0/562957753 conn(0x7f54d40fd1b0 msgr2=0x7f54d40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.407+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 shutdown_connections 2024-04-24T08:39:31.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.407+0000 7f54db000640 1 -- 172.21.15.90:0/562957753 wait complete. 2024-04-24T08:39:31.424 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-04-24T08:39:31.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-24T08:39:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 -- 172.21.15.90:0/3519222370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401076c0 msgr2=0x7f8640107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 --2- 172.21.15.90:0/3519222370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401076c0 0x7f8640107a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8630009880 tx=0x7f863002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 -- 172.21.15.90:0/3519222370 shutdown_connections 2024-04-24T08:39:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 --2- 172.21.15.90:0/3519222370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86401016b0 0x7f8640101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 --2- 172.21.15.90:0/3519222370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401076c0 0x7f8640107a90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 -- 172.21.15.90:0/3519222370 >> 172.21.15.90:0/3519222370 conn(0x7f86400fd1b0 msgr2=0x7f86400ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:31.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 -- 172.21.15.90:0/3519222370 shutdown_connections 2024-04-24T08:39:31.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.625+0000 7f8646000640 1 -- 172.21.15.90:0/3519222370 wait complete. 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 Processor -- start 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 -- start start 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86401076c0 0x7f8640197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8640197ec0 con 0x7f86401016b0 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.626+0000 7f8646000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864019bc90 con 0x7f86401076c0 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35156/0 (socket says 172.21.15.90:35156) 2024-04-24T08:39:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 -- 172.21.15.90:0/1535398974 learned_addr learned my addr 172.21.15.90:0/1535398974 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 -- 172.21.15.90:0/1535398974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86401076c0 msgr2=0x7f8640197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86401076c0 0x7f8640197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 -- 172.21.15.90:0/1535398974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009560 con 0x7f86401016b0 2024-04-24T08:39:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f8644c00640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8630009230 tx=0x7f863002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.627+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86300035f0 con 0x7f86401016b0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.628+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8630003c10 con 0x7f86401016b0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.628+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864019bf10 con 0x7f86401016b0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.628+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8630005690 con 0x7f86401016b0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.628+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864019c490 con 0x7f86401016b0 2024-04-24T08:39:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.629+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864004e000 con 0x7f86401016b0 2024-04-24T08:39:31.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.630+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8630004ed0 con 0x7f86401016b0 2024-04-24T08:39:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.631+0000 7f863e000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 0x7f86240710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.631+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f86300b1380 con 0x7f86401016b0 2024-04-24T08:39:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.631+0000 7f8637e00640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 0x7f86240710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.632+0000 7f8637e00640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 0x7f86240710e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8628005fd0 tx=0x7f8628005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.636+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f863007a040 con 0x7f86401016b0 2024-04-24T08:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.862+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f864010e9c0 con 0x7f86401016b0 2024-04-24T08:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.863+0000 7f863e000640 1 -- 172.21.15.90:0/1535398974 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+4941 (secure 0 0 0) 0x7f863007d700 con 0x7f86401016b0 2024-04-24T08:39:31.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:31.864 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34430,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4061836021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4061836021},{"type":"v1","addr":"172.21.15.90:6829","nonce":4061836021}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-24T08:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.865+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 msgr2=0x7f86240710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.865+0000 7f8646000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 0x7f86240710e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8628005fd0 tx=0x7f8628005850 comp rx=0 tx=0).stop 2024-04-24T08:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.865+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 msgr2=0x7f86401972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.865+0000 7f8646000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8630009230 tx=0x7f863002c850 comp rx=0 tx=0).stop 2024-04-24T08:39:31.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 shutdown_connections 2024-04-24T08:39:31.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f862406ec30 0x7f86240710e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86401076c0 0x7f8640197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 --2- 172.21.15.90:0/1535398974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86401016b0 0x7f86401972f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 >> 172.21.15.90:0/1535398974 conn(0x7f86400fd1b0 msgr2=0x7f86400feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:31.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 shutdown_connections 2024-04-24T08:39:31.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:31.866+0000 7f8646000640 1 -- 172.21.15.90:0/1535398974 wait complete. 2024-04-24T08:39:31.885 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-04-24T08:39:31.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-24T08:39:32.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 -- 172.21.15.90:0/1460292535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 msgr2=0x7f9ed010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 --2- 172.21.15.90:0/1460292535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed010ff80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009880 tx=0x7f9eb802bd90 comp rx=0 tx=0).stop 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 -- 172.21.15.90:0/1460292535 shutdown_connections 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 --2- 172.21.15.90:0/1460292535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed010ff80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 --2- 172.21.15.90:0/1460292535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed00729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 -- 172.21.15.90:0/1460292535 >> 172.21.15.90:0/1460292535 conn(0x7f9ed00fafb0 msgr2=0x7f9ed00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 -- 172.21.15.90:0/1460292535 shutdown_connections 2024-04-24T08:39:32.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.074+0000 7f9ecf200640 1 -- 172.21.15.90:0/1460292535 wait complete. 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.075+0000 7f9ecf200640 1 Processor -- start 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecf200640 1 -- start start 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecf200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019c2a0 con 0x7f9ed0072f10 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019f000 con 0x7f9ed0072600 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35186/0 (socket says 172.21.15.90:35186) 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecd400640 1 -- 172.21.15.90:0/4200891986 learned_addr learned my addr 172.21.15.90:0/4200891986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.076+0000 7f9ecde00640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecd400640 1 -- 172.21.15.90:0/4200891986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 msgr2=0x7f9ed019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecd400640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecd400640 1 -- 172.21.15.90:0/4200891986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8009560 con 0x7f9ed0072f10 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecde00640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecd400640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8036040 tx=0x7f9eb8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb80035f0 con 0x7f9ed0072f10 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9eb8003c10 con 0x7f9ed0072f10 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed019f280 con 0x7f9ed0072f10 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb803e450 con 0x7f9ed0072f10 2024-04-24T08:39:32.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.077+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed019f7e0 con 0x7f9ed0072f10 2024-04-24T08:39:32.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.078+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed004e000 con 0x7f9ed0072f10 2024-04-24T08:39:32.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.079+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9eb8004fd0 con 0x7f9ed0072f10 2024-04-24T08:39:32.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.080+0000 7f9ec6a00640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 0x7f9eb40710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.080+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9eb80b1b00 con 0x7f9ed0072f10 2024-04-24T08:39:32.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.081+0000 7f9ecde00640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 0x7f9eb40710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.081+0000 7f9ecde00640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 0x7f9eb40710e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0008e00 tx=0x7f9ec0008350 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.085+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9eb807a860 con 0x7f9ed0072f10 2024-04-24T08:39:32.272 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:31 smithi090 ceph-mon[85282]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:32.272 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:31 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/562957753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T08:39:32.272 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:31 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1535398974' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T08:39:32.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:31 smithi149 ceph-mon[66490]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:32.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:31 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/562957753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T08:39:32.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:31 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1535398974' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T08:39:32.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.336+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f9ed0112c20 con 0x7f9ed0072f10 2024-04-24T08:39:32.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.337+0000 7f9ec6a00640 1 -- 172.21.15.90:0/4200891986 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+4162 (secure 0 0 0) 0x7f9eb807df20 con 0x7f9ed0072f10 2024-04-24T08:39:32.338 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:32.339 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:32.339 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 22 2024-04-24T08:39:32.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 msgr2=0x7f9eb40710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 0x7f9eb40710e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0008e00 tx=0x7f9ec0008350 comp rx=0 tx=0).stop 2024-04-24T08:39:32.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 msgr2=0x7f9ed019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8036040 tx=0x7f9eb8004040 comp rx=0 tx=0).stop 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 shutdown_connections 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9eb406ec30 0x7f9eb40710e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ed0072f10 0x7f9ed019bc50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 --2- 172.21.15.90:0/4200891986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0072600 0x7f9ed019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 >> 172.21.15.90:0/4200891986 conn(0x7f9ed00fafb0 msgr2=0x7f9ed0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 shutdown_connections 2024-04-24T08:39:32.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.340+0000 7f9ecf200640 1 -- 172.21.15.90:0/4200891986 wait complete. 2024-04-24T08:39:32.361 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-04-24T08:39:32.362 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-24T08:39:32.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 -- 172.21.15.90:0/1360233326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67540699a0 msgr2=0x7f6754069df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 --2- 172.21.15.90:0/1360233326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67540699a0 0x7f6754069df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f67400098b0 tx=0x7f674002bcd0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 -- 172.21.15.90:0/1360233326 shutdown_connections 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 --2- 172.21.15.90:0/1360233326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67540699a0 0x7f6754069df0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 --2- 172.21.15.90:0/1360233326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f675406b1f0 0x7f6754069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 -- 172.21.15.90:0/1360233326 >> 172.21.15.90:0/1360233326 conn(0x7f6754071d10 msgr2=0x7f675406e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 -- 172.21.15.90:0/1360233326 shutdown_connections 2024-04-24T08:39:32.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.555+0000 7f675c400640 1 -- 172.21.15.90:0/1360233326 wait complete. 2024-04-24T08:39:32.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.556+0000 7f675c400640 1 Processor -- start 2024-04-24T08:39:32.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.556+0000 7f675c400640 1 -- start start 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f675c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67540699a0 0x7f675419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f675c400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f675c400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541a0860 con 0x7f675406b1f0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f675c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541a09d0 con 0x7f67540699a0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35210/0 (socket says 172.21.15.90:35210) 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 -- 172.21.15.90:0/1663494648 learned_addr learned my addr 172.21.15.90:0/1663494648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759a00640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67540699a0 0x7f675419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 -- 172.21.15.90:0/1663494648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67540699a0 msgr2=0x7f675419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67540699a0 0x7f675419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.557+0000 7f6759000640 1 -- 172.21.15.90:0/1663494648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740009560 con 0x7f675406b1f0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f6759000640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6740009880 tx=0x7f6740004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:32.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67400037c0 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6740003de0 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67541a4800 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6740034440 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.558+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67541a4d00 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.559+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675404e000 con 0x7f675406b1f0 2024-04-24T08:39:32.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.560+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6740004fd0 con 0x7f675406b1f0 2024-04-24T08:39:32.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.562+0000 7f6752000640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 0x7f673c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:32.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.562+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f67400b1470 con 0x7f675406b1f0 2024-04-24T08:39:32.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.562+0000 7f6759a00640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 0x7f673c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:32.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.562+0000 7f6759a00640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 0x7f673c0710e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f674800ba40 tx=0x7f6748005cc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:32.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.566+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f674007a130 con 0x7f675406b1f0 2024-04-24T08:39:32.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.808+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f6754117300 con 0x7f675406b1f0 2024-04-24T08:39:32.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.809+0000 7f6752000640 1 -- 172.21.15.90:0/1663494648 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f674007d7f0 con 0x7f675406b1f0 2024-04-24T08:39:32.810 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:32.810 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:00.123022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24223":{"gid":24223,"name":"cephfs.smithi149.eazcmc","rank":0,"incarnation":13,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6825/1555648344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1555648344},{"type":"v1","addr":"172.21.15.149:6825","nonce":1555648344}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24223,"qdb_cluster":[24223]},"id":1}]} 2024-04-24T08:39:32.810 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 23 2024-04-24T08:39:32.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 msgr2=0x7f673c0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 0x7f673c0710e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f674800ba40 tx=0x7f6748005cc0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 msgr2=0x7f67541a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6740009880 tx=0x7f6740004270 comp rx=0 tx=0).stop 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 shutdown_connections 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f673c06ec30 0x7f673c0710e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f675406b1f0 0x7f67541a0260 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 --2- 172.21.15.90:0/1663494648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67540699a0 0x7f675419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.811+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 >> 172.21.15.90:0/1663494648 conn(0x7f6754071d10 msgr2=0x7f6754112b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.812+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 shutdown_connections 2024-04-24T08:39:32.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:32.812+0000 7f675c400640 1 -- 172.21.15.90:0/1663494648 wait complete. 2024-04-24T08:39:32.831 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-04-24T08:39:32.832 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 -- 172.21.15.90:0/2347531276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 msgr2=0x7f3dfc107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 --2- 172.21.15.90:0/2347531276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc107a90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009880 tx=0x7f3df002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 -- 172.21.15.90:0/2347531276 shutdown_connections 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 --2- 172.21.15.90:0/2347531276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 0x7f3dfc101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 --2- 172.21.15.90:0/2347531276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc107a90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 -- 172.21.15.90:0/2347531276 >> 172.21.15.90:0/2347531276 conn(0x7f3dfc0fd1b0 msgr2=0x7f3dfc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:33.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 -- 172.21.15.90:0/2347531276 shutdown_connections 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.035+0000 7f3e02a00640 1 -- 172.21.15.90:0/2347531276 wait complete. 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 Processor -- start 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 -- start start 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 0x7f3dfc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc197ef0 con 0x7f3dfc1076c0 2024-04-24T08:39:33.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.036+0000 7f3e02a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc19bcc0 con 0x7f3dfc1016b0 2024-04-24T08:39:33.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35230/0 (socket says 172.21.15.90:35230) 2024-04-24T08:39:33.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 -- 172.21.15.90:0/1307483074 learned_addr learned my addr 172.21.15.90:0/1307483074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:33.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 -- 172.21.15.90:0/1307483074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 msgr2=0x7f3dfc197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e01600640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 0x7f3dfc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 0x7f3dfc197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 -- 172.21.15.90:0/1307483074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0009560 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.037+0000 7f3e00c00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d6f0 tx=0x7f3de800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.038+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de8009a30 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.038+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3de80054e0 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.038+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de8010460 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.038+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc19bfa0 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.038+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc19c580 con 0x7f3dfc1076c0 2024-04-24T08:39:33.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.039+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc04e000 con 0x7f3dfc1076c0 2024-04-24T08:39:33.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.040+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3de8009b90 con 0x7f3dfc1076c0 2024-04-24T08:39:33.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.041+0000 7f3dfa000640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 0x7f3de40710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.041+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3de808ea40 con 0x7f3dfc1076c0 2024-04-24T08:39:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.042+0000 7f3e01600640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 0x7f3de40710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.042+0000 7f3e01600640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 0x7f3de40710e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3df0036040 tx=0x7f3df002cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:33.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.046+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3de8057780 con 0x7f3dfc1076c0 2024-04-24T08:39:33.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:32 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/4200891986' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T08:39:33.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:32 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1663494648' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T08:39:33.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:32 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/4200891986' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T08:39:33.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:32 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1663494648' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T08:39:33.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.294+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f3dfc198670 con 0x7f3dfc1076c0 2024-04-24T08:39:33.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.296+0000 7f3dfa000640 1 -- 172.21.15.90:0/1307483074 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+4208 (secure 0 0 0) 0x7f3de8017090 con 0x7f3dfc1076c0 2024-04-24T08:39:33.297 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:33.297 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:59.007303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:33.297 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 24 2024-04-24T08:39:33.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 msgr2=0x7f3de40710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 0x7f3de40710e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3df0036040 tx=0x7f3df002cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 msgr2=0x7f3dfc197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d6f0 tx=0x7f3de800dbb0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 shutdown_connections 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3de406ec30 0x7f3de40710e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1076c0 0x7f3dfc197860 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 --2- 172.21.15.90:0/1307483074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dfc1016b0 0x7f3dfc197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 >> 172.21.15.90:0/1307483074 conn(0x7f3dfc0fd1b0 msgr2=0x7f3dfc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 shutdown_connections 2024-04-24T08:39:33.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.299+0000 7f3e02a00640 1 -- 172.21.15.90:0/1307483074 wait complete. 2024-04-24T08:39:33.320 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-04-24T08:39:33.320 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-24T08:39:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- 172.21.15.90:0/2759210385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072e90 msgr2=0x7f6fd410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 --2- 172.21.15.90:0/2759210385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072e90 0x7f6fd410bbf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009880 tx=0x7f6fc002bd70 comp rx=0 tx=0).stop 2024-04-24T08:39:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- 172.21.15.90:0/2759210385 shutdown_connections 2024-04-24T08:39:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 --2- 172.21.15.90:0/2759210385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072e90 0x7f6fd410bbf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 --2- 172.21.15.90:0/2759210385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072580 0x7f6fd4072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- 172.21.15.90:0/2759210385 >> 172.21.15.90:0/2759210385 conn(0x7f6fd40fb370 msgr2=0x7f6fd40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:33.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- 172.21.15.90:0/2759210385 shutdown_connections 2024-04-24T08:39:33.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- 172.21.15.90:0/2759210385 wait complete. 2024-04-24T08:39:33.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 Processor -- start 2024-04-24T08:39:33.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- start start 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072e90 0x7f6fd419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41975b0 con 0x7f6fd4072580 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.526+0000 7f6fd9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4197720 con 0x7f6fd4072e90 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35240/0 (socket says 172.21.15.90:35240) 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 -- 172.21.15.90:0/3170148992 learned_addr learned my addr 172.21.15.90:0/3170148992 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 -- 172.21.15.90:0/3170148992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072e90 msgr2=0x7f6fd419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2000640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072e90 0x7f6fd419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072e90 0x7f6fd419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 -- 172.21.15.90:0/3170148992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0009560 con 0x7f6fd4072580 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fd2a00640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d6f0 tx=0x7f6fc400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.527+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc4009a30 con 0x7f6fd4072580 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.528+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fc40054e0 con 0x7f6fd4072580 2024-04-24T08:39:33.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.528+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd4197a00 con 0x7f6fd4072580 2024-04-24T08:39:33.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.528+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc4010460 con 0x7f6fd4072580 2024-04-24T08:39:33.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.528+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd4197f60 con 0x7f6fd4072580 2024-04-24T08:39:33.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.529+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd404e000 con 0x7f6fd4072580 2024-04-24T08:39:33.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.530+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6fc4009b90 con 0x7f6fd4072580 2024-04-24T08:39:33.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.531+0000 7f6fcb400640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 0x7f6fb80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:33.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.531+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6fc408eb00 con 0x7f6fd4072580 2024-04-24T08:39:33.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.531+0000 7f6fd2000640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 0x7f6fb80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:33.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.532+0000 7f6fd2000640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 0x7f6fb80710e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0036040 tx=0x7f6fc0005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:33.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.536+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6fc40577c0 con 0x7f6fd4072580 2024-04-24T08:39:33.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.780+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f6fd4198ac0 con 0x7f6fd4072580 2024-04-24T08:39:33.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.781+0000 7f6fcb400640 1 -- 172.21.15.90:0/3170148992 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4219 (secure 0 0 0) 0x7f6fc4017090 con 0x7f6fd4072580 2024-04-24T08:39:33.782 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:33.782 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:35:59.019790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24237":{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:33.782 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-24T08:39:33.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 msgr2=0x7f6fb80710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 0x7f6fb80710e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0036040 tx=0x7f6fc0005fd0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 msgr2=0x7f6fd419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:33.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d6f0 tx=0x7f6fc400dbb0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 shutdown_connections 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f6fb806ec30 0x7f6fb80710e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4072e90 0x7f6fd419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 --2- 172.21.15.90:0/3170148992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd4072580 0x7f6fd419d540 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 >> 172.21.15.90:0/3170148992 conn(0x7f6fd40fb370 msgr2=0x7f6fd4106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 shutdown_connections 2024-04-24T08:39:33.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:33.783+0000 7f6fd9600640 1 -- 172.21.15.90:0/3170148992 wait complete. 2024-04-24T08:39:33.803 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-04-24T08:39:33.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-24T08:39:34.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 -- 172.21.15.90:0/486526977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 msgr2=0x7f29401064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 --2- 172.21.15.90:0/486526977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401064e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2928009880 tx=0x7f292802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 -- 172.21.15.90:0/486526977 shutdown_connections 2024-04-24T08:39:34.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 --2- 172.21.15.90:0/486526977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401064e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 --2- 172.21.15.90:0/486526977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 0x7f294010bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 -- 172.21.15.90:0/486526977 >> 172.21.15.90:0/486526977 conn(0x7f2940075260 msgr2=0x7f2940077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:34.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 -- 172.21.15.90:0/486526977 shutdown_connections 2024-04-24T08:39:34.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.007+0000 7f2945e00640 1 -- 172.21.15.90:0/486526977 wait complete. 2024-04-24T08:39:34.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.008+0000 7f2945e00640 1 Processor -- start 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.008+0000 7f2945e00640 1 -- start start 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2945e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2945e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 0x7f29401a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2945e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294019b5c0 con 0x7f2940106090 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2945e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294019b700 con 0x7f294010bb60 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2944a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2944a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35256/0 (socket says 172.21.15.90:35256) 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2944a00640 1 -- 172.21.15.90:0/837070516 learned_addr learned my addr 172.21.15.90:0/837070516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.009+0000 7f2937e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 0x7f29401a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2944a00640 1 -- 172.21.15.90:0/837070516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 msgr2=0x7f29401a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2944a00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 0x7f29401a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2944a00640 1 -- 172.21.15.90:0/837070516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928009560 con 0x7f2940106090 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2944a00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f293000d8d0 tx=0x7f293000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f293000bea0 con 0x7f2940106090 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2930009c00 con 0x7f2940106090 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294019b9e0 con 0x7f2940106090 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f293000f710 con 0x7f2940106090 2024-04-24T08:39:34.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.010+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294019bf40 con 0x7f2940106090 2024-04-24T08:39:34.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.012+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294004e000 con 0x7f2940106090 2024-04-24T08:39:34.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.013+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f293000f990 con 0x7f2940106090 2024-04-24T08:39:34.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.014+0000 7f293e000640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 0x7f29240710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.014+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f293008eca0 con 0x7f2940106090 2024-04-24T08:39:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.014+0000 7f2937e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 0x7f29240710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.015+0000 7f2937e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 0x7f29240710e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f294019cef0 tx=0x7f292802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.019+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2930057960 con 0x7f2940106090 2024-04-24T08:39:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.266+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f2940112c70 con 0x7f2940106090 2024-04-24T08:39:34.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.267+0000 7f293e000640 1 -- 172.21.15.90:0/837070516 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+4224 (secure 0 0 0) 0x7f293005b020 con 0x7f2940106090 2024-04-24T08:39:34.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:34.268 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:08.621894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24237":{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":0,"incarnation":25,"state":"up:reconnect","state_seq":237,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:34.268 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 msgr2=0x7f29240710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 0x7f29240710e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f294019cef0 tx=0x7f292802cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 msgr2=0x7f29401a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f293000d8d0 tx=0x7f293000dd90 comp rx=0 tx=0).stop 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 shutdown_connections 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f292406ec30 0x7f29240710e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294010bb60 0x7f29401a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 --2- 172.21.15.90:0/837070516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940106090 0x7f29401a15e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 >> 172.21.15.90:0/837070516 conn(0x7f2940075260 msgr2=0x7f2940076cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:34.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 shutdown_connections 2024-04-24T08:39:34.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.271+0000 7f2945e00640 1 -- 172.21.15.90:0/837070516 wait complete. 2024-04-24T08:39:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:33 smithi149 ceph-mon[66490]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:33 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1307483074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T08:39:34.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:33 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3170148992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T08:39:34.287 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:33 smithi090 ceph-mon[85282]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:34.287 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:33 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1307483074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T08:39:34.287 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:33 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3170148992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T08:39:34.290 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-24T08:39:34.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-24T08:39:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 -- 172.21.15.90:0/3125187925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072f10 msgr2=0x7f00b010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 --2- 172.21.15.90:0/3125187925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072f10 0x7f00b010ff80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f009c009880 tx=0x7f009c02bd70 comp rx=0 tx=0).stop 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 -- 172.21.15.90:0/3125187925 shutdown_connections 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 --2- 172.21.15.90:0/3125187925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072f10 0x7f00b010ff80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 --2- 172.21.15.90:0/3125187925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072600 0x7f00b00729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 -- 172.21.15.90:0/3125187925 >> 172.21.15.90:0/3125187925 conn(0x7f00b00fafb0 msgr2=0x7f00b00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 -- 172.21.15.90:0/3125187925 shutdown_connections 2024-04-24T08:39:34.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.492+0000 7f00b5600640 1 -- 172.21.15.90:0/3125187925 wait complete. 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.493+0000 7f00b5600640 1 Processor -- start 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.493+0000 7f00b5600640 1 -- start start 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.493+0000 7f00b5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00b5600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072f10 0x7f00b01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00b5600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b019b570 con 0x7f00b0072600 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00b5600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b019b6e0 con 0x7f00b0072f10 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35266/0 (socket says 172.21.15.90:35266) 2024-04-24T08:39:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 -- 172.21.15.90:0/3126780009 learned_addr learned my addr 172.21.15.90:0/3126780009 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00af400640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072f10 0x7f00b01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 -- 172.21.15.90:0/3126780009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072f10 msgr2=0x7f00b01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072f10 0x7f00b01a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.494+0000 7f00afe00640 1 -- 172.21.15.90:0/3126780009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c009560 con 0x7f00b0072600 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00afe00640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f00a000d6f0 tx=0x7f00a000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a0009a30 con 0x7f00b0072600 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00a00054e0 con 0x7f00b0072600 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b019b9c0 con 0x7f00b0072600 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a0010460 con 0x7f00b0072600 2024-04-24T08:39:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.495+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b019bf20 con 0x7f00b0072600 2024-04-24T08:39:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.496+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b004e000 con 0x7f00b0072600 2024-04-24T08:39:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.497+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f00a0009b90 con 0x7f00b0072600 2024-04-24T08:39:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.498+0000 7f00acc00640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 0x7f00940710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.498+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f00a008ea60 con 0x7f00b0072600 2024-04-24T08:39:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.498+0000 7f00af400640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 0x7f00940710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.499+0000 7f00af400640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 0x7f00940710e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f009c036040 tx=0x7f009c000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.504+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f00a00577c0 con 0x7f00b0072600 2024-04-24T08:39:34.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.747+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f00b010d670 con 0x7f00b0072600 2024-04-24T08:39:34.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.748+0000 7f00acc00640 1 -- 172.21.15.90:0/3126780009 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+4221 (secure 0 0 0) 0x7f00a0017090 con 0x7f00b0072600 2024-04-24T08:39:34.748 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:34.748 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:09.273268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24237":{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":0,"incarnation":25,"state":"up:rejoin","state_seq":238,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:34.749 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T08:39:34.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 msgr2=0x7f00940710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 0x7f00940710e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f009c036040 tx=0x7f009c000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 msgr2=0x7f00b01a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f00a000d6f0 tx=0x7f00a000dbb0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 shutdown_connections 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f009406ec30 0x7f00940710e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00b0072f10 0x7f00b01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 --2- 172.21.15.90:0/3126780009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00b0072600 0x7f00b01a1500 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 >> 172.21.15.90:0/3126780009 conn(0x7f00b00fafb0 msgr2=0x7f00b0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 shutdown_connections 2024-04-24T08:39:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.751+0000 7f00b5600640 1 -- 172.21.15.90:0/3126780009 wait complete. 2024-04-24T08:39:34.769 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-24T08:39:34.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-24T08:39:34.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 -- 172.21.15.90:0/173752059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 msgr2=0x7f78cc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 --2- 172.21.15.90:0/173752059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc101d00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009880 tx=0x7f78b802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 -- 172.21.15.90:0/173752059 shutdown_connections 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 --2- 172.21.15.90:0/173752059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc101d00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 --2- 172.21.15.90:0/173752059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 -- 172.21.15.90:0/173752059 >> 172.21.15.90:0/173752059 conn(0x7f78cc0fd390 msgr2=0x7f78cc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 -- 172.21.15.90:0/173752059 shutdown_connections 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.970+0000 7f78d2000640 1 -- 172.21.15.90:0/173752059 wait complete. 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 Processor -- start 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 -- start start 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1980e0 con 0x7f78cc101890 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.971+0000 7f78d2000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc19beb0 con 0x7f78cc1078a0 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35280/0 (socket says 172.21.15.90:35280) 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 -- 172.21.15.90:0/3746755121 learned_addr learned my addr 172.21.15.90:0/3746755121 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78caa00640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 -- 172.21.15.90:0/3746755121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 msgr2=0x7f78cc197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78cb400640 1 -- 172.21.15.90:0/3746755121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8009560 con 0x7f78cc101890 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.972+0000 7f78caa00640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78cb400640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d8d0 tx=0x7f78bc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78bc009a30 con 0x7f78cc101890 2024-04-24T08:39:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78bc009b90 con 0x7f78cc101890 2024-04-24T08:39:34.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc19c190 con 0x7f78cc101890 2024-04-24T08:39:34.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78bc00f680 con 0x7f78cc101890 2024-04-24T08:39:34.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.973+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc19c770 con 0x7f78cc101890 2024-04-24T08:39:34.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.974+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc04e000 con 0x7f78cc101890 2024-04-24T08:39:34.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.975+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f78bc009d00 con 0x7f78cc101890 2024-04-24T08:39:34.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.976+0000 7f78c3e00640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 0x7f78b00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:34.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.976+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f78bc08edc0 con 0x7f78cc101890 2024-04-24T08:39:34.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.976+0000 7f78caa00640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 0x7f78b00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:34.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.977+0000 7f78caa00640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 0x7f78b00710e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78b8036040 tx=0x7f78b802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:34.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:34.981+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f78bc092030 con 0x7f78cc101890 2024-04-24T08:39:35.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:34 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/837070516' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T08:39:35.043 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:34 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3126780009' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T08:39:35.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.223+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f78cc198860 con 0x7f78cc101890 2024-04-24T08:39:35.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.225+0000 7f78c3e00640 1 -- 172.21.15.90:0/3746755121 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+5080 (secure 0 0 0) 0x7f78bc057a80 con 0x7f78cc101890 2024-04-24T08:39:35.228 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:35.228 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:11.278979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24237":{"gid":24237,"name":"cephfs.smithi149.zwjtmw","rank":0,"incarnation":25,"state":"up:active","state_seq":239,"addr":"172.21.15.149:6827/2080624312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2080624312},{"type":"v1","addr":"172.21.15.149:6827","nonce":2080624312}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24237,"qdb_cluster":[24237]},"id":1}]} 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 28 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.227+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 msgr2=0x7f78b00710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.227+0000 7f78d2000640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 0x7f78b00710e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78b8036040 tx=0x7f78b802cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.227+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 msgr2=0x7f78cc197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.227+0000 7f78d2000640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d8d0 tx=0x7f78bc00dd90 comp rx=0 tx=0).stop 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 shutdown_connections 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f78b006ec30 0x7f78b00710e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78cc1078a0 0x7f78cc197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 --2- 172.21.15.90:0/3746755121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78cc101890 0x7f78cc197510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 >> 172.21.15.90:0/3746755121 conn(0x7f78cc0fd390 msgr2=0x7f78cc0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:35.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 shutdown_connections 2024-04-24T08:39:35.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.228+0000 7f78d2000640 1 -- 172.21.15.90:0/3746755121 wait complete. 2024-04-24T08:39:35.250 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-24T08:39:35.251 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-24T08:39:35.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:34 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/837070516' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T08:39:35.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:34 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3126780009' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T08:39:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 -- 172.21.15.90:0/2671635048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 msgr2=0x7f25a0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 --2- 172.21.15.90:0/2671635048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0105990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2590009880 tx=0x7f259002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 -- 172.21.15.90:0/2671635048 shutdown_connections 2024-04-24T08:39:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 --2- 172.21.15.90:0/2671635048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a0067310 0x7f25a0067760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 --2- 172.21.15.90:0/2671635048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0105990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 -- 172.21.15.90:0/2671635048 >> 172.21.15.90:0/2671635048 conn(0x7f25a00743b0 msgr2=0x7f25a00747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:35.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 -- 172.21.15.90:0/2671635048 shutdown_connections 2024-04-24T08:39:35.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.467+0000 7f25a6000640 1 -- 172.21.15.90:0/2671635048 wait complete. 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 Processor -- start 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 -- start start 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a0067310 0x7f25a0195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0195dd0 con 0x7f25a01055c0 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.468+0000 7f25a6000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0199ba0 con 0x7f25a0067310 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35298/0 (socket says 172.21.15.90:35298) 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 -- 172.21.15.90:0/940054069 learned_addr learned my addr 172.21.15.90:0/940054069 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:35.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 -- 172.21.15.90:0/940054069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a0067310 msgr2=0x7f25a0195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a0067310 0x7f25a0195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 -- 172.21.15.90:0/940054069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009560 con 0x7f25a01055c0 2024-04-24T08:39:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.469+0000 7f259fe00640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d6f0 tx=0x7f258c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.470+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c009a30 con 0x7f25a01055c0 2024-04-24T08:39:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.470+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f258c0054e0 con 0x7f25a01055c0 2024-04-24T08:39:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.470+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0199e80 con 0x7f25a01055c0 2024-04-24T08:39:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.470+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c010460 con 0x7f25a01055c0 2024-04-24T08:39:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.470+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a019a3e0 con 0x7f25a01055c0 2024-04-24T08:39:35.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.471+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a004e850 con 0x7f25a01055c0 2024-04-24T08:39:35.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.472+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f258c009b90 con 0x7f25a01055c0 2024-04-24T08:39:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.473+0000 7f259d600640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 0x7f25840710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.473+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f258c08e9a0 con 0x7f25a01055c0 2024-04-24T08:39:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.473+0000 7f25a4c00640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 0x7f25840710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.474+0000 7f25a4c00640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 0x7f25840710e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2590036040 tx=0x7f2590000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.478+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f258c057780 con 0x7f25a01055c0 2024-04-24T08:39:35.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.727+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f25a0107990 con 0x7f25a01055c0 2024-04-24T08:39:35.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.728+0000 7f259d600640 1 -- 172.21.15.90:0/940054069 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+4276 (secure 0 0 0) 0x7f258c017090 con 0x7f25a01055c0 2024-04-24T08:39:35.729 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:35.729 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:30.490623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:35.729 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-04-24T08:39:35.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 msgr2=0x7f25840710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 0x7f25840710e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2590036040 tx=0x7f2590000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 msgr2=0x7f25a0195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d6f0 tx=0x7f258c00dbb0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 shutdown_connections 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f258406ec30 0x7f25840710e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25a01055c0 0x7f25a0195740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 --2- 172.21.15.90:0/940054069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a0067310 0x7f25a0195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 >> 172.21.15.90:0/940054069 conn(0x7f25a00743b0 msgr2=0x7f25a00fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 shutdown_connections 2024-04-24T08:39:35.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.731+0000 7f25a6000640 1 -- 172.21.15.90:0/940054069 wait complete. 2024-04-24T08:39:35.751 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-24T08:39:35.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3938691574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 msgr2=0x7ff0d410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3938691574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d410ff80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009880 tx=0x7ff0bc02bd70 comp rx=0 tx=0).stop 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3938691574 shutdown_connections 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3938691574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d410ff80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3938691574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 0x7ff0d40729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.956+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3938691574 >> 172.21.15.90:0/3938691574 conn(0x7ff0d40fafb0 msgr2=0x7ff0d40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.957+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3938691574 shutdown_connections 2024-04-24T08:39:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.957+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3938691574 wait complete. 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.957+0000 7ff0d9a00640 1 Processor -- start 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d9a00640 1 -- start start 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 0x7ff0d41a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d9a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d419b570 con 0x7ff0d4072f10 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d419b6e0 con 0x7ff0d4072600 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:35.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35318/0 (socket says 172.21.15.90:35318) 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.958+0000 7ff0d3400640 1 -- 172.21.15.90:0/3767309046 learned_addr learned my addr 172.21.15.90:0/3767309046 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d3e00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 0x7ff0d41a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d3400640 1 -- 172.21.15.90:0/3767309046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 msgr2=0x7ff0d41a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d3400640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 0x7ff0d41a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d3400640 1 -- 172.21.15.90:0/3767309046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc009560 con 0x7ff0d4072f10 2024-04-24T08:39:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d3400640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc036040 tx=0x7ff0bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc0035f0 con 0x7ff0d4072f10 2024-04-24T08:39:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0bc003c10 con 0x7ff0d4072f10 2024-04-24T08:39:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d419b960 con 0x7ff0d4072f10 2024-04-24T08:39:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.959+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc005690 con 0x7ff0d4072f10 2024-04-24T08:39:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.960+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d419bec0 con 0x7ff0d4072f10 2024-04-24T08:39:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.961+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d404e000 con 0x7ff0d4072f10 2024-04-24T08:39:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.962+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff0bc004ed0 con 0x7ff0d4072f10 2024-04-24T08:39:35.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.963+0000 7ff0d0c00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 0x7ff0b80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:35.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.963+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff0bc0b1990 con 0x7ff0d4072f10 2024-04-24T08:39:35.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.964+0000 7ff0d3e00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 0x7ff0b80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:35.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.964+0000 7ff0d3e00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 0x7ff0b80710e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4005fd0 tx=0x7ff0c4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:35.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:35.969+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff0bc07a770 con 0x7ff0d4072f10 2024-04-24T08:39:36.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:35 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3746755121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T08:39:36.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:35 smithi090 ceph-mon[85282]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:36.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:35 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/940054069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T08:39:36.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.218+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7ff0d410d670 con 0x7ff0d4072f10 2024-04-24T08:39:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.220+0000 7ff0d0c00640 1 -- 172.21.15.90:0/3767309046 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+4355 (secure 0 0 0) 0x7ff0bc07de30 con 0x7ff0d4072f10 2024-04-24T08:39:36.220 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:36.220 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:30.503086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 msgr2=0x7ff0b80710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 0x7ff0b80710e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4005fd0 tx=0x7ff0c4005850 comp rx=0 tx=0).stop 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 msgr2=0x7ff0d41a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc036040 tx=0x7ff0bc004040 comp rx=0 tx=0).stop 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 shutdown_connections 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7ff0b806ec30 0x7ff0b80710e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d4072f10 0x7ff0d41a1a40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 --2- 172.21.15.90:0/3767309046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0d4072600 0x7ff0d41a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 >> 172.21.15.90:0/3767309046 conn(0x7ff0d40fafb0 msgr2=0x7ff0d4101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 shutdown_connections 2024-04-24T08:39:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.223+0000 7ff0d9a00640 1 -- 172.21.15.90:0/3767309046 wait complete. 2024-04-24T08:39:36.242 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-24T08:39:36.242 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-24T08:39:36.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:35 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3746755121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T08:39:36.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:35 smithi149 ceph-mon[66490]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:36.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:35 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/940054069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T08:39:36.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 -- 172.21.15.90:0/1605707322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072f10 msgr2=0x7f154410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 --2- 172.21.15.90:0/1605707322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072f10 0x7f154410ff80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f152c009880 tx=0x7f152c02bd80 comp rx=0 tx=0).stop 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 -- 172.21.15.90:0/1605707322 shutdown_connections 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 --2- 172.21.15.90:0/1605707322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072f10 0x7f154410ff80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 --2- 172.21.15.90:0/1605707322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1544072600 0x7f15440729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 -- 172.21.15.90:0/1605707322 >> 172.21.15.90:0/1605707322 conn(0x7f15440fafb0 msgr2=0x7f15440fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 -- 172.21.15.90:0/1605707322 shutdown_connections 2024-04-24T08:39:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.444+0000 7f1549c00640 1 -- 172.21.15.90:0/1605707322 wait complete. 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 Processor -- start 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 -- start start 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1544072f10 0x7f154419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419c2a0 con 0x7f1544072600 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.445+0000 7f1549c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419f000 con 0x7f1544072f10 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35330/0 (socket says 172.21.15.90:35330) 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 -- 172.21.15.90:0/1744313898 learned_addr learned my addr 172.21.15.90:0/1744313898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 -- 172.21.15.90:0/1744313898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1544072f10 msgr2=0x7f154419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1544072f10 0x7f154419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 -- 172.21.15.90:0/1744313898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c009560 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.446+0000 7f1543e00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f153400cc60 tx=0x7f153400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.447+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f153400f450 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.447+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f153400fa70 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.447+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154419f2e0 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.447+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1534018b80 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.447+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154419f840 con 0x7f1544072600 2024-04-24T08:39:36.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.448+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154404e000 con 0x7f1544072600 2024-04-24T08:39:36.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.449+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1534018ce0 con 0x7f1544072600 2024-04-24T08:39:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.450+0000 7f1540c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 0x7f15280710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.450+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f153408f3f0 con 0x7f1544072600 2024-04-24T08:39:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.450+0000 7f1543400640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 0x7f15280710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.451+0000 7f1543400640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 0x7f15280710e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f152c036040 tx=0x7f152c000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.455+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f15340581e0 con 0x7f1544072600 2024-04-24T08:39:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.695+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f1544112d10 con 0x7f1544072600 2024-04-24T08:39:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.696+0000 7f1540c00640 1 -- 172.21.15.90:0/1744313898 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5205 (secure 0 0 0) 0x7f153405b8a0 con 0x7f1544072600 2024-04-24T08:39:36.696 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:36.696 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:30.503086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 31 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.698+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 msgr2=0x7f15280710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.698+0000 7f1549c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 0x7f15280710e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f152c036040 tx=0x7f152c000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.698+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 msgr2=0x7f154419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.698+0000 7f1549c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f153400cc60 tx=0x7f153400a620 comp rx=0 tx=0).stop 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 shutdown_connections 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f152806ec30 0x7f15280710e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1544072f10 0x7f154419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 --2- 172.21.15.90:0/1744313898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544072600 0x7f154419b710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 >> 172.21.15.90:0/1744313898 conn(0x7f15440fafb0 msgr2=0x7f1544101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 shutdown_connections 2024-04-24T08:39:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.699+0000 7f1549c00640 1 -- 172.21.15.90:0/1744313898 wait complete. 2024-04-24T08:39:36.718 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-24T08:39:36.718 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.914+0000 7fb857200640 1 -- 172.21.15.90:0/1920606388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 msgr2=0x7fb8581064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.914+0000 7fb857200640 1 --2- 172.21.15.90:0/1920606388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581064e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb844009880 tx=0x7fb84402bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 -- 172.21.15.90:0/1920606388 shutdown_connections 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 --2- 172.21.15.90:0/1920606388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581064e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 --2- 172.21.15.90:0/1920606388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85810bb60 0x7fb85810bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 -- 172.21.15.90:0/1920606388 >> 172.21.15.90:0/1920606388 conn(0x7fb858075260 msgr2=0x7fb858077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 -- 172.21.15.90:0/1920606388 shutdown_connections 2024-04-24T08:39:36.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.915+0000 7fb857200640 1 -- 172.21.15.90:0/1920606388 wait complete. 2024-04-24T08:39:36.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.916+0000 7fb857200640 1 Processor -- start 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.916+0000 7fb857200640 1 -- start start 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.916+0000 7fb857200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb857200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85810bb60 0x7fb8581a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb857200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85819b5c0 con 0x7fb858106090 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb857200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85819b700 con 0x7fb85810bb60 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35344/0 (socket says 172.21.15.90:35344) 2024-04-24T08:39:36.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 -- 172.21.15.90:0/2846149734 learned_addr learned my addr 172.21.15.90:0/2846149734 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 -- 172.21.15.90:0/2846149734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85810bb60 msgr2=0x7fb8581a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85810bb60 0x7fb8581a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.917+0000 7fb855e00640 1 -- 172.21.15.90:0/2846149734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb844009560 con 0x7fb858106090 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb855e00640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c810 tx=0x7fb84800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8480041f0 con 0x7fb858106090 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb848007e70 con 0x7fb858106090 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8480049d0 con 0x7fb858106090 2024-04-24T08:39:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85819b9e0 con 0x7fb858106090 2024-04-24T08:39:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.918+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85819bef0 con 0x7fb858106090 2024-04-24T08:39:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.919+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85804e000 con 0x7fb858106090 2024-04-24T08:39:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.920+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb8480079b0 con 0x7fb858106090 2024-04-24T08:39:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.921+0000 7fb84ea00640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 0x7fb83c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.921+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb84808ed90 con 0x7fb858106090 2024-04-24T08:39:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.921+0000 7fb855400640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 0x7fb83c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.922+0000 7fb855400640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 0x7fb83c0710e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb84402efd0 tx=0x7fb844000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:36.926+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb848057a50 con 0x7fb858106090 2024-04-24T08:39:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.170+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fb858112c70 con 0x7fb858106090 2024-04-24T08:39:37.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.171+0000 7fb84ea00640 1 -- 172.21.15.90:0/2846149734 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+5209 (secure 0 0 0) 0x7fb84805b110 con 0x7fb858106090 2024-04-24T08:39:37.171 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:37.171 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:39.816978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:37.172 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-04-24T08:39:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 msgr2=0x7fb83c0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 0x7fb83c0710e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb84402efd0 tx=0x7fb844000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 msgr2=0x7fb8581a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c810 tx=0x7fb84800ccd0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 shutdown_connections 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fb83c06ec30 0x7fb83c0710e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85810bb60 0x7fb8581a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 --2- 172.21.15.90:0/2846149734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb858106090 0x7fb8581a15e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 >> 172.21.15.90:0/2846149734 conn(0x7fb858075260 msgr2=0x7fb858076cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 shutdown_connections 2024-04-24T08:39:37.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.174+0000 7fb857200640 1 -- 172.21.15.90:0/2846149734 wait complete. 2024-04-24T08:39:37.192 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-24T08:39:37.192 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-24T08:39:37.258 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:36 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3767309046' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T08:39:37.258 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:36 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1744313898' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-24T08:39:37.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:36 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3767309046' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T08:39:37.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:36 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1744313898' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-24T08:39:37.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 -- 172.21.15.90:0/3727621698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65180ff870 msgr2=0x7f65180ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 --2- 172.21.15.90:0/3727621698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65180ff870 0x7f65180ffce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6500009880 tx=0x7f650002bcf0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 -- 172.21.15.90:0/3727621698 shutdown_connections 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 --2- 172.21.15.90:0/3727621698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65180ff870 0x7f65180ffce0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 --2- 172.21.15.90:0/3727621698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6518105880 0x7f6518105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 -- 172.21.15.90:0/3727621698 >> 172.21.15.90:0/3727621698 conn(0x7f65180fb370 msgr2=0x7f65180fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 -- 172.21.15.90:0/3727621698 shutdown_connections 2024-04-24T08:39:37.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.388+0000 7f651ca00640 1 -- 172.21.15.90:0/3727621698 wait complete. 2024-04-24T08:39:37.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.389+0000 7f651ca00640 1 Processor -- start 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.389+0000 7f651ca00640 1 -- start start 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f651ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65180ff870 0x7f651819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f651ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f651ca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518197680 con 0x7f6518105880 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f651ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65181977f0 con 0x7f65180ff870 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f650d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f650d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35358/0 (socket says 172.21.15.90:35358) 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.390+0000 7f650d400640 1 -- 172.21.15.90:0/1775600267 learned_addr learned my addr 172.21.15.90:0/1775600267 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f6515e00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65180ff870 0x7f651819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650d400640 1 -- 172.21.15.90:0/1775600267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65180ff870 msgr2=0x7f651819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650d400640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65180ff870 0x7f651819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650d400640 1 -- 172.21.15.90:0/1775600267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009560 con 0x7f6518105880 2024-04-24T08:39:37.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650d400640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6500036040 tx=0x7f6500004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:37.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65000035f0 con 0x7f6518105880 2024-04-24T08:39:37.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6500003c10 con 0x7f6518105880 2024-04-24T08:39:37.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6500034440 con 0x7f6518105880 2024-04-24T08:39:37.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.391+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6518197a70 con 0x7f6518105880 2024-04-24T08:39:37.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.392+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518198050 con 0x7f6518105880 2024-04-24T08:39:37.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.393+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651804e000 con 0x7f6518105880 2024-04-24T08:39:37.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.394+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6500005060 con 0x7f6518105880 2024-04-24T08:39:37.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.395+0000 7f650f400640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 0x7f64fc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.395+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f65000b1300 con 0x7f6518105880 2024-04-24T08:39:37.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.395+0000 7f6515e00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 0x7f64fc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:37.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.396+0000 7f6515e00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 0x7f64fc0710e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f65181009b0 tx=0x7f6508008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:37.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.400+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f650007a040 con 0x7f6518105880 2024-04-24T08:39:37.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.647+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f6518066860 con 0x7f6518105880 2024-04-24T08:39:37.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.649+0000 7f650f400640 1 -- 172.21.15.90:0/1775600267 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+5206 (secure 0 0 0) 0x7f650007d700 con 0x7f6518105880 2024-04-24T08:39:37.649 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:37.649 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:40.084668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T08:39:37.649 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 33 2024-04-24T08:39:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 msgr2=0x7f64fc0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 0x7f64fc0710e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f65181009b0 tx=0x7f6508008040 comp rx=0 tx=0).stop 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 msgr2=0x7f651819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6500036040 tx=0x7f6500004270 comp rx=0 tx=0).stop 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 shutdown_connections 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f64fc06ec30 0x7f64fc0710e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6518105880 0x7f651819db50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 --2- 172.21.15.90:0/1775600267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65180ff870 0x7f651819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.651+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 >> 172.21.15.90:0/1775600267 conn(0x7f65180fb370 msgr2=0x7f65180fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.652+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 shutdown_connections 2024-04-24T08:39:37.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.652+0000 7f651ca00640 1 -- 172.21.15.90:0/1775600267 wait complete. 2024-04-24T08:39:37.671 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-04-24T08:39:37.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-24T08:39:37.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 -- 172.21.15.90:0/3989624262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c074790 msgr2=0x7fcc9c074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3989624262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c074790 0x7fcc9c074b60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009880 tx=0x7fcc8c02bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 -- 172.21.15.90:0/3989624262 shutdown_connections 2024-04-24T08:39:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3989624262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c10fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3989624262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c074790 0x7fcc9c074b60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 -- 172.21.15.90:0/3989624262 >> 172.21.15.90:0/3989624262 conn(0x7fcc9c0fd170 msgr2=0x7fcc9c0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 -- 172.21.15.90:0/3989624262 shutdown_connections 2024-04-24T08:39:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.859+0000 7fcc9be00640 1 -- 172.21.15.90:0/3989624262 wait complete. 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 Processor -- start 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 -- start start 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc9c074790 0x7fcc9c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c19c340 con 0x7fcc9c0750a0 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.860+0000 7fcc9be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c19efa0 con 0x7fcc9c074790 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35380/0 (socket says 172.21.15.90:35380) 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 -- 172.21.15.90:0/3417831528 learned_addr learned my addr 172.21.15.90:0/3417831528 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 -- 172.21.15.90:0/3417831528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc9c074790 msgr2=0x7fcc9c19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc9c074790 0x7fcc9c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:37.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 -- 172.21.15.90:0/3417831528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009560 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.861+0000 7fcc9a000640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c810 tx=0x7fcc8800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.862+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc880041f0 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.862+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc88007e70 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.862+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc880049d0 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.862+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c19f280 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.862+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c19f7e0 con 0x7fcc9c0750a0 2024-04-24T08:39:37.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.863+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9c077530 con 0x7fcc9c0750a0 2024-04-24T08:39:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.864+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcc880079b0 con 0x7fcc9c0750a0 2024-04-24T08:39:37.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.865+0000 7fcc93400640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 0x7fcc800710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:37.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.865+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcc8808eca0 con 0x7fcc9c0750a0 2024-04-24T08:39:37.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.865+0000 7fcc9aa00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 0x7fcc800710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:37.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.866+0000 7fcc9aa00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 0x7fcc800710e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036040 tx=0x7fcc8c000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:37.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:37.870+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcc880579e0 con 0x7fcc9c0750a0 2024-04-24T08:39:38.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.120+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fcc9c04e000 con 0x7fcc9c0750a0 2024-04-24T08:39:38.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.121+0000 7fcc93400640 1 -- 172.21.15.90:0/3417831528 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+5215 (secure 0 0 0) 0x7fcc8801d090 con 0x7fcc9c0750a0 2024-04-24T08:39:38.121 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:38.121 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:42.091184+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34462,"qdb_cluster":[34462]},"id":1}]} 2024-04-24T08:39:38.122 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 34 2024-04-24T08:39:38.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 msgr2=0x7fcc800710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 0x7fcc800710e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036040 tx=0x7fcc8c000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 msgr2=0x7fcc9c19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c810 tx=0x7fcc8800ccd0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 shutdown_connections 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fcc8006ec30 0x7fcc800710e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc9c0750a0 0x7fcc9c19bc60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 --2- 172.21.15.90:0/3417831528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc9c074790 0x7fcc9c19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 >> 172.21.15.90:0/3417831528 conn(0x7fcc9c0fd170 msgr2=0x7fcc9c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 shutdown_connections 2024-04-24T08:39:38.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.124+0000 7fcc9be00640 1 -- 172.21.15.90:0/3417831528 wait complete. 2024-04-24T08:39:38.142 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-04-24T08:39:38.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-24T08:39:38.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:37 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2846149734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-24T08:39:38.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:37 smithi149 ceph-mon[66490]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:38.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:37 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1775600267' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-24T08:39:38.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:37 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2846149734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-24T08:39:38.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:37 smithi090 ceph-mon[85282]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:38.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:37 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1775600267' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-24T08:39:38.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 -- 172.21.15.90:0/3375931710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d24101890 msgr2=0x7f1d24101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 --2- 172.21.15.90:0/3375931710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d24101890 0x7f1d24101d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009880 tx=0x7f1d0c02bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 -- 172.21.15.90:0/3375931710 shutdown_connections 2024-04-24T08:39:38.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 --2- 172.21.15.90:0/3375931710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d24101890 0x7f1d24101d00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 --2- 172.21.15.90:0/3375931710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d241078a0 0x7f1d24107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 -- 172.21.15.90:0/3375931710 >> 172.21.15.90:0/3375931710 conn(0x7f1d240fd390 msgr2=0x7f1d240ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:38.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 -- 172.21.15.90:0/3375931710 shutdown_connections 2024-04-24T08:39:38.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.346+0000 7f1d2a200640 1 -- 172.21.15.90:0/3375931710 wait complete. 2024-04-24T08:39:38.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.347+0000 7f1d2a200640 1 Processor -- start 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.347+0000 7f1d2a200640 1 -- start start 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d2a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 0x7f1d241974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d2a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d2a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24198030 con 0x7f1d241078a0 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d2a200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d241981a0 con 0x7f1d24101890 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35396/0 (socket says 172.21.15.90:35396) 2024-04-24T08:39:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 -- 172.21.15.90:0/233860217 learned_addr learned my addr 172.21.15.90:0/233860217 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d23400640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 0x7f1d241974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 -- 172.21.15.90:0/233860217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 msgr2=0x7f1d241974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 0x7f1d241974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d1aa00640 1 -- 172.21.15.90:0/233860217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c009560 con 0x7f1d241078a0 2024-04-24T08:39:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.348+0000 7f1d23400640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 0x7f1d241974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d1aa00640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c0037c0 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d0c003de0 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c034460 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2419bfd0 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.349+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2419c550 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.350+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2404e000 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.351+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1d0c003920 con 0x7f1d241078a0 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.352+0000 7f1d20c00640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 0x7f1d080710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.352+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d0c0b12f0 con 0x7f1d241078a0 2024-04-24T08:39:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.353+0000 7f1d23400640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 0x7f1d080710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.353+0000 7f1d23400640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 0x7f1d080710e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d241029d0 tx=0x7f1d14008370 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:38.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.357+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1d0c07a190 con 0x7f1d241078a0 2024-04-24T08:39:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.596+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f1d2410ecd0 con 0x7f1d241078a0 2024-04-24T08:39:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.598+0000 7f1d20c00640 1 -- 172.21.15.90:0/233860217 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f1d0c07d850 con 0x7f1d241078a0 2024-04-24T08:39:38.598 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:38.598 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:52.117352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34462,"qdb_cluster":[34462]},"id":1}]} 2024-04-24T08:39:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 msgr2=0x7f1d080710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 0x7f1d080710e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d241029d0 tx=0x7f1d14008370 comp rx=0 tx=0).stop 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 msgr2=0x7f1d24197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c004270 comp rx=0 tx=0).stop 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 shutdown_connections 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f1d0806ec30 0x7f1d080710e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d241078a0 0x7f1d24197a30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 --2- 172.21.15.90:0/233860217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d24101890 0x7f1d241974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.600+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 >> 172.21.15.90:0/233860217 conn(0x7f1d240fd390 msgr2=0x7f1d240fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.601+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 shutdown_connections 2024-04-24T08:39:38.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.601+0000 7f1d2a200640 1 -- 172.21.15.90:0/233860217 wait complete. 2024-04-24T08:39:38.619 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-04-24T08:39:38.620 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 -- 172.21.15.90:0/3473546659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105550 msgr2=0x7f75ec105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 --2- 172.21.15.90:0/3473546659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105550 0x7f75ec105920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009880 tx=0x7f75e002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 -- 172.21.15.90:0/3473546659 shutdown_connections 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 --2- 172.21.15.90:0/3473546659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105e60 0x7f75ec067310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 --2- 172.21.15.90:0/3473546659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105550 0x7f75ec105920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.812+0000 7f75f3000640 1 -- 172.21.15.90:0/3473546659 >> 172.21.15.90:0/3473546659 conn(0x7f75ec0743d0 msgr2=0x7f75ec0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.813+0000 7f75f3000640 1 -- 172.21.15.90:0/3473546659 shutdown_connections 2024-04-24T08:39:38.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.813+0000 7f75f3000640 1 -- 172.21.15.90:0/3473546659 wait complete. 2024-04-24T08:39:38.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.813+0000 7f75f3000640 1 Processor -- start 2024-04-24T08:39:38.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f3000640 1 -- start start 2024-04-24T08:39:38.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f3000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105550 0x7f75ec197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f3000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f3000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec197f20 con 0x7f75ec105e60 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f3000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19bcf0 con 0x7f75ec105550 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35426/0 (socket says 172.21.15.90:35426) 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.814+0000 7f75f1200640 1 -- 172.21.15.90:0/212807860 learned_addr learned my addr 172.21.15.90:0/212807860 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f1c00640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105550 0x7f75ec197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f1200640 1 -- 172.21.15.90:0/212807860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105550 msgr2=0x7f75ec197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f1200640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105550 0x7f75ec197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f1200640 1 -- 172.21.15.90:0/212807860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009560 con 0x7f75ec105e60 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f1200640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75d800d6d0 tx=0x7f75d800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d80051f0 con 0x7f75ec105e60 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75d8009ce0 con 0x7f75ec105e60 2024-04-24T08:39:38.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec19bfd0 con 0x7f75ec105e60 2024-04-24T08:39:38.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.815+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d8004e40 con 0x7f75ec105e60 2024-04-24T08:39:38.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.816+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec19c5b0 con 0x7f75ec105e60 2024-04-24T08:39:38.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.817+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec04e000 con 0x7f75ec105e60 2024-04-24T08:39:38.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.818+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f75d8005350 con 0x7f75ec105e60 2024-04-24T08:39:38.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.820+0000 7f75ea000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 0x7f75d4071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:38.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.820+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f75d808ebc0 con 0x7f75ec105e60 2024-04-24T08:39:38.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.820+0000 7f75f1c00640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 0x7f75d4071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:38.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.821+0000 7f75f1c00640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 0x7f75d4071080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75e0000c00 tx=0x7f75e002cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:38.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:38.824+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f75d8057880 con 0x7f75ec105e60 2024-04-24T08:39:39.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.071+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f75ec1986a0 con 0x7f75ec105e60 2024-04-24T08:39:39.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.073+0000 7f75ea000640 1 -- 172.21.15.90:0/212807860 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5214 (secure 0 0 0) 0x7f75d805af40 con 0x7f75ec105e60 2024-04-24T08:39:39.073 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:39.073 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:53.116863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34462,"qdb_cluster":[34462]},"id":1}]} 2024-04-24T08:39:39.073 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 36 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.075+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 msgr2=0x7f75d4071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.075+0000 7f75f3000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 0x7f75d4071080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75e0000c00 tx=0x7f75e002cf90 comp rx=0 tx=0).stop 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.075+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 msgr2=0x7f75ec197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.075+0000 7f75f3000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75d800d6d0 tx=0x7f75d800db90 comp rx=0 tx=0).stop 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 shutdown_connections 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f75d406ebd0 0x7f75d4071080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec105e60 0x7f75ec197890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 --2- 172.21.15.90:0/212807860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec105550 0x7f75ec197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 >> 172.21.15.90:0/212807860 conn(0x7f75ec0743d0 msgr2=0x7f75ec0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:39.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 shutdown_connections 2024-04-24T08:39:39.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.076+0000 7f75f3000640 1 -- 172.21.15.90:0/212807860 wait complete. 2024-04-24T08:39:39.095 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-24T08:39:39.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:38 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3417831528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-24T08:39:39.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:38 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/233860217' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-24T08:39:39.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:38 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3417831528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-24T08:39:39.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:38 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/233860217' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-24T08:39:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 -- 172.21.15.90:0/80978998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 msgr2=0x7fef7c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 --2- 172.21.15.90:0/80978998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c101ae0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fef68009880 tx=0x7fef6802bcf0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 -- 172.21.15.90:0/80978998 shutdown_connections 2024-04-24T08:39:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 --2- 172.21.15.90:0/80978998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c101ae0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 --2- 172.21.15.90:0/80978998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef7c107680 0x7fef7c107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 -- 172.21.15.90:0/80978998 >> 172.21.15.90:0/80978998 conn(0x7fef7c0fd170 msgr2=0x7fef7c0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:39.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 -- 172.21.15.90:0/80978998 shutdown_connections 2024-04-24T08:39:39.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.295+0000 7fef81000640 1 -- 172.21.15.90:0/80978998 wait complete. 2024-04-24T08:39:39.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 Processor -- start 2024-04-24T08:39:39.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 -- start start 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef7c107680 0x7fef7c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c197e70 con 0x7fef7c101670 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef81000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c19bc40 con 0x7fef7c107680 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.296+0000 7fef7b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35436/0 (socket says 172.21.15.90:35436) 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 -- 172.21.15.90:0/2783882793 learned_addr learned my addr 172.21.15.90:0/2783882793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 -- 172.21.15.90:0/2783882793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef7c107680 msgr2=0x7fef7c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef7c107680 0x7fef7c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 -- 172.21.15.90:0/2783882793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009560 con 0x7fef7c101670 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef7b400640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c8f0 tx=0x7fef6c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:39.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.297+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c007b80 con 0x7fef7c101670 2024-04-24T08:39:39.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.298+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef6c007ce0 con 0x7fef7c101670 2024-04-24T08:39:39.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.298+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c19bf20 con 0x7fef7c101670 2024-04-24T08:39:39.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.298+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c018870 con 0x7fef7c101670 2024-04-24T08:39:39.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.298+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c19c500 con 0x7fef7c101670 2024-04-24T08:39:39.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.299+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c04e000 con 0x7fef7c101670 2024-04-24T08:39:39.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.300+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fef6c018ae0 con 0x7fef7c101670 2024-04-24T08:39:39.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.301+0000 7fef73e00640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 0x7fef600710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.301+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fef6c08f010 con 0x7fef7c101670 2024-04-24T08:39:39.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.301+0000 7fef7aa00640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 0x7fef600710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:39.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.302+0000 7fef7aa00640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 0x7fef600710e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef68036040 tx=0x7fef68000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:39.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.306+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef6c057d70 con 0x7fef7c101670 2024-04-24T08:39:39.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.546+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fef7c1985f0 con 0x7fef7c101670 2024-04-24T08:39:39.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.548+0000 7fef73e00640 1 -- 172.21.15.90:0/2783882793 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5221 (secure 0 0 0) 0x7fef6c05b430 con 0x7fef7c101670 2024-04-24T08:39:39.548 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:39.548 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44297,"name":"cephfs.smithi149.eazcmc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/4101562674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4101562674},{"type":"v1","addr":"172.21.15.149:6825","nonce":4101562674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44301,"name":"cephfs.smithi149.zwjtmw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1676173850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1676173850},{"type":"v1","addr":"172.21.15.149:6827","nonce":1676173850}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:20:07.120167+0000","modified":"2024-04-24T08:36:54.127064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi090.qbyvpv","rank":0,"incarnation":30,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6827/1385436580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1385436580},{"type":"v1","addr":"172.21.15.90:6827","nonce":1385436580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34494":{"gid":34494,"name":"cephfs.smithi090.lumztb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6829/4088472812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4088472812},{"type":"v1","addr":"172.21.15.90:6829","nonce":4088472812}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34462,"qdb_cluster":[34462]},"id":1}]} 2024-04-24T08:39:39.548 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 msgr2=0x7fef600710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 0x7fef600710e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef68036040 tx=0x7fef68000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 msgr2=0x7fef7c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c8f0 tx=0x7fef6c00cdb0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 shutdown_connections 2024-04-24T08:39:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fef6006ec30 0x7fef600710e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef7c107680 0x7fef7c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 --2- 172.21.15.90:0/2783882793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef7c101670 0x7fef7c1972a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 >> 172.21.15.90:0/2783882793 conn(0x7fef7c0fd170 msgr2=0x7fef7c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:39.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 shutdown_connections 2024-04-24T08:39:39.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.550+0000 7fef81000640 1 -- 172.21.15.90:0/2783882793 wait complete. 2024-04-24T08:39:39.568 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T08:39:39.579 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T08:39:39.580 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:39:39.580 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T08:39:39.629 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:39:39.629 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T08:39:39.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.924+0000 7f4383e00640 1 -- 172.21.15.90:0/2825999017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 msgr2=0x7f4384109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.924+0000 7f4383e00640 1 --2- 172.21.15.90:0/2825999017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f4384109660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f436c009880 tx=0x7f436c02bcf0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 -- 172.21.15.90:0/2825999017 shutdown_connections 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 --2- 172.21.15.90:0/2825999017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f4384109660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 --2- 172.21.15.90:0/2825999017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 0x7f43841066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 -- 172.21.15.90:0/2825999017 >> 172.21.15.90:0/2825999017 conn(0x7f43840752c0 msgr2=0x7f43840776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 -- 172.21.15.90:0/2825999017 shutdown_connections 2024-04-24T08:39:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.925+0000 7f4383e00640 1 -- 172.21.15.90:0/2825999017 wait complete. 2024-04-24T08:39:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 Processor -- start 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 -- start start 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 0x7f438419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438419c1a0 con 0x7f4384106ca0 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.926+0000 7f4383e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438419c310 con 0x7f4384106300 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35072/0 (socket says 172.21.15.90:35072) 2024-04-24T08:39:39.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 -- 172.21.15.90:0/3068138774 learned_addr learned my addr 172.21.15.90:0/3068138774 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f4382a00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 0x7f438419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 -- 172.21.15.90:0/3068138774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 msgr2=0x7f438419b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 0x7f438419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.927+0000 7f437a000640 1 -- 172.21.15.90:0/3068138774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c009560 con 0x7f4384106ca0 2024-04-24T08:39:39.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f437a000640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f436c036040 tx=0x7f436c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f436c0035f0 con 0x7f4384106ca0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f436c003c10 con 0x7f4384106ca0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f436c034440 con 0x7f4384106ca0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43841a0140 con 0x7f4384106ca0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.928+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43841a0720 con 0x7f4384106ca0 2024-04-24T08:39:39.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.929+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438404e850 con 0x7f4384106ca0 2024-04-24T08:39:39.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.930+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f436c004f60 con 0x7f4384106ca0 2024-04-24T08:39:39.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.931+0000 7f437be00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 0x7f43680711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:39.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.931+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f436c0b1300 con 0x7f4384106ca0 2024-04-24T08:39:39.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.931+0000 7f4382a00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 0x7f43680711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:39.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.932+0000 7f4382a00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 0x7f43680711a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f437400ba40 tx=0x7f4374005cc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:39.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:39.936+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f436c07a040 con 0x7f4384106ca0 2024-04-24T08:39:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.144+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4384112fb0 con 0x7f4384106ca0 2024-04-24T08:39:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.147+0000 7f437be00640 1 -- 172.21.15.90:0/3068138774 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f436c07d700 con 0x7f4384106ca0 2024-04-24T08:39:40.147 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:40.147 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:36:30.490085+0000","last_up_change":"2024-04-24T08:34:51.191821+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T08:20:06.040754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T08:20:06.663119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6803","nonce":4221900826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6805","nonce":4221900826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6809","nonce":4221900826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6807","nonce":4221900826}]},"public_addr":"172.21.15.90:6803/4221900826","cluster_addr":"172.21.15.90:6805/4221900826","heartbeat_back_addr":"172.21.15.90:6809/4221900826","heartbeat_front_addr":"172.21.15.90:6807/4221900826","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6811","nonce":1858244546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6813","nonce":1858244546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6817","nonce":1858244546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6815","nonce":1858244546}]},"public_addr":"172.21.15.90:6811/1858244546","cluster_addr":"172.21.15.90:6813/1858244546","heartbeat_back_addr":"172.21.15.90:6817/1858244546","heartbeat_front_addr":"172.21.15.90:6815/1858244546","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6819","nonce":701071143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6821","nonce":701071143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6825","nonce":701071143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6823","nonce":701071143}]},"public_addr":"172.21.15.90:6819/701071143","cluster_addr":"172.21.15.90:6821/701071143","heartbeat_back_addr":"172.21.15.90:6825/701071143","heartbeat_front_addr":"172.21.15.90:6823/701071143","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6801","nonce":4083464090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6803","nonce":4083464090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6807","nonce":4083464090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6805","nonce":4083464090}]},"public_addr":"172.21.15.149:6801/4083464090","cluster_addr":"172.21.15.149:6803/4083464090","heartbeat_back_addr":"172.21.15.149:6807/4083464090","heartbeat_front_addr":"172.21.15.149:6805/4083464090","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6809","nonce":3688072354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6811","nonce":3688072354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6815","nonce":3688072354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6813","nonce":3688072354}]},"public_addr":"172.21.15.149:6809/3688072354","cluster_addr":"172.21.15.149:6811/3688072354","heartbeat_back_addr":"172.21.15.149:6815/3688072354","heartbeat_front_addr":"172.21.15.149:6813/3688072354","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6817","nonce":1799649429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6819","nonce":1799649429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6823","nonce":1799649429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6821","nonce":1799649429}]},"public_addr":"172.21.15.149:6817/1799649429","cluster_addr":"172.21.15.149:6819/1799649429","heartbeat_back_addr":"172.21.15.149:6823/1799649429","heartbeat_front_addr":"172.21.15.149:6821/1799649429","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6826/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6825/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.149:6824/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3635708408":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/903772982":"2024-04-25T08:29:15.008628+0000","172.21.15.149:0/3314646633":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6826/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3128177159":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.149:6827/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6828/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.149:0/3752867478":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6827/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/2244430099":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.149:0/3222289279":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/2953349914":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/563018427":"2024-04-25T08:23:20.611807+0000","172.21.15.149:6829/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.149:0/683486625":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/182534323":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/78008447":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3406148041":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/152802986":"2024-04-25T08:29:15.008628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:39:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.149+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 msgr2=0x7f43680711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.149+0000 7f4383e00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 0x7f43680711a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f437400ba40 tx=0x7f4374005cc0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.149+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 msgr2=0x7f438419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.149+0000 7f4383e00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f436c036040 tx=0x7f436c004040 comp rx=0 tx=0).stop 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 shutdown_connections 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f436806ecf0 0x7f43680711a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4384106ca0 0x7f438419bba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 --2- 172.21.15.90:0/3068138774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4384106300 0x7f438419b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 >> 172.21.15.90:0/3068138774 conn(0x7f43840752c0 msgr2=0x7f43840776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 shutdown_connections 2024-04-24T08:39:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.150+0000 7f4383e00640 1 -- 172.21.15.90:0/3068138774 wait complete. 2024-04-24T08:39:40.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:16:48.618419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T08:20:06.040754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T08:20:06.663119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:39:40.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:39:40.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:39 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/212807860' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-24T08:39:40.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:39 smithi149 ceph-mon[66490]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:40.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:39 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2783882793' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-24T08:39:40.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:39 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/212807860' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-24T08:39:40.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:39 smithi090 ceph-mon[85282]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:40.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:39 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2783882793' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-24T08:39:40.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 -- 172.21.15.90:0/1631453982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 msgr2=0x7f21d80ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 --2- 172.21.15.90:0/1631453982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d80ff920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009c20 tx=0x7f21c002c1b0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 -- 172.21.15.90:0/1631453982 shutdown_connections 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 --2- 172.21.15.90:0/1631453982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d80ff920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 --2- 172.21.15.90:0/1631453982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d81054c0 0x7f21d8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 -- 172.21.15.90:0/1631453982 >> 172.21.15.90:0/1631453982 conn(0x7f21d80fafb0 msgr2=0x7f21d80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 -- 172.21.15.90:0/1631453982 shutdown_connections 2024-04-24T08:39:40.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.374+0000 7f21d7200640 1 -- 172.21.15.90:0/1631453982 wait complete. 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.375+0000 7f21d7200640 1 Processor -- start 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.375+0000 7f21d7200640 1 -- start start 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d81054c0 0x7f21d8072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d7200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8072fe0 con 0x7f21d80ff4b0 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8073120 con 0x7f21d81054c0 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35084/0 (socket says 172.21.15.90:35084) 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 -- 172.21.15.90:0/4208798693 learned_addr learned my addr 172.21.15.90:0/4208798693 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:40.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 -- 172.21.15.90:0/4208798693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d81054c0 msgr2=0x7f21d8072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d81054c0 0x7f21d8072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.376+0000 7f21d5e00640 1 -- 172.21.15.90:0/4208798693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009900 con 0x7f21d80ff4b0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21d5e00640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c8a0 tx=0x7f21c800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c8007b80 con 0x7f21d80ff4b0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21c8007ce0 con 0x7f21d80ff4b0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c8018810 con 0x7f21d80ff4b0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80733b0 con 0x7f21d80ff4b0 2024-04-24T08:39:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.377+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d81a16b0 con 0x7f21d80ff4b0 2024-04-24T08:39:40.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.378+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d804e000 con 0x7f21d80ff4b0 2024-04-24T08:39:40.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.379+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f21c8018a80 con 0x7f21d80ff4b0 2024-04-24T08:39:40.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.380+0000 7f21cea00640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 0x7f21b80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.380+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f21c808f170 con 0x7f21d80ff4b0 2024-04-24T08:39:40.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.380+0000 7f21d5400640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 0x7f21b80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.381+0000 7f21d5400640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 0x7f21b80710e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006010 tx=0x7f21c0009360 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.385+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f21c8057e30 con 0x7f21d80ff4b0 2024-04-24T08:39:40.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.584+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21d8073b40 con 0x7f21d80ff4b0 2024-04-24T08:39:40.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.585+0000 7f21cea00640 1 -- 172.21.15.90:0/4208798693 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21c805b4f0 con 0x7f21d80ff4b0 2024-04-24T08:39:40.585 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T08:39:40.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 msgr2=0x7f21b80710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 0x7f21b80710e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006010 tx=0x7f21c0009360 comp rx=0 tx=0).stop 2024-04-24T08:39:40.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 msgr2=0x7f21d8072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c8a0 tx=0x7f21c800cd60 comp rx=0 tx=0).stop 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 shutdown_connections 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f21b806ec30 0x7f21b80710e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d81054c0 0x7f21d8072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 --2- 172.21.15.90:0/4208798693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d80ff4b0 0x7f21d8072560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 >> 172.21.15.90:0/4208798693 conn(0x7f21d80fafb0 msgr2=0x7f21d80fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 shutdown_connections 2024-04-24T08:39:40.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.588+0000 7f21d7200640 1 -- 172.21.15.90:0/4208798693 wait complete. 2024-04-24T08:39:40.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T08:39:40.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 -- 172.21.15.90:0/3818282675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 msgr2=0x7f4fe0074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/3818282675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe0074b60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009880 tx=0x7f4fd002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 -- 172.21.15.90:0/3818282675 shutdown_connections 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/3818282675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 0x7f4fe010fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/3818282675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe0074b60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 -- 172.21.15.90:0/3818282675 >> 172.21.15.90:0/3818282675 conn(0x7f4fe00fd170 msgr2=0x7f4fe00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 -- 172.21.15.90:0/3818282675 shutdown_connections 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.814+0000 7f4fdfe00640 1 -- 172.21.15.90:0/3818282675 wait complete. 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.815+0000 7f4fdfe00640 1 Processor -- start 2024-04-24T08:39:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.815+0000 7f4fdfe00640 1 -- start start 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 0x7f4fe01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe019b580 con 0x7f4fe0074790 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe019b6f0 con 0x7f4fe00750a0 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35102/0 (socket says 172.21.15.90:35102) 2024-04-24T08:39:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fdea00640 1 -- 172.21.15.90:0/1686640516 learned_addr learned my addr 172.21.15.90:0/1686640516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.816+0000 7f4fde000640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 0x7f4fe01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fdea00640 1 -- 172.21.15.90:0/1686640516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 msgr2=0x7f4fe01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fdea00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 0x7f4fe01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fdea00640 1 -- 172.21.15.90:0/1686640516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0009560 con 0x7f4fe0074790 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fdea00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4fd002efd0 tx=0x7f4fd002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd00035f0 con 0x7f4fe0074790 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fd0003c10 con 0x7f4fe0074790 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.817+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe019b970 con 0x7f4fe0074790 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.818+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd0005690 con 0x7f4fe0074790 2024-04-24T08:39:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.818+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe019be70 con 0x7f4fe0074790 2024-04-24T08:39:40.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.819+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe004e000 con 0x7f4fe0074790 2024-04-24T08:39:40.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.820+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4fd0004ed0 con 0x7f4fe0074790 2024-04-24T08:39:40.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.821+0000 7f4fd7400640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 0x7f4fc40710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.821+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4fd00b13a0 con 0x7f4fe0074790 2024-04-24T08:39:40.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.821+0000 7f4fde000640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 0x7f4fc40710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.822+0000 7f4fde000640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 0x7f4fc40710e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005fd0 tx=0x7f4fcc005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:40.826+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4fd007a0e0 con 0x7f4fe0074790 2024-04-24T08:39:41.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.031+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4fe010d4b0 con 0x7f4fe0074790 2024-04-24T08:39:41.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.032+0000 7f4fd7400640 1 -- 172.21.15.90:0/1686640516 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4fd007d7a0 con 0x7f4fe0074790 2024-04-24T08:39:41.032 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.034+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 msgr2=0x7f4fc40710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.034+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 0x7f4fc40710e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005fd0 tx=0x7f4fcc005850 comp rx=0 tx=0).stop 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.034+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 msgr2=0x7f4fe01a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.034+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4fd002efd0 tx=0x7f4fd002c850 comp rx=0 tx=0).stop 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 shutdown_connections 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f4fc406ec30 0x7f4fc40710e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe00750a0 0x7f4fe01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 --2- 172.21.15.90:0/1686640516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0074790 0x7f4fe01a1510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 >> 172.21.15.90:0/1686640516 conn(0x7f4fe00fd170 msgr2=0x7f4fe00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:41.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 shutdown_connections 2024-04-24T08:39:41.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.035+0000 7f4fdfe00640 1 -- 172.21.15.90:0/1686640516 wait complete. 2024-04-24T08:39:41.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T08:39:41.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 -- 172.21.15.90:0/3269715928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c98106090 msgr2=0x7f9c981064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 --2- 172.21.15.90:0/3269715928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c98106090 0x7f9c981064e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009880 tx=0x7f9c8002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 -- 172.21.15.90:0/3269715928 shutdown_connections 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 --2- 172.21.15.90:0/3269715928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c98106090 0x7f9c981064e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 --2- 172.21.15.90:0/3269715928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c9810bb60 0x7f9c9810bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 -- 172.21.15.90:0/3269715928 >> 172.21.15.90:0/3269715928 conn(0x7f9c98075260 msgr2=0x7f9c98077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 -- 172.21.15.90:0/3269715928 shutdown_connections 2024-04-24T08:39:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.257+0000 7f9c97200640 1 -- 172.21.15.90:0/3269715928 wait complete. 2024-04-24T08:39:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.258+0000 7f9c97200640 1 Processor -- start 2024-04-24T08:39:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.258+0000 7f9c97200640 1 -- start start 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c97200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 0x7f9c981a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c97200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c97200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819b690 con 0x7f9c9810bb60 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c97200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819b800 con 0x7f9c98106090 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35114/0 (socket says 172.21.15.90:35114) 2024-04-24T08:39:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c8fe00640 1 -- 172.21.15.90:0/1994136636 learned_addr learned my addr 172.21.15.90:0/1994136636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.259+0000 7f9c95e00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 0x7f9c981a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8fe00640 1 -- 172.21.15.90:0/1994136636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 msgr2=0x7f9c981a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8fe00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 0x7f9c981a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8fe00640 1 -- 172.21.15.90:0/1994136636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009560 con 0x7f9c9810bb60 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c95e00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 0x7f9c981a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8fe00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c8002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c80004520 con 0x7f9c9810bb60 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c80003ab0 con 0x7f9c9810bb60 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c80034440 con 0x7f9c9810bb60 2024-04-24T08:39:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9819ba80 con 0x7f9c9810bb60 2024-04-24T08:39:41.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.260+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9819bf80 con 0x7f9c9810bb60 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.262+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9804e000 con 0x7f9c9810bb60 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.263+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9c80003c20 con 0x7f9c9810bb60 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.264+0000 7f9c8ea00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 0x7f9c780710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.264+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c800b1170 con 0x7f9c9810bb60 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.264+0000 7f9c95e00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 0x7f9c780710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:41.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.265+0000 7f9c95e00640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 0x7f9c780710e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88005e50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:41.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.269+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9c80079f50 con 0x7f9c9810bb60 2024-04-24T08:39:41.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:41 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3068138774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:41.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:41 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/4208798693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:40 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3068138774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:41.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:40 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/4208798693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:41.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.473+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f9c9819cc30 con 0x7f9c9810bb60 2024-04-24T08:39:41.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.474+0000 7f9c8ea00640 1 -- 172.21.15.90:0/1994136636 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9c8007d610 con 0x7f9c9810bb60 2024-04-24T08:39:41.475 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 msgr2=0x7f9c780710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 0x7f9c780710e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88005e50 comp rx=0 tx=0).stop 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 msgr2=0x7f9c981a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c8002c850 comp rx=0 tx=0).stop 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 shutdown_connections 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f9c7806ec30 0x7f9c780710e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c9810bb60 0x7f9c981a1b60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 --2- 172.21.15.90:0/1994136636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c98106090 0x7f9c981a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.477+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 >> 172.21.15.90:0/1994136636 conn(0x7f9c98075260 msgr2=0x7f9c98076d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:41.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.478+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 shutdown_connections 2024-04-24T08:39:41.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.478+0000 7f9c97200640 1 -- 172.21.15.90:0/1994136636 wait complete. 2024-04-24T08:39:41.497 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T08:39:41.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 -- 172.21.15.90:0/4181718813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 msgr2=0x7f3ca0107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/4181718813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca0107c50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009880 tx=0x7f3c9402bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 -- 172.21.15.90:0/4181718813 shutdown_connections 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/4181718813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ca0101870 0x7f3ca0101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/4181718813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca0107c50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 -- 172.21.15.90:0/4181718813 >> 172.21.15.90:0/4181718813 conn(0x7f3ca00fd370 msgr2=0x7f3ca00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 -- 172.21.15.90:0/4181718813 shutdown_connections 2024-04-24T08:39:41.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.698+0000 7f3ca7e00640 1 -- 172.21.15.90:0/4181718813 wait complete. 2024-04-24T08:39:41.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.699+0000 7f3ca7e00640 1 Processor -- start 2024-04-24T08:39:41.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.699+0000 7f3ca7e00640 1 -- start start 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ca0101870 0x7f3ca0197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0197f30 con 0x7f3ca0107880 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019bd00 con 0x7f3ca0101870 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35130/0 (socket says 172.21.15.90:35130) 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 -- 172.21.15.90:0/1666588744 learned_addr learned my addr 172.21.15.90:0/1666588744 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 -- 172.21.15.90:0/1666588744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ca0101870 msgr2=0x7f3ca0197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ca0101870 0x7f3ca0197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.700+0000 7f3ca4a00640 1 -- 172.21.15.90:0/1666588744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94009560 con 0x7f3ca0107880 2024-04-24T08:39:41.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3ca4a00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c810 tx=0x7f3c8c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:41.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8c0041f0 con 0x7f3ca0107880 2024-04-24T08:39:41.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c8c007e70 con 0x7f3ca0107880 2024-04-24T08:39:41.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8c0049d0 con 0x7f3ca0107880 2024-04-24T08:39:41.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca019bfe0 con 0x7f3ca0107880 2024-04-24T08:39:41.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.701+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca019c5c0 con 0x7f3ca0107880 2024-04-24T08:39:41.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.702+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca004e000 con 0x7f3ca0107880 2024-04-24T08:39:41.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.704+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3c8c0079b0 con 0x7f3ca0107880 2024-04-24T08:39:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.705+0000 7f3c9e000640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 0x7f3c880711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.705+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3c8c08ed90 con 0x7f3ca0107880 2024-04-24T08:39:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.705+0000 7f3ca5400640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 0x7f3c880711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.706+0000 7f3ca5400640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 0x7f3c880711a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c94036040 tx=0x7f3c94000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:41.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.709+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3c8c057a50 con 0x7f3ca0107880 2024-04-24T08:39:41.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.928+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3ca01986b0 con 0x7f3ca0107880 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.929+0000 7f3c9e000640 1 -- 172.21.15.90:0/1666588744 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v86) v1 ==== 81+0+2434 (secure 0 0 0) 0x7f3c8c01d9a0 con 0x7f3ca0107880 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6826/2080624312 2024-04-25T08:36:30.490015+0000 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6825/1555648344 2024-04-25T08:35:59.006604+0000 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6824/1555648344 2024-04-25T08:35:59.006604+0000 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3702780720 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:41.929 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3635708408 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/903772982 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3314646633 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6826/3029914438 2024-04-25T08:20:28.563578+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3218656850 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3041763742 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3702780720 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3128177159 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3797324612 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6827/2080624312 2024-04-25T08:36:30.490015+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6828/1162352260 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1408697916 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3752867478 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6827/3029914438 2024-04-25T08:20:28.563578+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2244430099 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/927688456 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3222289279 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1367639192 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/2953349914 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1430695580 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3254464007 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3900128298 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1162492184 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/563018427 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6829/1162352260 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3041763742 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:41.931 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/683486625 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3900128298 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/833694895 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/833694895 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/182534323 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/78008447 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2564072908 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3406148041 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1367639192 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2999876850 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:41.932 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/152802986 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:listed 41 entries 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 msgr2=0x7f3c880711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 0x7f3c880711a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c94036040 tx=0x7f3c94000bc0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 msgr2=0x7f3ca01978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c810 tx=0x7f3c8c00ccd0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 shutdown_connections 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f3c8806ecf0 0x7f3c880711a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca0107880 0x7f3ca01978a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 --2- 172.21.15.90:0/1666588744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ca0101870 0x7f3ca0197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.931+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 >> 172.21.15.90:0/1666588744 conn(0x7f3ca00fd370 msgr2=0x7f3ca00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.932+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 shutdown_connections 2024-04-24T08:39:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:41.932+0000 7f3ca7e00640 1 -- 172.21.15.90:0/1666588744 wait complete. 2024-04-24T08:39:41.951 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T08:39:41.951 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T08:39:42.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 -- 172.21.15.90:0/991351978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18700ff4c0 msgr2=0x7f18700ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 --2- 172.21.15.90:0/991351978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18700ff4c0 0x7f18700ff930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f185c009880 tx=0x7f185c02bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 -- 172.21.15.90:0/991351978 shutdown_connections 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 --2- 172.21.15.90:0/991351978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18700ff4c0 0x7f18700ff930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 --2- 172.21.15.90:0/991351978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18701054d0 0x7f18701058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 -- 172.21.15.90:0/991351978 >> 172.21.15.90:0/991351978 conn(0x7f18700fafc0 msgr2=0x7f18700fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 -- 172.21.15.90:0/991351978 shutdown_connections 2024-04-24T08:39:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.242+0000 7f1875000640 1 -- 172.21.15.90:0/991351978 wait complete. 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 Processor -- start 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 -- start start 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 0x7f1870072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870072fe0 con 0x7f18701054d0 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.243+0000 7f1875000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870073150 con 0x7f18700ff4c0 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35154/0 (socket says 172.21.15.90:35154) 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 -- 172.21.15.90:0/3127028759 learned_addr learned my addr 172.21.15.90:0/3127028759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186f400640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 0x7f1870072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 -- 172.21.15.90:0/3127028759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 msgr2=0x7f1870072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 0x7f1870072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 -- 172.21.15.90:0/3127028759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009560 con 0x7f18701054d0 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186f400640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 0x7f1870072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:39:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.244+0000 7f186ea00640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f185c009230 tx=0x7f185c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.245+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f185c0035f0 con 0x7f18701054d0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.245+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f185c003c10 con 0x7f18701054d0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.245+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f185c034440 con 0x7f18701054d0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.245+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1870073370 con 0x7f18701054d0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.245+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18701a1570 con 0x7f18701054d0 2024-04-24T08:39:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.246+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187004e850 con 0x7f18701054d0 2024-04-24T08:39:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.247+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f185c004fd0 con 0x7f18701054d0 2024-04-24T08:39:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.248+0000 7f1867e00640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 0x7f18540711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.248+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f185c0b1260 con 0x7f18701054d0 2024-04-24T08:39:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.248+0000 7f186f400640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 0x7f18540711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.249+0000 7f186f400640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 0x7f18540711a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f18600058c0 tx=0x7f1860005d30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:42.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.253+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f185c07a040 con 0x7f18701054d0 2024-04-24T08:39:42.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:42 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1686640516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:42.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:42 smithi149 ceph-mon[66490]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:42.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:42 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1994136636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:42.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:42 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1666588744' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:39:42.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:42 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1686640516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:42.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:42 smithi090 ceph-mon[85282]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:42.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:42 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1994136636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:42.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:42 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1666588744' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:39:42.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.456+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18701079a0 con 0x7f18701054d0 2024-04-24T08:39:42.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.459+0000 7f1867e00640 1 -- 172.21.15.90:0/3127028759 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f185c07d700 con 0x7f18701054d0 2024-04-24T08:39:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:39:42.460 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"10d437de-0212-11ef-bc93-c7b262605968","created":"2024-04-24T08:10:57.273449+0000","modified":"2024-04-24T08:36:30.490085+0000","last_up_change":"2024-04-24T08:34:51.191821+0000","last_in_change":"2024-04-24T08:18:06.536400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:16:48.618419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T08:20:06.040754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T08:20:06.663119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f46ccda0-8129-4110-8a35-c132eb64e5ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6803","nonce":4221900826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6805","nonce":4221900826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6809","nonce":4221900826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4221900826},{"type":"v1","addr":"172.21.15.90:6807","nonce":4221900826}]},"public_addr":"172.21.15.90:6803/4221900826","cluster_addr":"172.21.15.90:6805/4221900826","heartbeat_back_addr":"172.21.15.90:6809/4221900826","heartbeat_front_addr":"172.21.15.90:6807/4221900826","state":["exists","up"]},{"osd":1,"uuid":"601421cf-b681-483d-93cf-5387d607186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6811","nonce":1858244546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6813","nonce":1858244546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6817","nonce":1858244546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1858244546},{"type":"v1","addr":"172.21.15.90:6815","nonce":1858244546}]},"public_addr":"172.21.15.90:6811/1858244546","cluster_addr":"172.21.15.90:6813/1858244546","heartbeat_back_addr":"172.21.15.90:6817/1858244546","heartbeat_front_addr":"172.21.15.90:6815/1858244546","state":["exists","up"]},{"osd":2,"uuid":"207e3d5c-8b5e-44dd-b55f-e67f20623236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6819","nonce":701071143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6821","nonce":701071143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6825","nonce":701071143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":701071143},{"type":"v1","addr":"172.21.15.90:6823","nonce":701071143}]},"public_addr":"172.21.15.90:6819/701071143","cluster_addr":"172.21.15.90:6821/701071143","heartbeat_back_addr":"172.21.15.90:6825/701071143","heartbeat_front_addr":"172.21.15.90:6823/701071143","state":["exists","up"]},{"osd":3,"uuid":"b060a8df-b343-438f-b840-fd635ac58da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6801","nonce":4083464090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6803","nonce":4083464090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6807","nonce":4083464090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":4083464090},{"type":"v1","addr":"172.21.15.149:6805","nonce":4083464090}]},"public_addr":"172.21.15.149:6801/4083464090","cluster_addr":"172.21.15.149:6803/4083464090","heartbeat_back_addr":"172.21.15.149:6807/4083464090","heartbeat_front_addr":"172.21.15.149:6805/4083464090","state":["exists","up"]},{"osd":4,"uuid":"6abf1d17-2210-4a69-b3d9-bef2daf51ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6809","nonce":3688072354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6811","nonce":3688072354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6815","nonce":3688072354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3688072354},{"type":"v1","addr":"172.21.15.149:6813","nonce":3688072354}]},"public_addr":"172.21.15.149:6809/3688072354","cluster_addr":"172.21.15.149:6811/3688072354","heartbeat_back_addr":"172.21.15.149:6815/3688072354","heartbeat_front_addr":"172.21.15.149:6813/3688072354","state":["exists","up"]},{"osd":5,"uuid":"729bbd1d-e052-4a84-837c-8b19728361f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6817","nonce":1799649429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6819","nonce":1799649429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6823","nonce":1799649429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1799649429},{"type":"v1","addr":"172.21.15.149:6821","nonce":1799649429}]},"public_addr":"172.21.15.149:6817/1799649429","cluster_addr":"172.21.15.149:6819/1799649429","heartbeat_back_addr":"172.21.15.149:6823/1799649429","heartbeat_front_addr":"172.21.15.149:6821/1799649429","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:15:31.796903+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:07.783261+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:16:45.110157+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:19.094185+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:17:53.596177+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:18:30.043129+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6826/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6825/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.149:6824/1555648344":"2024-04-25T08:35:59.006604+0000","172.21.15.90:6800/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3635708408":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/903772982":"2024-04-25T08:29:15.008628+0000","172.21.15.149:0/3314646633":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6826/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/3218656850":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6800/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6801/3702780720":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3128177159":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/3797324612":"2024-04-25T08:12:18.358035+0000","172.21.15.149:6827/2080624312":"2024-04-25T08:36:30.490015+0000","172.21.15.149:6828/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1408697916":"2024-04-25T08:11:37.257159+0000","172.21.15.149:0/3752867478":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6827/3029914438":"2024-04-25T08:20:28.563578+0000","172.21.15.90:0/2244430099":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/927688456":"2024-04-25T08:13:49.914996+0000","172.21.15.149:0/3222289279":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/2953349914":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/1430695580":"2024-04-25T08:12:18.358035+0000","172.21.15.90:0/3254464007":"2024-04-25T08:12:18.358035+0000","172.21.15.90:6800/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:0/1162492184":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/563018427":"2024-04-25T08:23:20.611807+0000","172.21.15.149:6829/1162352260":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3041763742":"2024-04-25T08:12:18.358035+0000","172.21.15.149:0/683486625":"2024-04-25T08:24:20.384564+0000","172.21.15.90:6801/3900128298":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:6801/833694895":"2024-04-25T08:11:37.257159+0000","172.21.15.90:0/182534323":"2024-04-25T08:23:20.611807+0000","172.21.15.149:0/78008447":"2024-04-25T08:24:20.384564+0000","172.21.15.90:0/2564072908":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/3406148041":"2024-04-25T08:29:15.008628+0000","172.21.15.90:6800/1367639192":"2024-04-25T08:23:20.611807+0000","172.21.15.90:0/2999876850":"2024-04-25T08:13:49.914996+0000","172.21.15.90:0/152802986":"2024-04-25T08:29:15.008628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.461+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 msgr2=0x7f18540711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.461+0000 7f1875000640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 0x7f18540711a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f18600058c0 tx=0x7f1860005d30 comp rx=0 tx=0).stop 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.461+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 msgr2=0x7f1870072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.461+0000 7f1875000640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f185c009230 tx=0x7f185c004040 comp rx=0 tx=0).stop 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 shutdown_connections 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f185406ecf0 0x7f18540711a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18701054d0 0x7f1870072aa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 --2- 172.21.15.90:0/3127028759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18700ff4c0 0x7f1870072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 >> 172.21.15.90:0/3127028759 conn(0x7f18700fafc0 msgr2=0x7f18700fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 shutdown_connections 2024-04-24T08:39:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.462+0000 7f1875000640 1 -- 172.21.15.90:0/3127028759 wait complete. 2024-04-24T08:39:42.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:16:48.618419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T08:20:06.040754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T08:20:06.663119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:39:42.480 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:39:42.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 -- 172.21.15.90:0/231573592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81076c0 msgr2=0x7f62c8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 --2- 172.21.15.90:0/231573592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81076c0 0x7f62c8107a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009880 tx=0x7f62b802bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 -- 172.21.15.90:0/231573592 shutdown_connections 2024-04-24T08:39:42.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 --2- 172.21.15.90:0/231573592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81016b0 0x7f62c8101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 --2- 172.21.15.90:0/231573592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81076c0 0x7f62c8107a90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.685+0000 7f62c7e00640 1 -- 172.21.15.90:0/231573592 >> 172.21.15.90:0/231573592 conn(0x7f62c80fd1b0 msgr2=0x7f62c80ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:42.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.686+0000 7f62c7e00640 1 -- 172.21.15.90:0/231573592 shutdown_connections 2024-04-24T08:39:42.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.686+0000 7f62c7e00640 1 -- 172.21.15.90:0/231573592 wait complete. 2024-04-24T08:39:42.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 Processor -- start 2024-04-24T08:39:42.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 -- start start 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81076c0 0x7f62c8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c8197ec0 con 0x7f62c81016b0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c819bc90 con 0x7f62c81076c0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35178/0 (socket says 172.21.15.90:35178) 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.687+0000 7f62c6a00640 1 -- 172.21.15.90:0/1159169663 learned_addr learned my addr 172.21.15.90:0/1159169663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c6000640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81076c0 0x7f62c8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c6a00640 1 -- 172.21.15.90:0/1159169663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81076c0 msgr2=0x7f62c8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c6a00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81076c0 0x7f62c8197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c6a00640 1 -- 172.21.15.90:0/1159169663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b8009560 con 0x7f62c81016b0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c6a00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62b802c220 tx=0x7f62b802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b8004520 con 0x7f62c81016b0 2024-04-24T08:39:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62b8003ab0 con 0x7f62c81016b0 2024-04-24T08:39:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.688+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c819bf10 con 0x7f62c81016b0 2024-04-24T08:39:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.689+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b8034440 con 0x7f62c81016b0 2024-04-24T08:39:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.689+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c819c490 con 0x7f62c81016b0 2024-04-24T08:39:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.690+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c804e000 con 0x7f62c81016b0 2024-04-24T08:39:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.691+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f62b80035f0 con 0x7f62c81016b0 2024-04-24T08:39:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.692+0000 7f62bf400640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 0x7f62ac0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.692+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f62b80b12c0 con 0x7f62c81016b0 2024-04-24T08:39:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.692+0000 7f62c6000640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 0x7f62ac0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.693+0000 7f62c6000640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 0x7f62ac0710e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005fd0 tx=0x7f62b4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.697+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f62b8079f80 con 0x7f62c81016b0 2024-04-24T08:39:42.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.899+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62c80008d0 con 0x7f62c81016b0 2024-04-24T08:39:42.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.899+0000 7f62bf400640 1 -- 172.21.15.90:0/1159169663 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62b807d640 con 0x7f62c81016b0 2024-04-24T08:39:42.900 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T08:39:42.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 msgr2=0x7f62ac0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 0x7f62ac0710e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005fd0 tx=0x7f62b4005850 comp rx=0 tx=0).stop 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 msgr2=0x7f62c81972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62b802c220 tx=0x7f62b802c7a0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 shutdown_connections 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f62ac06ec30 0x7f62ac0710e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81076c0 0x7f62c8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 --2- 172.21.15.90:0/1159169663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c81016b0 0x7f62c81972f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 >> 172.21.15.90:0/1159169663 conn(0x7f62c80fd1b0 msgr2=0x7f62c80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 shutdown_connections 2024-04-24T08:39:42.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:42.902+0000 7f62c7e00640 1 -- 172.21.15.90:0/1159169663 wait complete. 2024-04-24T08:39:42.920 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 -- 172.21.15.90:0/2347786081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601076c0 msgr2=0x7fa560107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 --2- 172.21.15.90:0/2347786081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601076c0 0x7fa560107a90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa550009880 tx=0x7fa55002bce0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 -- 172.21.15.90:0/2347786081 shutdown_connections 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 --2- 172.21.15.90:0/2347786081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5601016b0 0x7fa560101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 --2- 172.21.15.90:0/2347786081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601076c0 0x7fa560107a90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.114+0000 7fa55fe00640 1 -- 172.21.15.90:0/2347786081 >> 172.21.15.90:0/2347786081 conn(0x7fa5600fd1b0 msgr2=0x7fa5600ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.115+0000 7fa55fe00640 1 -- 172.21.15.90:0/2347786081 shutdown_connections 2024-04-24T08:39:43.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.115+0000 7fa55fe00640 1 -- 172.21.15.90:0/2347786081 wait complete. 2024-04-24T08:39:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.116+0000 7fa55fe00640 1 Processor -- start 2024-04-24T08:39:43.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.116+0000 7fa55fe00640 1 -- start start 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.116+0000 7fa55fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.116+0000 7fa55fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5601076c0 0x7fa560197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560197ee0 con 0x7fa5601016b0 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56019bcb0 con 0x7fa5601076c0 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35204/0 (socket says 172.21.15.90:35204) 2024-04-24T08:39:43.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 -- 172.21.15.90:0/3961977514 learned_addr learned my addr 172.21.15.90:0/3961977514 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 -- 172.21.15.90:0/3961977514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5601076c0 msgr2=0x7fa560197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5601076c0 0x7fa560197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 -- 172.21.15.90:0/3961977514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa550009560 con 0x7fa5601016b0 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.117+0000 7fa55ea00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa550009230 tx=0x7fa55002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.118+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5500035f0 con 0x7fa5601016b0 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.118+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa550003c10 con 0x7fa5601016b0 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.118+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56019bf30 con 0x7fa5601016b0 2024-04-24T08:39:43.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.118+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa550005690 con 0x7fa5601016b0 2024-04-24T08:39:43.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.118+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56019c4b0 con 0x7fa5601016b0 2024-04-24T08:39:43.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.119+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56004e000 con 0x7fa5601016b0 2024-04-24T08:39:43.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.120+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa550004ed0 con 0x7fa5601016b0 2024-04-24T08:39:43.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.121+0000 7fa564a00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 0x7fa5440710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.121+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa5500b1260 con 0x7fa5601016b0 2024-04-24T08:39:43.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.121+0000 7fa557e00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 0x7fa5440710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:43.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.121+0000 7fa557e00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 0x7fa5440710e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa548005fd0 tx=0x7fa548005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.125+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa55007a040 con 0x7fa5601016b0 2024-04-24T08:39:43.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:43 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3127028759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:43.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:43 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/1159169663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:43.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:43 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3127028759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:39:43.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:43 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/1159169663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:39:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.328+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fa560198660 con 0x7fa5601016b0 2024-04-24T08:39:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.329+0000 7fa564a00640 1 -- 172.21.15.90:0/3961977514 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa55007d700 con 0x7fa5601016b0 2024-04-24T08:39:43.329 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 msgr2=0x7fa5440710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 0x7fa5440710e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa548005fd0 tx=0x7fa548005850 comp rx=0 tx=0).stop 2024-04-24T08:39:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 msgr2=0x7fa560197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa550009230 tx=0x7fa55002c850 comp rx=0 tx=0).stop 2024-04-24T08:39:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 shutdown_connections 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fa54406ec30 0x7fa5440710e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5601076c0 0x7fa560197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 --2- 172.21.15.90:0/3961977514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5601016b0 0x7fa560197310 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 >> 172.21.15.90:0/3961977514 conn(0x7fa5600fd1b0 msgr2=0x7fa5600feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 shutdown_connections 2024-04-24T08:39:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.332+0000 7fa55fe00640 1 -- 172.21.15.90:0/3961977514 wait complete. 2024-04-24T08:39:43.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T08:39:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 -- 172.21.15.90:0/3301865648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c10bb60 msgr2=0x7fdb0c10bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 --2- 172.21.15.90:0/3301865648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c10bb60 0x7fdb0c10bf30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009c20 tx=0x7fdafc02c1b0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 -- 172.21.15.90:0/3301865648 shutdown_connections 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 --2- 172.21.15.90:0/3301865648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c106090 0x7fdb0c1064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 --2- 172.21.15.90:0/3301865648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c10bb60 0x7fdb0c10bf30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 -- 172.21.15.90:0/3301865648 >> 172.21.15.90:0/3301865648 conn(0x7fdb0c075260 msgr2=0x7fdb0c077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 -- 172.21.15.90:0/3301865648 shutdown_connections 2024-04-24T08:39:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.553+0000 7fdb11200640 1 -- 172.21.15.90:0/3301865648 wait complete. 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.554+0000 7fdb11200640 1 Processor -- start 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.554+0000 7fdb11200640 1 -- start start 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb11200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb11200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10bb60 0x7fdb0c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb11200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19b5e0 con 0x7fdb0c106090 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb11200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19b750 con 0x7fdb0c10bb60 2024-04-24T08:39:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35222/0 (socket says 172.21.15.90:35222) 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 -- 172.21.15.90:0/2994662830 learned_addr learned my addr 172.21.15.90:0/2994662830 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 -- 172.21.15.90:0/2994662830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10bb60 msgr2=0x7fdb0c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10bb60 0x7fdb0c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.555+0000 7fdb0b400640 1 -- 172.21.15.90:0/2994662830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009900 con 0x7fdb0c106090 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb0b400640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdafc037040 tx=0x7fdafc032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdafc034820 con 0x7fdb0c106090 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c19b9d0 con 0x7fdb0c106090 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdafc003e30 con 0x7fdb0c106090 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdafc03f440 con 0x7fdb0c106090 2024-04-24T08:39:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.556+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c19bed0 con 0x7fdb0c106090 2024-04-24T08:39:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.557+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c04e000 con 0x7fdb0c106090 2024-04-24T08:39:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.558+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdafc03b040 con 0x7fdb0c106090 2024-04-24T08:39:43.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.559+0000 7fdb08c00640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 0x7fdaf00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.559+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdafc0b2060 con 0x7fdb0c106090 2024-04-24T08:39:43.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.559+0000 7fdb02a00640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 0x7fdaf00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.560+0000 7fdb02a00640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 0x7fdaf00710e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1059b0 tx=0x7fdaf4005bf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.564+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdafc07ade0 con 0x7fdb0c106090 2024-04-24T08:39:43.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.775+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdb0c10df30 con 0x7fdb0c106090 2024-04-24T08:39:43.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.776+0000 7fdb08c00640 1 -- 172.21.15.90:0/2994662830 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdafc07e4a0 con 0x7fdb0c106090 2024-04-24T08:39:43.776 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T08:39:43.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 msgr2=0x7fdaf00710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 0x7fdaf00710e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1059b0 tx=0x7fdaf4005bf0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 msgr2=0x7fdb0c1a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdafc037040 tx=0x7fdafc032b60 comp rx=0 tx=0).stop 2024-04-24T08:39:43.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 shutdown_connections 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7fdaf006ec30 0x7fdaf00710e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10bb60 0x7fdb0c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.778+0000 7fdb11200640 1 --2- 172.21.15.90:0/2994662830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb0c106090 0x7fdb0c1a1570 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.779+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 >> 172.21.15.90:0/2994662830 conn(0x7fdb0c075260 msgr2=0x7fdb0c076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.779+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 shutdown_connections 2024-04-24T08:39:43.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.779+0000 7fdb11200640 1 -- 172.21.15.90:0/2994662830 wait complete. 2024-04-24T08:39:43.798 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T08:39:43.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 -- 172.21.15.90:0/1518236835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09041054c0 msgr2=0x7f0904105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 --2- 172.21.15.90:0/1518236835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09041054c0 0x7f0904105890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0900009c20 tx=0x7f090002c1b0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 -- 172.21.15.90:0/1518236835 shutdown_connections 2024-04-24T08:39:43.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 --2- 172.21.15.90:0/1518236835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09040ff4b0 0x7f09040ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 --2- 172.21.15.90:0/1518236835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09041054c0 0x7f0904105890 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 -- 172.21.15.90:0/1518236835 >> 172.21.15.90:0/1518236835 conn(0x7f09040fafb0 msgr2=0x7f09040fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:43.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 -- 172.21.15.90:0/1518236835 shutdown_connections 2024-04-24T08:39:43.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.996+0000 7f090b200640 1 -- 172.21.15.90:0/1518236835 wait complete. 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 Processor -- start 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 -- start start 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09041054c0 0x7f0904072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904072fe0 con 0x7f09040ff4b0 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.997+0000 7f090b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904073150 con 0x7f09041054c0 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35242/0 (socket says 172.21.15.90:35242) 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 -- 172.21.15.90:0/2840024111 learned_addr learned my addr 172.21.15.90:0/2840024111 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 -- 172.21.15.90:0/2840024111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09041054c0 msgr2=0x7f0904072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09041054c0 0x7f0904072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:43.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 -- 172.21.15.90:0/2840024111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900009900 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.998+0000 7f0909e00640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0900009430 tx=0x7f0900032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.999+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0900034650 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.999+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0900003e30 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.999+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090003f440 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.999+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904073370 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:43.999+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09041a1600 con 0x7f09040ff4b0 2024-04-24T08:39:44.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.000+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090404e850 con 0x7f09040ff4b0 2024-04-24T08:39:44.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.001+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f090003b040 con 0x7f09040ff4b0 2024-04-24T08:39:44.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.002+0000 7f08faa00640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 0x7f08f0071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:44.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.002+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f09000b1b10 con 0x7f09040ff4b0 2024-04-24T08:39:44.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.002+0000 7f0909400640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 0x7f08f0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:44.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.003+0000 7f0909400640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 0x7f08f0071130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005fd0 tx=0x7f08f4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:44.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.007+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f090007a910 con 0x7f09040ff4b0 2024-04-24T08:39:44.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.221+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f090410c930 con 0x7f09040ff4b0 2024-04-24T08:39:44.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.222+0000 7f08faa00640 1 -- 172.21.15.90:0/2840024111 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v86) v1 ==== 81+0+2434 (secure 0 0 0) 0x7f090007dfd0 con 0x7f09040ff4b0 2024-04-24T08:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6826/2080624312 2024-04-25T08:36:30.490015+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6825/1555648344 2024-04-25T08:35:59.006604+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6824/1555648344 2024-04-25T08:35:59.006604+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3702780720 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3635708408 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/903772982 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3314646633 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6826/3029914438 2024-04-25T08:20:28.563578+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3218656850 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3041763742 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3702780720 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3128177159 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3797324612 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6827/2080624312 2024-04-25T08:36:30.490015+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6828/1162352260 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1408697916 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3752867478 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6827/3029914438 2024-04-25T08:20:28.563578+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2244430099 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/927688456 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/3222289279 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1367639192 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/2953349914 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1430695580 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3254464007 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3900128298 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1162492184 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/563018427 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:6829/1162352260 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3041763742 2024-04-25T08:12:18.358035+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/683486625 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3900128298 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/833694895 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/833694895 2024-04-25T08:11:37.257159+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/182534323 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.149:0/78008447 2024-04-25T08:24:20.384564+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2564072908 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3406148041 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1367639192 2024-04-25T08:23:20.611807+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2999876850 2024-04-25T08:13:49.914996+0000 2024-04-24T08:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/152802986 2024-04-25T08:29:15.008628+0000 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:listed 41 entries 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 msgr2=0x7f08f0071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 0x7f08f0071130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005fd0 tx=0x7f08f4005850 comp rx=0 tx=0).stop 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 msgr2=0x7f0904072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0900009430 tx=0x7f0900032b60 comp rx=0 tx=0).stop 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 shutdown_connections 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:6800/3216343886,v1:172.21.15.90:6801/3216343886] conn(0x7f08f006ec80 0x7f08f0071130 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09041054c0 0x7f0904072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:44.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 --2- 172.21.15.90:0/2840024111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09040ff4b0 0x7f0904072560 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:44.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 >> 172.21.15.90:0/2840024111 conn(0x7f09040fafb0 msgr2=0x7f09040fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:44.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 shutdown_connections 2024-04-24T08:39:44.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T08:39:44.225+0000 7f090b200640 1 -- 172.21.15.90:0/2840024111 wait complete. 2024-04-24T08:39:44.243 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-04-24T08:39:44.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:39:44.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T08:39:44.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:44 smithi149 ceph-mon[66490]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:44.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:44 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/3961977514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:44.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:44 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2994662830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:44.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:44 smithi090 ceph-mon[85282]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:44.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:44 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/3961977514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T08:39:44.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:44 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2994662830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T08:39:44.330 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T08:39:45.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:45 smithi149 ceph-mon[66490]: from='client.? 172.21.15.90:0/2840024111' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:39:45.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:45 smithi090 ceph-mon[85282]: from='client.? 172.21.15.90:0/2840024111' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:39:46.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:46 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:46.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:46 smithi090 ceph-mon[85282]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:46 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:39:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:46 smithi149 ceph-mon[66490]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:47.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:47 smithi090 ceph-mon[85282]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:47 smithi149 ceph-mon[66490]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:49 smithi149 ceph-mon[66490]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:49 smithi090 ceph-mon[85282]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:51 smithi149 ceph-mon[66490]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:51 smithi090 ceph-mon[85282]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:53.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:53 smithi090 ceph-mon[85282]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:53 smithi149 ceph-mon[66490]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:55.746 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:55 smithi090 ceph-mon[85282]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:55 smithi149 ceph-mon[66490]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:57.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:57 smithi149 ceph-mon[66490]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:57.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:57 smithi090 ceph-mon[85282]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:39:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:39:59 smithi149 ceph-mon[66490]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:39:59.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:39:59 smithi090 ceph-mon[85282]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:00 smithi149 ceph-mon[66490]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:40:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:00 smithi090 ceph-mon[85282]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T08:40:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:01 smithi149 ceph-mon[66490]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:01 smithi090 ceph-mon[85282]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:03 smithi090 ceph-mon[85282]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:03 smithi149 ceph-mon[66490]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:05.676 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:05 smithi149 ceph-mon[66490]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:05.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:05 smithi090 ceph-mon[85282]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:07 smithi149 ceph-mon[66490]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:07.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:07 smithi090 ceph-mon[85282]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:09 smithi149 ceph-mon[66490]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:09 smithi090 ceph-mon[85282]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:11 smithi149 ceph-mon[66490]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:11 smithi090 ceph-mon[85282]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:13.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:13 smithi090 ceph-mon[85282]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:13.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:13 smithi149 ceph-mon[66490]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:15.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:16.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:16 smithi149 ceph-mon[66490]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:16 smithi090 ceph-mon[85282]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:17.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:17 smithi149 ceph-mon[66490]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:17.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:17 smithi090 ceph-mon[85282]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:19 smithi149 ceph-mon[66490]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:19 smithi090 ceph-mon[85282]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:21.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:21 smithi149 ceph-mon[66490]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:21 smithi090 ceph-mon[85282]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:23.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:23 smithi090 ceph-mon[85282]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:40:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:23 smithi149 ceph-mon[66490]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:40:25.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:25 smithi149 ceph-mon[66490]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:25.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:25 smithi090 ceph-mon[85282]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:27.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:27 smithi149 ceph-mon[66490]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:27.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:27 smithi090 ceph-mon[85282]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:29.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:29 smithi149 ceph-mon[66490]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:29 smithi090 ceph-mon[85282]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:31 smithi149 ceph-mon[66490]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:31.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:40:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:31 smithi090 ceph-mon[85282]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:40:33.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:33 smithi090 ceph-mon[85282]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:40:33.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:33 smithi149 ceph-mon[66490]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:40:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:35 smithi149 ceph-mon[66490]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:35 smithi090 ceph-mon[85282]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:37.677 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:37 smithi149 ceph-mon[66490]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:37 smithi090 ceph-mon[85282]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:39 smithi149 ceph-mon[66490]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:39 smithi090 ceph-mon[85282]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:41.663 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:41 smithi149 ceph-mon[66490]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:41.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:41 smithi090 ceph-mon[85282]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:43.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:43 smithi090 ceph-mon[85282]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:40:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:43 smithi149 ceph-mon[66490]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:40:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:40:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:46 smithi149 ceph-mon[66490]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:46 smithi090 ceph-mon[85282]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:47 smithi149 ceph-mon[66490]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:47.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:47 smithi090 ceph-mon[85282]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:49.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:49 smithi149 ceph-mon[66490]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:49 smithi090 ceph-mon[85282]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:51 smithi149 ceph-mon[66490]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:51 smithi090 ceph-mon[85282]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:53.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:53 smithi090 ceph-mon[85282]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:53 smithi149 ceph-mon[66490]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:40:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:55 smithi149 ceph-mon[66490]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-24T08:40:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:55 smithi090 ceph-mon[85282]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-24T08:40:57.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:57 smithi149 ceph-mon[66490]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:57.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:57 smithi090 ceph-mon[85282]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:40:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:40:59 smithi149 ceph-mon[66490]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:40:59.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:40:59 smithi090 ceph-mon[85282]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:01 smithi149 ceph-mon[66490]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:01 smithi090 ceph-mon[85282]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:03.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:03 smithi090 ceph-mon[85282]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:03.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:03 smithi149 ceph-mon[66490]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:05 smithi149 ceph-mon[66490]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:05.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:05 smithi090 ceph-mon[85282]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:07 smithi149 ceph-mon[66490]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:07.794 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:07 smithi090 ceph-mon[85282]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:09 smithi149 ceph-mon[66490]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:09 smithi090 ceph-mon[85282]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:11 smithi149 ceph-mon[66490]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:11 smithi090 ceph-mon[85282]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:13.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:13 smithi090 ceph-mon[85282]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:13 smithi149 ceph-mon[66490]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:15.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:16 smithi149 ceph-mon[66490]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:16 smithi090 ceph-mon[85282]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:17.701 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:17 smithi149 ceph-mon[66490]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:17.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:17 smithi090 ceph-mon[85282]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:19 smithi149 ceph-mon[66490]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:19 smithi090 ceph-mon[85282]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:21.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:21 smithi149 ceph-mon[66490]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:21 smithi090 ceph-mon[85282]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:23.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:23 smithi090 ceph-mon[85282]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:23 smithi149 ceph-mon[66490]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:25.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:25 smithi149 ceph-mon[66490]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:25.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:25 smithi090 ceph-mon[85282]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:27.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:27 smithi149 ceph-mon[66490]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:27.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:27 smithi090 ceph-mon[85282]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:29.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:29 smithi149 ceph-mon[66490]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:29 smithi090 ceph-mon[85282]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:31 smithi090 ceph-mon[85282]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:31.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:31 smithi149 ceph-mon[66490]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:32 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:32.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:32 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:33 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:33 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:33 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:41:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:33 smithi149 ceph-mon[66490]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:33.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:33.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:33.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:41:33.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:33 smithi090 ceph-mon[85282]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:35.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:35 smithi149 ceph-mon[66490]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:35.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:35 smithi090 ceph-mon[85282]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:37.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:37 smithi149 ceph-mon[66490]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:37 smithi090 ceph-mon[85282]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:39 smithi149 ceph-mon[66490]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:39 smithi090 ceph-mon[85282]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:41 smithi149 ceph-mon[66490]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:41.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:41 smithi090 ceph-mon[85282]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:43.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:43 smithi090 ceph-mon[85282]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:43 smithi149 ceph-mon[66490]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:41:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:41:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:46 smithi149 ceph-mon[66490]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:46.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:46 smithi090 ceph-mon[85282]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:47 smithi149 ceph-mon[66490]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:47.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:47 smithi090 ceph-mon[85282]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:49 smithi149 ceph-mon[66490]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:49 smithi090 ceph-mon[85282]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:51 smithi149 ceph-mon[66490]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:51 smithi090 ceph-mon[85282]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:41:53.667 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:53 smithi149 ceph-mon[66490]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:41:53.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:53 smithi090 ceph-mon[85282]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:41:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:55 smithi149 ceph-mon[66490]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:55 smithi090 ceph-mon[85282]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:57.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:57 smithi149 ceph-mon[66490]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:57.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:57 smithi090 ceph-mon[85282]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:41:59.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:41:59 smithi149 ceph-mon[66490]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:41:59.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:41:59 smithi090 ceph-mon[85282]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:01.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:01 smithi149 ceph-mon[66490]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:01.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:01 smithi090 ceph-mon[85282]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:03.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:03 smithi090 ceph-mon[85282]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:03.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:03 smithi149 ceph-mon[66490]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:05.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:05 smithi149 ceph-mon[66490]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:05.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:05 smithi090 ceph-mon[85282]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:07.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:07 smithi149 ceph-mon[66490]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:07.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:07 smithi090 ceph-mon[85282]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:09 smithi149 ceph-mon[66490]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:09 smithi090 ceph-mon[85282]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:11.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:11 smithi149 ceph-mon[66490]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:11.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:11 smithi090 ceph-mon[85282]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:13.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:13 smithi090 ceph-mon[85282]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:13.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:13 smithi149 ceph-mon[66490]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:15.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:16.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:16 smithi149 ceph-mon[66490]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:16 smithi090 ceph-mon[85282]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:17.758 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:17 smithi090 ceph-mon[85282]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:17.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:17 smithi149 ceph-mon[66490]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:20.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:20 smithi149 ceph-mon[66490]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:20.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:20 smithi090 ceph-mon[85282]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:22.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:22 smithi149 ceph-mon[66490]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:22.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:22 smithi090 ceph-mon[85282]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:24.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:24 smithi149 ceph-mon[66490]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:24.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:24 smithi090 ceph-mon[85282]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:42:26.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:26 smithi149 ceph-mon[66490]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:26.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:26 smithi090 ceph-mon[85282]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:28.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:28 smithi149 ceph-mon[66490]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:28.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:28 smithi090 ceph-mon[85282]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:30 smithi149 ceph-mon[66490]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:30.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:30 smithi090 ceph-mon[85282]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:32.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:32 smithi149 ceph-mon[66490]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:32.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:32 smithi090 ceph-mon[85282]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:33.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:33 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:33.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:33 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:35.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:35 smithi149 ceph-mon[66490]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:35.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:35.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:35 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:35.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:35 smithi090 ceph-mon[85282]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:35.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:35.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:35 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:36.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:36.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:42:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:37 smithi090 ceph-mon[85282]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:37.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:37 smithi149 ceph-mon[66490]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:39.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:39 smithi090 ceph-mon[85282]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:39.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:39 smithi149 ceph-mon[66490]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:41.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:41 smithi149 ceph-mon[66490]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:41.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:41 smithi090 ceph-mon[85282]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:43.583 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:43 smithi090 ceph-mon[85282]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:43.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:43 smithi149 ceph-mon[66490]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:45 smithi149 ceph-mon[66490]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:45.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:45 smithi090 ceph-mon[85282]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:45.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:42:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:47 smithi149 ceph-mon[66490]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:47.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:47 smithi090 ceph-mon[85282]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:49.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:49 smithi149 ceph-mon[66490]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:49.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:49 smithi090 ceph-mon[85282]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:42:51.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:51 smithi149 ceph-mon[66490]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:51.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:51 smithi090 ceph-mon[85282]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:53.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:53 smithi090 ceph-mon[85282]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:53.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:53 smithi149 ceph-mon[66490]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:55.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:55 smithi149 ceph-mon[66490]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:55.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:55 smithi090 ceph-mon[85282]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:42:56.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:56 smithi149 ceph-mon[66490]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:56.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:56 smithi090 ceph-mon[85282]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:42:58.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:42:58 smithi149 ceph-mon[66490]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:42:58.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:42:58 smithi090 ceph-mon[85282]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:00 smithi149 ceph-mon[66490]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:00.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:00 smithi090 ceph-mon[85282]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:00.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:02.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:02 smithi149 ceph-mon[66490]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:02.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:02 smithi090 ceph-mon[85282]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:04.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:04 smithi090 ceph-mon[85282]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:04.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:04 smithi149 ceph-mon[66490]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:07.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:07 smithi149 ceph-mon[66490]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:07.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:07 smithi090 ceph-mon[85282]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:09.447 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:09 smithi090 ceph-mon[85282]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:09.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:09 smithi149 ceph-mon[66490]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:11.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:11 smithi149 ceph-mon[66490]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:11.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:11 smithi090 ceph-mon[85282]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:13.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:13 smithi149 ceph-mon[66490]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:13.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:13 smithi090 ceph-mon[85282]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:15.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:15 smithi149 ceph-mon[66490]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:15.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:15 smithi090 ceph-mon[85282]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:16.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:16 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:16.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:16 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:17.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:17 smithi149 ceph-mon[66490]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:17.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:17 smithi090 ceph-mon[85282]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:19.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:19 smithi149 ceph-mon[66490]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:19.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:19 smithi090 ceph-mon[85282]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:21.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:21 smithi149 ceph-mon[66490]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:21.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:21 smithi090 ceph-mon[85282]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:23.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:23 smithi149 ceph-mon[66490]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:23.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:23 smithi090 ceph-mon[85282]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:25.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:25 smithi090 ceph-mon[85282]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:25.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:25 smithi149 ceph-mon[66490]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:27.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:27 smithi149 ceph-mon[66490]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:27.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:27 smithi090 ceph-mon[85282]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:29.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:29 smithi149 ceph-mon[66490]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:29.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:29 smithi090 ceph-mon[85282]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:31.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:31 smithi149 ceph-mon[66490]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:31.471 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:31 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:31 smithi090 ceph-mon[85282]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:31.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:31 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:33.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:33 smithi149 ceph-mon[66490]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:33.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:33 smithi090 ceph-mon[85282]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:35.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:35 smithi149 ceph-mon[66490]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:35.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:35 smithi090 ceph-mon[85282]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:36.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:36 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:36.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:36 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:37 smithi149 ceph-mon[66490]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:37.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:43:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:37 smithi090 ceph-mon[85282]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:37.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:43:39.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:39 smithi149 ceph-mon[66490]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:39.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:39 smithi090 ceph-mon[85282]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:41.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:41 smithi149 ceph-mon[66490]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:41.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:41 smithi090 ceph-mon[85282]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:43.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:43 smithi149 ceph-mon[66490]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:43.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:43 smithi090 ceph-mon[85282]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:45 smithi149 ceph-mon[66490]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:45.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:45 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:45 smithi090 ceph-mon[85282]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:43:45.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:45 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:43:47.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:47 smithi149 ceph-mon[66490]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:47.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:47 smithi090 ceph-mon[85282]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:49.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:49 smithi149 ceph-mon[66490]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:49.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:49 smithi090 ceph-mon[85282]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:51.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:51 smithi149 ceph-mon[66490]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:51.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:51 smithi090 ceph-mon[85282]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:53.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:53 smithi149 ceph-mon[66490]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:53.541 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:53 smithi090 ceph-mon[85282]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:43:55.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:55 smithi149 ceph-mon[66490]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:55.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:55 smithi090 ceph-mon[85282]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-24T08:43:57.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:57 smithi149 ceph-mon[66490]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:57.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:57 smithi090 ceph-mon[85282]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:43:59.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:43:59 smithi149 ceph-mon[66490]: pgmap v480: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:43:59.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:43:59 smithi090 ceph-mon[85282]: pgmap v480: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-24T08:44:00.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:00 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:00.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:00 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:01.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:01 smithi149 ceph-mon[66490]: pgmap v481: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:01.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:01 smithi090 ceph-mon[85282]: pgmap v481: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:03.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:03 smithi149 ceph-mon[66490]: pgmap v482: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:03.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:03 smithi090 ceph-mon[85282]: pgmap v482: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:05.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:05 smithi090 ceph-mon[85282]: pgmap v483: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:05.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:05 smithi149 ceph-mon[66490]: pgmap v483: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:07.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:07 smithi090 ceph-mon[85282]: pgmap v484: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:07.578 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:07 smithi149 ceph-mon[66490]: pgmap v484: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:09.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:09 smithi149 ceph-mon[66490]: pgmap v485: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:09.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:09 smithi090 ceph-mon[85282]: pgmap v485: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:11.556 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:11 smithi090 ceph-mon[85282]: pgmap v486: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:11.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:11 smithi149 ceph-mon[66490]: pgmap v486: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:13.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:13 smithi090 ceph-mon[85282]: pgmap v487: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:13.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:13 smithi149 ceph-mon[66490]: pgmap v487: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:15.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:15 smithi149 ceph-mon[66490]: pgmap v488: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:15.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:15 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:15.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:15 smithi090 ceph-mon[85282]: pgmap v488: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:15.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:15 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:17.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:17 smithi149 ceph-mon[66490]: pgmap v489: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:17.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:17 smithi090 ceph-mon[85282]: pgmap v489: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:19.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:19 smithi149 ceph-mon[66490]: pgmap v490: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:19.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:19 smithi090 ceph-mon[85282]: pgmap v490: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:21.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:21 smithi149 ceph-mon[66490]: pgmap v491: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:21.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:21 smithi090 ceph-mon[85282]: pgmap v491: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:23.675 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:23 smithi090 ceph-mon[85282]: pgmap v492: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:23.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:23 smithi149 ceph-mon[66490]: pgmap v492: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:25.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:25 smithi149 ceph-mon[66490]: pgmap v493: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:25.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:25 smithi090 ceph-mon[85282]: pgmap v493: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:27.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:27 smithi149 ceph-mon[66490]: pgmap v494: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:27.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:27 smithi090 ceph-mon[85282]: pgmap v494: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:29.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:29 smithi149 ceph-mon[66490]: pgmap v495: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:29.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:29 smithi090 ceph-mon[85282]: pgmap v495: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:30.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:30 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:30.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:30 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T08:44:31.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:31 smithi149 ceph-mon[66490]: pgmap v496: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:31.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:31 smithi090 ceph-mon[85282]: pgmap v496: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:32.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:32 smithi090 ceph-mon[85282]: pgmap v497: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:32.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:32 smithi149 ceph-mon[66490]: pgmap v497: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:34.777 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:34 smithi149 ceph-mon[66490]: pgmap v498: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:34.793 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:34 smithi090 ceph-mon[85282]: pgmap v498: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-24T08:44:36.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:36 smithi149 ceph-mon[66490]: pgmap v499: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:36.791 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:36 smithi090 ceph-mon[85282]: pgmap v499: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:37.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:37 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:37.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:37 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:38.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:38 smithi149 ceph-mon[66490]: pgmap v500: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:38.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:38 smithi090 ceph-mon[85282]: pgmap v500: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-24T08:44:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:39.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:39 smithi149 ceph-mon[66490]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:44:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:39.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:39 smithi090 ceph-mon[85282]: from='mgr.34100 172.21.15.90:0/230904076' entity='mgr.smithi090.jsrfxo' 2024-04-24T08:44:40.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:40 smithi090 ceph-mon[85282]: pgmap v501: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:41.025 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:40 smithi149 ceph-mon[66490]: pgmap v501: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-24T08:44:43.025 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:42 smithi149 ceph-mon[66490]: pgmap v502: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:44:43.042 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:42 smithi090 ceph-mon[85282]: pgmap v502: 65 pgs: 65 active+clean; 259 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-24T08:44:43.402 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-24T08:44:43.402 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T08:44:43.404 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T08:44:43.419 INFO:tasks.cephadm:Teardown begin 2024-04-24T08:44:43.419 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T08:44:43.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:44:43.515 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:44:43.621 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T08:44:43.621 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T08:44:43.683 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T08:44:43.738 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T08:44:43.738 INFO:tasks.cephadm.mon.smithi090:Stopping mon.smithi090... 2024-04-24T08:44:43.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090 2024-04-24T08:44:44.276 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:43 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:44:44.542 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:44 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[85278]: 2024-04-24T08:44:44.276+0000 7f9cc7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:44:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:44 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090[85278]: 2024-04-24T08:44:44.276+0000 7f9cc7600700 -1 mon.smithi090@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T08:44:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:44 smithi090 podman[132514]: 2024-04-24 08:44:44.38999909 +0000 UTC m=+0.260575447 container stop 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T08:44:44.543 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:44 smithi090 podman[132514]: 2024-04-24 08:44:44.419894148 +0000 UTC m=+0.290470490 container died 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:44:45.031 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:45 smithi090 podman[132514]: 2024-04-24 08:44:45.011422544 +0000 UTC m=+0.881998946 container cleanup 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:44:45.031 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:45 smithi090 bash[132514]: ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090 2024-04-24T08:44:45.292 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 24 08:44:45 smithi090 podman[132538]: 2024-04-24 08:44:45.280060119 +0000 UTC m=+0.869716374 container remove 8bb68ff47c3611bd1709b4863b6b0b100260603ba1fad1b5e608c479baeaa770 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T08:44:45.837 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi090.service' 2024-04-24T08:44:45.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:44:45.969 INFO:tasks.cephadm.mon.smithi090:Stopped mon.smithi090 2024-04-24T08:44:45.969 INFO:tasks.cephadm.mon.smithi149:Stopping mon.smithi149... 2024-04-24T08:44:45.970 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi149 2024-04-24T08:44:46.526 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:46 smithi149 systemd[1]: Stopping Ceph mon.smithi149 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:44:47.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:46 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149[66486]: 2024-04-24T08:44:46.646+0000 7f0ef3a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi149 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:44:47.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:46 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149[66486]: 2024-04-24T08:44:46.646+0000 7f0ef3a00700 -1 mon.smithi149@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T08:44:47.026 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:46 smithi149 podman[104023]: 2024-04-24 08:44:46.895782116 +0000 UTC m=+0.463825693 container died f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T08:44:47.513 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:47 smithi149 podman[104023]: 2024-04-24 08:44:47.505487422 +0000 UTC m=+1.073531023 container cleanup f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:44:47.513 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:47 smithi149 bash[104023]: ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149 2024-04-24T08:44:47.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Apr 24 08:44:47 smithi149 podman[104037]: 2024-04-24 08:44:47.73221092 +0000 UTC m=+0.824275287 container remove f99b1e7df199b2f63e54d5468002e24bc838af8b41f408822e80100081f6aec1 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-mon-smithi149, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:44:48.287 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@mon.smithi149.service' 2024-04-24T08:44:48.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:44:48.422 INFO:tasks.cephadm.mon.smithi149:Stopped mon.smithi149 2024-04-24T08:44:48.422 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T08:44:48.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.0 2024-04-24T08:44:48.792 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:48 smithi090 systemd[1]: Stopping Ceph osd.0 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:44:49.292 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:48 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:44:48.966+0000 7fb4f0400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:44:49.292 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:48 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:44:48.966+0000 7fb4f0400700 -1 osd.0 86 *** Got signal Terminated *** 2024-04-24T08:44:49.292 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:48 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0[97088]: 2024-04-24T08:44:48.966+0000 7fb4f0400700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:44:54.792 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:54 smithi090 podman[132648]: 2024-04-24 08:44:54.510428106 +0000 UTC m=+5.699285584 container died a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:44:55.254 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:55 smithi090 podman[132648]: 2024-04-24 08:44:55.044351556 +0000 UTC m=+6.233209068 container cleanup a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T08:44:55.254 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:55 smithi090 bash[132648]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0 2024-04-24T08:44:55.542 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:55 smithi090 podman[132666]: 2024-04-24 08:44:55.254065791 +0000 UTC m=+0.727772260 container remove a5db84b0ac4ba7828707be9154d9c8408459543306f11dc1469a5b728bafc1a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T08:44:56.336 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:55.954316713 +0000 UTC m=+0.034921356 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:44:56.336 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:56.138666899 +0000 UTC m=+0.219271579 container create 473fe222b982bcfb865722f2b8e13dcf31d34db90fdbad33634fbfebaa47952b (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:44:56.792 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:56.45329715 +0000 UTC m=+0.533901840 container init 473fe222b982bcfb865722f2b8e13dcf31d34db90fdbad33634fbfebaa47952b (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T08:44:56.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:56.464754155 +0000 UTC m=+0.545358842 container start 473fe222b982bcfb865722f2b8e13dcf31d34db90fdbad33634fbfebaa47952b (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:44:56.793 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:56.551543016 +0000 UTC m=+0.632147722 container attach 473fe222b982bcfb865722f2b8e13dcf31d34db90fdbad33634fbfebaa47952b (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T08:44:57.281 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 08:44:56 smithi090 podman[132755]: 2024-04-24 08:44:56.921646333 +0000 UTC m=+1.002251026 container died 473fe222b982bcfb865722f2b8e13dcf31d34db90fdbad33634fbfebaa47952b (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:44:57.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T08:44:57.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:44:57.776 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T08:44:57.776 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T08:44:57.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.1 2024-04-24T08:44:58.292 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:44:57 smithi090 systemd[1]: Stopping Ceph osd.1 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:44:58.792 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:44:58 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:44:58.367+0000 7f5bbf600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:44:58.792 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:44:58 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:44:58.367+0000 7f5bbf600700 -1 osd.1 86 *** Got signal Terminated *** 2024-04-24T08:44:58.792 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:44:58 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1[102644]: 2024-04-24T08:44:58.367+0000 7f5bbf600700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:45:04.042 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:03 smithi090 podman[132860]: 2024-04-24 08:45:03.853152654 +0000 UTC m=+5.627471685 container died 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T08:45:04.542 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:04 smithi090 podman[132860]: 2024-04-24 08:45:04.337633283 +0000 UTC m=+6.111952374 container cleanup 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T08:45:04.542 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:04 smithi090 bash[132860]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1 2024-04-24T08:45:05.041 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:04 smithi090 podman[132879]: 2024-04-24 08:45:04.581055677 +0000 UTC m=+0.713703082 container remove 8bb613c07b4747ec39468953b2c0b674bb12f8dae147ee17482bcd36dc7f73bc (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:45:05.663 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:05 smithi090 podman[132972]: 2024-04-24 08:45:05.259647094 +0000 UTC m=+0.036989902 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:45:05.664 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:05 smithi090 podman[132972]: 2024-04-24 08:45:05.365537871 +0000 UTC m=+0.142880719 container create 98289beaa6c8ae515178466dcdba7509813286650aafb50635926f6b5f71d077 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:45:05.664 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:05 smithi090 podman[132972]: 2024-04-24 08:45:05.663172571 +0000 UTC m=+0.440515425 container init 98289beaa6c8ae515178466dcdba7509813286650aafb50635926f6b5f71d077 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T08:45:06.042 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:05 smithi090 podman[132972]: 2024-04-24 08:45:05.674828322 +0000 UTC m=+0.452171147 container start 98289beaa6c8ae515178466dcdba7509813286650aafb50635926f6b5f71d077 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:45:06.042 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:05 smithi090 podman[132972]: 2024-04-24 08:45:05.753271824 +0000 UTC m=+0.530614691 container attach 98289beaa6c8ae515178466dcdba7509813286650aafb50635926f6b5f71d077 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T08:45:06.521 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 24 08:45:06 smithi090 podman[132972]: 2024-04-24 08:45:06.124232956 +0000 UTC m=+0.901575764 container died 98289beaa6c8ae515178466dcdba7509813286650aafb50635926f6b5f71d077 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:06.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T08:45:07.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:45:07.007 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T08:45:07.008 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T08:45:07.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.2 2024-04-24T08:45:07.542 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:07 smithi090 systemd[1]: Stopping Ceph osd.2 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:45:08.042 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:07 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:45:07.567+0000 7f30f5e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:45:08.042 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:07 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:45:07.567+0000 7f30f5e00700 -1 osd.2 86 *** Got signal Terminated *** 2024-04-24T08:45:08.042 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:07 smithi090 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2[108115]: 2024-04-24T08:45:07.567+0000 7f30f5e00700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:45:13.246 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:12 smithi090 podman[133079]: 2024-04-24 08:45:12.971574055 +0000 UTC m=+5.539005111 container died 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T08:45:13.542 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:13 smithi090 podman[133079]: 2024-04-24 08:45:13.362974525 +0000 UTC m=+5.930405621 container cleanup 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T08:45:13.542 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:13 smithi090 bash[133079]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2 2024-04-24T08:45:14.042 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:13 smithi090 podman[133098]: 2024-04-24 08:45:13.648073294 +0000 UTC m=+0.659742691 container remove 88a827e60f7812d856265d63438fb0a7efab8bfa6d2691f80a3cee91283afd11 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T08:45:14.792 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:14 smithi090 podman[133186]: 2024-04-24 08:45:14.44773536 +0000 UTC m=+0.039967280 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:45:14.792 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:14 smithi090 podman[133186]: 2024-04-24 08:45:14.624311346 +0000 UTC m=+0.216543297 container create 2b9ebc39e176a56419c5a656da88d133bba7042fc80c257f1165d57cce1f5e8a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:15.292 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:14 smithi090 podman[133186]: 2024-04-24 08:45:14.888246285 +0000 UTC m=+0.480478243 container init 2b9ebc39e176a56419c5a656da88d133bba7042fc80c257f1165d57cce1f5e8a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T08:45:15.292 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:14 smithi090 podman[133186]: 2024-04-24 08:45:14.899321652 +0000 UTC m=+0.491553629 container start 2b9ebc39e176a56419c5a656da88d133bba7042fc80c257f1165d57cce1f5e8a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T08:45:15.292 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:14 smithi090 podman[133186]: 2024-04-24 08:45:14.978606827 +0000 UTC m=+0.570838834 container attach 2b9ebc39e176a56419c5a656da88d133bba7042fc80c257f1165d57cce1f5e8a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T08:45:15.664 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 24 08:45:15 smithi090 podman[133186]: 2024-04-24 08:45:15.354618501 +0000 UTC m=+0.946850445 container died 2b9ebc39e176a56419c5a656da88d133bba7042fc80c257f1165d57cce1f5e8a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:45:16.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T08:45:16.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:45:16.134 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T08:45:16.134 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T08:45:16.134 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.3 2024-04-24T08:45:16.525 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:16 smithi149 systemd[1]: Stopping Ceph osd.3 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:45:17.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:16 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:16.722+0000 7f5fda200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:45:17.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:16 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:16.722+0000 7f5fda200700 -1 osd.3 86 *** Got signal Terminated *** 2024-04-24T08:45:17.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:16 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:16.722+0000 7f5fda200700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:45:17.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:17 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:17.191+0000 7f5fd5a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:51.718548+0000 front 2024-04-24T08:44:51.718372+0000 (oldest deadline 2024-04-24T08:45:17.017835+0000) 2024-04-24T08:45:18.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:17 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:17.718+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:18.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:18 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:18.158+0000 7f5fd5a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:51.718548+0000 front 2024-04-24T08:44:51.718372+0000 (oldest deadline 2024-04-24T08:45:17.017835+0000) 2024-04-24T08:45:18.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:18 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:18.340+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:19.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:18 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:18.690+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:19.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:19 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:19.162+0000 7f5fd5a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:51.718548+0000 front 2024-04-24T08:44:51.718372+0000 (oldest deadline 2024-04-24T08:45:17.017835+0000) 2024-04-24T08:45:19.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:19 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:19.321+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:20.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:19 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:19.738+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:20.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:20 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:20.142+0000 7f5fd5a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:51.718548+0000 front 2024-04-24T08:44:51.718372+0000 (oldest deadline 2024-04-24T08:45:17.017835+0000) 2024-04-24T08:45:20.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:20 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:20.330+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:21.025 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:20 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:20.763+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:21.526 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:21 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3[76316]: 2024-04-24T08:45:21.158+0000 7f5fd5a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:51.718548+0000 front 2024-04-24T08:44:51.718372+0000 (oldest deadline 2024-04-24T08:45:17.017835+0000) 2024-04-24T08:45:21.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:21 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:21.297+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:22.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:21 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:21.721+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:22.520 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:22 smithi149 podman[104159]: 2024-04-24 08:45:22.151202252 +0000 UTC m=+5.566350912 container died 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:45:22.520 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:22 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:22.303+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:22.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:22 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:22.757+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:22.776 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:22 smithi149 podman[104159]: 2024-04-24 08:45:22.650616635 +0000 UTC m=+6.065765339 container cleanup 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T08:45:22.776 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:22 smithi149 bash[104159]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3 2024-04-24T08:45:23.276 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:22 smithi149 podman[104179]: 2024-04-24 08:45:22.94357183 +0000 UTC m=+0.755303017 container remove 17e278364e0312ee1baa701c7597f8cd769a9e19e73a68c489f909c55ba58ca0 (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:45:23.771 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:23 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:23.317+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:24.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:23 smithi149 podman[104266]: 2024-04-24 08:45:23.692545225 +0000 UTC m=+0.033335670 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:45:24.026 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:23 smithi149 podman[104266]: 2024-04-24 08:45:23.836460973 +0000 UTC m=+0.177251455 container create 10e56dc6d7a123908c8d4203b356e5f25d85a3bd5025211e7d0b8e70242dfa0a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:24.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:23 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:23.770+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:24.521 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:24 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:24.317+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:24.521 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:24 smithi149 podman[104266]: 2024-04-24 08:45:24.125151657 +0000 UTC m=+0.465942182 container init 10e56dc6d7a123908c8d4203b356e5f25d85a3bd5025211e7d0b8e70242dfa0a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:24.521 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:24 smithi149 podman[104266]: 2024-04-24 08:45:24.137317651 +0000 UTC m=+0.478108163 container start 10e56dc6d7a123908c8d4203b356e5f25d85a3bd5025211e7d0b8e70242dfa0a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T08:45:24.521 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:24 smithi149 podman[104266]: 2024-04-24 08:45:24.222391851 +0000 UTC m=+0.563182383 container attach 10e56dc6d7a123908c8d4203b356e5f25d85a3bd5025211e7d0b8e70242dfa0a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T08:45:24.776 INFO:journalctl@ceph.osd.3.smithi149.stdout:Apr 24 08:45:24 smithi149 podman[104266]: 2024-04-24 08:45:24.60343335 +0000 UTC m=+0.944223892 container died 10e56dc6d7a123908c8d4203b356e5f25d85a3bd5025211e7d0b8e70242dfa0a (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T08:45:25.159 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:24 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:24.813+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:25.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:25 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:25.345+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:25.526 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:25 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:25.345+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:25.616 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T08:45:25.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:45:25.727 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T08:45:25.728 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T08:45:25.728 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.4 2024-04-24T08:45:26.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:25 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:25.859+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:26.277 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:25 smithi149 systemd[1]: Stopping Ceph osd.4 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:45:26.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:26.340+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:26.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:26.340+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:26.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:26.403+0000 7f2b10c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:45:26.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:26.403+0000 7f2b10c00700 -1 osd.4 86 *** Got signal Terminated *** 2024-04-24T08:45:26.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:26.403+0000 7f2b10c00700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:45:27.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:26 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:26.899+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:27.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:27 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:27.376+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:27.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:27 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:27.376+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:28.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:27 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:27.875+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:28.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:27 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:27.875+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:02.734327+0000 front 2024-04-24T08:45:02.734177+0000 (oldest deadline 2024-04-24T08:45:27.433909+0000) 2024-04-24T08:45:28.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:28 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:28.356+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:28.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:28 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:28.356+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:29.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:28 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:28.916+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:29.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:28 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:28.916+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:02.734327+0000 front 2024-04-24T08:45:02.734177+0000 (oldest deadline 2024-04-24T08:45:27.433909+0000) 2024-04-24T08:45:29.617 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:29 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:29.351+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:29.617 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:29 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:29.351+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:30.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:29 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:29.869+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:30.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:29 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:29.869+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:02.734327+0000 front 2024-04-24T08:45:02.734177+0000 (oldest deadline 2024-04-24T08:45:27.433909+0000) 2024-04-24T08:45:30.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:30 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:30.306+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:30.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:30 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:30.306+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:31.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:30 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:30.861+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:50.531473+0000 front 2024-04-24T08:44:50.531274+0000 (oldest deadline 2024-04-24T08:45:16.931975+0000) 2024-04-24T08:45:31.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:30 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4[81484]: 2024-04-24T08:45:30.861+0000 7f2b07400700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:02.734327+0000 front 2024-04-24T08:45:02.734177+0000 (oldest deadline 2024-04-24T08:45:27.433909+0000) 2024-04-24T08:45:31.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:31 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:31.286+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:31.777 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:31 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:31.286+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:32.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:31 smithi149 podman[104371]: 2024-04-24 08:45:31.997158187 +0000 UTC m=+5.768802177 container died 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T08:45:32.734 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:32 smithi149 podman[104371]: 2024-04-24 08:45:32.479622894 +0000 UTC m=+6.251266913 container cleanup 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T08:45:32.734 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:32 smithi149 bash[104371]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4 2024-04-24T08:45:32.734 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:32 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:32.330+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:32.734 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:32 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:32.330+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:33.026 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:32 smithi149 podman[104390]: 2024-04-24 08:45:32.733710738 +0000 UTC m=+0.721881461 container remove 3298f76997687049f05ca41be9e13ff1e4b45cf9dcda2e91ddf0fc3ff4fd8dac (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:33.606 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:33 smithi149 podman[104479]: 2024-04-24 08:45:33.427451118 +0000 UTC m=+0.036605569 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:45:33.606 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:33 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:33.324+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:33.606 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:33 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:33.324+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:33.927 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:33 smithi149 podman[104479]: 2024-04-24 08:45:33.606412761 +0000 UTC m=+0.215567251 container create ea244c537e8bfdbab5160292dfe2a6dde0356ebcee24efcedb9e500d8c1b17cd (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T08:45:33.927 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:33 smithi149 podman[104479]: 2024-04-24 08:45:33.926951754 +0000 UTC m=+0.536106228 container init ea244c537e8bfdbab5160292dfe2a6dde0356ebcee24efcedb9e500d8c1b17cd (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:45:34.275 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:33 smithi149 podman[104479]: 2024-04-24 08:45:33.938857241 +0000 UTC m=+0.548011680 container start ea244c537e8bfdbab5160292dfe2a6dde0356ebcee24efcedb9e500d8c1b17cd (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:45:34.276 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:34 smithi149 podman[104479]: 2024-04-24 08:45:34.033613641 +0000 UTC m=+0.642768118 container attach ea244c537e8bfdbab5160292dfe2a6dde0356ebcee24efcedb9e500d8c1b17cd (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T08:45:34.713 INFO:journalctl@ceph.osd.4.smithi149.stdout:Apr 24 08:45:34 smithi149 podman[104479]: 2024-04-24 08:45:34.387011101 +0000 UTC m=+0.996165574 container died ea244c537e8bfdbab5160292dfe2a6dde0356ebcee24efcedb9e500d8c1b17cd (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T08:45:34.713 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:34 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:34.305+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:34.713 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:34 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:34.305+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:35.246 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T08:45:35.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:45:35.364 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T08:45:35.365 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T08:45:35.365 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.5 2024-04-24T08:45:35.481 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:35.321+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:35.481 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:35.321+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:35.775 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 systemd[1]: Stopping Ceph osd.5 for 10d437de-0212-11ef-bc93-c7b262605968... 2024-04-24T08:45:36.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:35.972+0000 7fc3f0400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:45:36.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:35.972+0000 7fc3f0400700 -1 osd.5 86 *** Got signal Terminated *** 2024-04-24T08:45:36.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:35 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:35.972+0000 7fc3f0400700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T08:45:36.775 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:36 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:36.302+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:36.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:36 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:36.302+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:37.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:37 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:37.320+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:37.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:37 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:37.320+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:37.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:37 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:37.320+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-04-24T08:45:10.449032+0000 front 2024-04-24T08:45:10.449138+0000 (oldest deadline 2024-04-24T08:45:36.348720+0000) 2024-04-24T08:45:38.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:38 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:38.346+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:38.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:38 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:38.346+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:38.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:38 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:38.346+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-04-24T08:45:10.449032+0000 front 2024-04-24T08:45:10.449138+0000 (oldest deadline 2024-04-24T08:45:36.348720+0000) 2024-04-24T08:45:39.678 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:39 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:39.366+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:39.678 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:39 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:39.366+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:39.678 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:39 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:39.366+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-04-24T08:45:10.449032+0000 front 2024-04-24T08:45:10.449138+0000 (oldest deadline 2024-04-24T08:45:36.348720+0000) 2024-04-24T08:45:40.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:40 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:40.343+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T08:44:48.843859+0000 front 2024-04-24T08:44:48.843932+0000 (oldest deadline 2024-04-24T08:45:17.645181+0000) 2024-04-24T08:45:40.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:40 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:40.343+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-24T08:45:01.147361+0000 front 2024-04-24T08:45:01.147287+0000 (oldest deadline 2024-04-24T08:45:24.646829+0000) 2024-04-24T08:45:40.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:40 smithi149 ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5[86370]: 2024-04-24T08:45:40.343+0000 7fc3ebc00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-04-24T08:45:10.449032+0000 front 2024-04-24T08:45:10.449138+0000 (oldest deadline 2024-04-24T08:45:36.348720+0000) 2024-04-24T08:45:41.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:41 smithi149 podman[104583]: 2024-04-24 08:45:41.50175833 +0000 UTC m=+5.665523976 container died ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T08:45:42.199 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:41 smithi149 podman[104583]: 2024-04-24 08:45:41.964866741 +0000 UTC m=+6.128632382 container cleanup ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T08:45:42.200 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:41 smithi149 bash[104583]: ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5 2024-04-24T08:45:42.525 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:42 smithi149 podman[104611]: 2024-04-24 08:45:42.199804619 +0000 UTC m=+0.686336834 container remove ecd6e5e644e3d625aa2857e93124ecb62dd4eaaeaa80d65f44ec9e46659f6d9d (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T08:45:43.431 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:42.959384798 +0000 UTC m=+0.033954786 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:45:43.431 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:43.108643393 +0000 UTC m=+0.183213426 container create 0ad40d630cece6c6061db625ba731fec0a7ad5bb8e81a25a07fe5a7e5ff242ce (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T08:45:43.431 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:43.430617746 +0000 UTC m=+0.505187786 container init 0ad40d630cece6c6061db625ba731fec0a7ad5bb8e81a25a07fe5a7e5ff242ce (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:45:43.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:43.445701095 +0000 UTC m=+0.520271077 container start 0ad40d630cece6c6061db625ba731fec0a7ad5bb8e81a25a07fe5a7e5ff242ce (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T08:45:43.776 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:43.519776556 +0000 UTC m=+0.594346605 container attach 0ad40d630cece6c6061db625ba731fec0a7ad5bb8e81a25a07fe5a7e5ff242ce (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T08:45:44.276 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 08:45:43 smithi149 podman[104700]: 2024-04-24 08:45:43.907954401 +0000 UTC m=+0.982524405 container died 0ad40d630cece6c6061db625ba731fec0a7ad5bb8e81a25a07fe5a7e5ff242ce (image=quay.ceph.io/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-10d437de-0212-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T08:45:44.712 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10d437de-0212-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T08:45:44.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T08:45:44.828 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T08:45:44.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10d437de-0212-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T08:45:45.102 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:45:52.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[63005]: fuse finished with error 0 and tester_r 0 2024-04-24T08:46:16.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10d437de-0212-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T08:46:16.374 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:46:37.465 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:46:37.555 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:46:37.641 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T08:46:37.643 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi090/crash 2024-04-24T08:46:37.643 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash -- . 2024-04-24T08:46:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T08:46:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-04-24T08:46:37.733 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi149/crash 2024-04-24T08:46:37.734 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash -- . 2024-04-24T08:46:37.820 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/10d437de-0212-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T08:46:37.820 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-04-24T08:46:37.823 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T08:46:37.824 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-24T08:46:37.944 INFO:tasks.cephadm:Compressing logs... 2024-04-24T08:46:37.945 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T08:46:37.949 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T08:46:38.004 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T08:46:38.004 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-24T08:46:38.005 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T08:46:38.005 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log 2024-04-24T08:46:38.007 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T08:46:38.008 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T08:46:38.010 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi090.log 2024-04-24T08:46:38.011 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T08:46:38.012 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi090.log: 88.6% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T08:46:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T08:46:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T08:46:38.017 INFO:teuthology.orchestra.run.smithi090.stderr: 91.4% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T08:46:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.qbyvpv.log 2024-04-24T08:46:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.lumztb.log 2024-04-24T08:46:38.020 INFO:teuthology.orchestra.run.smithi090.stderr: 85.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T08:46:38.029 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.qbyvpv.log: /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.lumztb.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T08:46:38.033 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi090.jsrfxo.log 2024-04-24T08:46:38.037 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T08:46:38.041 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi090.jsrfxo.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log 2024-04-24T08:46:38.043 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T08:46:38.043 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-24T08:46:38.045 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T08:46:38.046 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log 2024-04-24T08:46:38.047 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T08:46:38.050 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi149.vavftl.log 2024-04-24T08:46:38.050 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi149.log 2024-04-24T08:46:38.051 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi149.vavftl.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T08:46:38.051 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi149.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T08:46:38.052 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.eazcmc.log 2024-04-24T08:46:38.053 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T08:46:38.054 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.eazcmc.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.zwjtmw.log 2024-04-24T08:46:38.055 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T08:46:38.056 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.zwjtmw.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi149.log 2024-04-24T08:46:38.057 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log: 94.3% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log.gz 2024-04-24T08:46:38.057 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T08:46:38.060 INFO:teuthology.orchestra.run.smithi090.stderr: 91.4% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T08:46:38.060 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi149.log: /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log: 94.2% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi149.log.gz 2024-04-24T08:46:38.184 INFO:teuthology.orchestra.run.smithi149.stderr: 85.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T08:46:38.215 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T08:46:38.233 INFO:teuthology.orchestra.run.smithi149.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T08:46:38.295 INFO:teuthology.orchestra.run.smithi149.stderr: 91.3% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T08:46:38.417 INFO:teuthology.orchestra.run.smithi149.stderr: 88.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T08:46:38.673 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi090.jsrfxo.log.gz 2024-04-24T08:46:38.872 INFO:teuthology.orchestra.run.smithi149.stderr: 89.2% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mgr.smithi149.vavftl.log.gz 2024-04-24T08:46:39.694 INFO:teuthology.orchestra.run.smithi090.stderr: 90.3% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi090.log.gz 2024-04-24T08:46:40.264 INFO:teuthology.orchestra.run.smithi149.stderr: 92.5% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mon.smithi149.log.gz 2024-04-24T08:46:42.584 INFO:teuthology.orchestra.run.smithi090.stderr: 93.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T08:46:44.825 INFO:teuthology.orchestra.run.smithi090.stderr: 93.6% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T08:46:45.285 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T08:46:45.978 INFO:teuthology.orchestra.run.smithi149.stderr: 94.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.zwjtmw.log.gz 2024-04-24T08:46:46.039 INFO:teuthology.orchestra.run.smithi090.stderr: 94.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.qbyvpv.log.gz 2024-04-24T08:46:52.070 INFO:teuthology.orchestra.run.smithi149.stderr: 93.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T08:46:52.469 INFO:teuthology.orchestra.run.smithi090.stderr: 94.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.lumztb.log.gz 2024-04-24T08:46:53.036 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-24T08:46:53.037 INFO:teuthology.orchestra.run.smithi149.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-24T08:46:54.516 INFO:teuthology.orchestra.run.smithi090.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T08:46:54.520 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-24T08:46:54.520 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m16.540s 2024-04-24T08:46:54.521 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m54.963s 2024-04-24T08:46:54.521 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m4.353s 2024-04-24T08:46:56.301 INFO:teuthology.orchestra.run.smithi149.stderr: 93.8% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T08:46:57.852 INFO:teuthology.orchestra.run.smithi149.stderr: 93.7% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T08:48:29.526 INFO:teuthology.orchestra.run.smithi149.stderr: 92.8% -- replaced with /var/log/ceph/10d437de-0212-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi149.eazcmc.log.gz 2024-04-24T08:48:29.530 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-04-24T08:48:29.530 INFO:teuthology.orchestra.run.smithi149.stderr:real 1m51.527s 2024-04-24T08:48:29.530 INFO:teuthology.orchestra.run.smithi149.stderr:user 2m4.927s 2024-04-24T08:48:29.530 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m12.093s 2024-04-24T08:48:29.531 INFO:tasks.cephadm:Archiving logs... 2024-04-24T08:48:29.532 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi090/log 2024-04-24T08:48:29.533 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T08:48:36.496 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi149/log 2024-04-24T08:48:36.497 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T08:48:55.448 INFO:tasks.cephadm:Removing cluster... 2024-04-24T08:48:55.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10d437de-0212-11ef-bc93-c7b262605968 --force 2024-04-24T08:48:55.723 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:48:56.460 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10d437de-0212-11ef-bc93-c7b262605968 --force 2024-04-24T08:48:56.866 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 10d437de-0212-11ef-bc93-c7b262605968 2024-04-24T08:48:57.885 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T08:48:57.885 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T08:48:57.941 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T08:48:58.001 INFO:tasks.cephadm:Teardown complete 2024-04-24T08:48:58.001 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T08:48:58.014 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T08:48:58.015 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T08:48:58.015 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T08:48:58.022 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T08:48:58.175 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T08:48:58.186 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T08:48:58.186 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:48:58.194 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:48:58.241 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:48:58.248 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:^- hv01.front.sepia.ceph.com 2 8 377 87 +1272us[+1376us] +/- 569ms 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 7 377 82 -386us[ -282us] +/- 67ms 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +549us[ +549us] +/- 63ms 2024-04-24T08:48:58.250 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.359 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:48:58.359 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-04-24T08:48:58.360 INFO:teuthology.orchestra.run.smithi149.stdout:^- hv01.front.sepia.ceph.com 2 6 377 16 +1156us[+1156us] +/- 570ms 2024-04-24T08:48:58.360 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -523us[ -546us] +/- 68ms 2024-04-24T08:48:58.360 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +443us[ +443us] +/- 63ms 2024-04-24T08:48:58.360 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.364 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T08:48:58.376 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T08:48:58.377 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T08:48:58.388 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T08:48:58.586 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T08:48:58.797 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:48:58.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:48:58.894 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:48:58.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:48:58.989 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T08:48:59.001 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T08:48:59.014 INFO:teuthology.task.internal:Duration was 2794.627695 seconds 2024-04-24T08:48:59.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T08:48:59.026 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T08:48:59.026 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T08:48:59.034 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T08:48:59.159 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:48:59.203 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:48:59.635 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T08:48:59.636 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:48:59.636 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T08:48:59.699 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-04-24T08:48:59.699 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T08:48:59.774 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T08:48:59.775 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T08:48:59.782 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T08:48:59.864 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T08:48:59.865 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T08:48:59.869 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T08:49:00.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T08:49:00.673 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T08:49:00.673 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T08:49:00.752 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T08:49:00.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T08:49:00.881 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T08:49:00.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T08:49:00.981 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-04-24T08:49:01.006 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-04-24T08:49:01.061 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T08:49:01.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:49:01.115 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T08:49:01.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:49:01.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T08:49:01.183 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T08:49:01.185 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi090 2024-04-24T08:49:01.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T08:49:01.297 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212/remote/smithi149 2024-04-24T08:49:01.297 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T08:49:01.401 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T08:49:01.401 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T08:49:01.410 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T08:49:01.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T08:49:01.478 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T08:49:01.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T08:49:01.489 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T08:49:01.490 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T08:49:01.497 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T08:49:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: 393229 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 24 08:49 /home/ubuntu/cephtest 2024-04-24T08:49:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: 681354 4 d--------- 2 ubuntu ubuntu 4096 Apr 24 08:20 /home/ubuntu/cephtest/mnt.0 2024-04-24T08:49:01.547 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-24T08:49:01.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:49:01.550 INFO:teuthology.orchestra.run.smithi090.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-24T08:49:01.550 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-24T08:49:01.551 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T08:49:01.563 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T08:49:01.576 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T08:49:01.576 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T08:49:01.600 DEBUG:teuthology.nuke:shortname: smithi090 2024-04-24T08:49:01.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T08:49:01.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 07:51:58.796811', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJm27NWkUvn7FQKvtuY2zOrYZnGYdgvsb5QaGmZ9d01nF2rRjsV4+2BTYnsmMe64icBRQBzRTVRW62YR2f4blA='} 2024-04-24T08:49:01.627 DEBUG:teuthology.nuke:shortname: smithi149 2024-04-24T08:49:01.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T08:49:01.651 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671212', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 07:51:58.798781', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCIplYqbcIpOMlShY3VrL4gBfN3by1EWstzcyolU16HgJ9GbAurPtVbxOL15MErBMhmoon1wYFD3LnY0zhrPdh4='} 2024-04-24T08:49:01.682 INFO:teuthology.orchestra.console.smithi090:Power off 2024-04-24T08:49:01.682 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T08:49:01.705 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-04-24T08:49:01.705 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:01.720 INFO:teuthology.orchestra.console.smithi149:Power off 2024-04-24T08:49:01.721 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T08:49:01.731 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T08:49:01.747 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-04-24T08:49:01.747 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:01.771 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-04-24T08:49:05.732 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:05.772 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:05.843 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T08:49:05.881 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-04-24T08:49:09.848 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:09.882 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:09.958 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T08:49:09.991 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-04-24T08:49:13.964 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:13.992 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T08:49:14.074 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-24T08:49:14.100 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-04-24T08:49:14.175 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-04-24T08:49:14.201 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-04-24T08:49:14.306 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2794.6276953220367 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 host.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology status: fail success: false 2024-04-24T08:49:14.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T08:49:14.384 INFO:teuthology.run:FAIL